Add basic nextcloud deployment

Uses php-fpm image and is served through a simple caddy server.
Automatically deploys by default and can be automatically deployed with
smtp e-mail sending and s3 primary object storage optionally if desired.

Utilizes some necessary hackery for container ordering and startup so
startup is relatively slow (takes around 2-5 minutes at least) but once
running should be stable and uninterrupted.

Implements health-checks for all involved containers.

Switch apache for php-fpm image

group_vars/testing.yml

@@ -12,5 +12,6 @@ miniflux_use_https: no
 searx_use_https: no
 traggo_use_https: no
 monica_use_https: no
+nextcloud_use_https: no
 
 #server_domain: mytest.com

roles/caddy/templates/docker-stack.yml.j2

@@ -4,6 +4,12 @@ services:
   app:
     image: caddy:{{ caddy_version }}
     command: caddy run --config /etc/caddy/config.json
+    healthcheck:
+        test: ["CMD", "wget", "--quiet", "--spider", "--tries=1", "http://localhost:2019/metrics"]
+        interval: 1m
+        timeout: 10s
+        retries: 3
+        start_period: 1m
     ports:
       - "80:80"
       - "443:443"

roles/nextcloud/README.md

@@ -0,0 +1,136 @@
+# Nextcloud
+
+A full office suite and groupware proposition,
+though its main draw for most is the file synchronization abilities.
+AKA Dropbox replacement.
+
+This software can grow enormous and enormously complicated, 
+this Ansible setup role concentrates on 3 things:
+* a stable and secure base setup from the official docker container
+* automatic setup of an email pipeline so users can reset passwords and be updated of changes
+* the ability to use S3 object storage as the primary way of storing users' files
+
+The rest should be taken care of either automatically, 
+or supplied after the fact (if using different plugins or similar).
+
+## Defaults
+
+```yml
+nextcloud_upstream_file_dir: "{{ docker_stack_files_dir }}/{{ stack_name }}"
+```
+
+The on-target directory where the proxy configuration files should be stashed.
+
+```yml
+nextcloud_use_https: true
+```
+
+Whether the service should be reachable through http (port 80) or through https (port 443) and provision an https certificate. Usually you will want this to stay `true` if facing the public internet.
+
+```yml
+nextcloud_version: fpm
+nextcloud_db_version: 12
+```
+
+The docker image version to be used in stack creation. 
+The role sets up the `php-fpm` version of the official Nextcloud image.
+That means, Caddy is used in front as the server which presents all pages
+and access to files, the Nextcloud image itself only serves as the PHP data store.
+
+If changing the version to one relying on Nextcloud's in-built Apache server,
+take care to change where the upstream proxy is pointing to since the Caddy server in front loses its meaning.
+
+The second variable points to the docker image that should be used for the PostgreSQL database,
+with 12 pre-filled as default. 
+You can put this to latest, but should take care to migrate the database correctly when an update rolls around,
+or it *will* destroy your data at some point.
+Generally, it seems easier to pin this to a specific version and then only update manually.
+
+```yml
+subdomain_alias: files
+```
+
+If the deployed container should be served over a uri that is not the stack name. 
+By default, it will be set to `files.yourdomain.com`  - 
+if this option is not set it will be served on `nextcloud.yourdomain.com` instead.
+If you change or delete this, you should also change what `nextcloud_trusted_domains` points to.
+
+## Basic setup
+
+```yml
+nextcloud_app_admin_username: mynextcloudusername
+nextcloud_app_admin_password: mynextcloudpassword
+nextcloud_redis_password: myredispass
+nextcloud_db_username: nextcloud
+nextcloud_db_password: secretnextcloud
+```
+
+Sets the default username and password for application and database.
+All of these variables are necessary to circumvent the manual installation process 
+you would usually be faced with on first creating a Nextcloud instance.
+Ideally change all of these for your personal setup,
+but it is especially important to change the app admin login data since they are what is public facing.
+
+```yml
+nextcloud_trusted_domains: "{{ subdomain_alias }}.{{ server_domain }}"
+```
+
+The domains that are allowed to access your Nextcloud instance.
+Should point to any domains that you want it accessible on,
+can be a space-separated list of them. 
+Take care to include the sub-domain if your are accessing it through one of them.
+[Further explanation](https://blog.martyoeh.me/posts/2021-11-18-nextcloud-trusted-domains/).
+
+## E-Mail setup
+
+```yml
+nextcloud_smtp_host: smtp.mailgun.org (no default)
+nextcloud_smtp_secure: ssl
+nextcloud_smtp_port: 465
+nextcloud_smtp_authtype: LOGIN
+nextcloud_smtp_username: <smtp-username> (no default)
+nextcloud_smtp_password: <smtp-password> (no default)
+nextcloud_smtp_from_address: noreply
+nextcloud_smtp_from_domain: "{{ server_domain }}"
+```
+
+To set up e-mail routing you will need to provide your smtp details here.
+The three lines absolutely necessary to fill in are:
+
+```yml
+nextcloud_smtp_host: smtp.mailgun.org (no default)
+nextcloud_smtp_username: <smtp-username> (no default)
+nextcloud_smtp_password: <smtp-password> (no default)
+```
+
+Since they carry no default, you will have to supply your own details here.
+If the default settings of the other variables work for your provider,
+e-mail sending will automatically be set up in your Nextcloud instance
+(as for e.g. mailgun)
+otherwise change those accordingly as well.
+
+## Primary S3 object storage
+
+```yml
+nextcloud_s3_host: s3.eu-central-1.wasabisys.com (no default)
+nextcloud_s3_bucket: nextcloud (no default)
+nextcloud_s3_key: <s3-key> (no default)
+nextcloud_s3_secret: <s3-secret> (no default)
+nextcloud_s3_port: 443 (no default)
+nextcloud_s3_ssl: true (no default)
+nextcloud_s3_region: eu-central-1 (no default)
+nextcloud_s3_usepath_style: true (no default)
+```
+
+To set up an object storage as primary file storage you will need to provide your S3-compatible details here.
+All lines are necessary to fill out correctly to enable S3.
+Since they carry no default, you will need to supply your own details for each variable.
+
+If your details are correct, Nextcloud should automatically set up S3 as its primary object storage.
+
+Be careful if you switch an existing data volume of the Nextcloud image to S3
+as you will lose all access to existing files.
+The files *should* not be deleted at this point,
+only access will be lost,
+but you are playing with fire at this point.
+

roles/nextcloud/defaults/main.yml

@@ -0,0 +1,44 @@
+---
+
+# set preferred application version
+nextcloud_version: fpm-alpine
+# set preferred postgres version
+nextcloud_db_version: 12-alpine
+
+nextcloud_upstream_file_dir: "{{ docker_stack_files_dir }}/{{ stack_name }}"
+
+nextcloud_use_https: true
+
+# the subdomain link nextcloud will be reachable under
+subdomain_alias: files
+
+# the following block is required for a basic setup
+nextcloud_app_admin_username: mynextcloudusername
+nextcloud_app_admin_password: mynextcloudpassword
+nextcloud_redis_password: myredispass
+nextcloud_db_username: nextcloud
+nextcloud_db_password: secretnextcloud
+
+# if you wish to access your nextcloud instance from the reverse proxy
+nextcloud_trusted_domains: "{{ subdomain_alias }}.{{ server_domain }}"
+
+# the following block is required *fully* for working smtp
+# nextcloud_smtp_host: smtp.eu.mailgun.org
+nextcloud_smtp_secure: ssl
+nextcloud_smtp_port: 465
+nextcloud_smtp_authtype: LOGIN
+# nextcloud_smtp_username: <smtp-username>
+# nextcloud_smtp_password: <smtp-password>
+nextcloud_smtp_from_address: noreply
+nextcloud_smtp_from_domain: "{{ server_domain }}"
+
+# the following block is required *fully* for primary object storage
+# nextcloud_s3_host: s3.eu-central-1.wasabisys.com
+# nextcloud_s3_bucket: nextcloud
+# nextcloud_s3_key: <s3-key>
+# nextcloud_s3_secret: <s3-secret>
+# nextcloud_s3_port: 443
+# nextcloud_s3_ssl: true
+# nextcloud_s3_region: eu-central-1
+# nextcloud_s3_usepath_style: true
+

roles/nextcloud/files/Caddyfile

@@ -0,0 +1,32 @@
+:80 {
+	root * /var/www/html
+	file_server
+
+	php_fastcgi app:9000
+	header {
+		# enable HSTS
+		# Strict-Transport-Security max-age=31536000;
+	}
+
+	redir /.well-known/carddav /remote.php/dav 301
+	redir /.well-known/caldav /remote.php/dav 301
+
+	# .htaccess / data / config / ... shouldn't be accessible from outside
+	@forbidden {
+		path /.htaccess
+		path /data/*
+		path /config/*
+		path /db_structure
+		path /.xml
+		path /README
+		path /3rdparty/*
+		path /lib/*
+		path /templates/*
+		path /occ
+		path /console.php
+	}
+
+	respond @forbidden 404
+
+}
+

roles/nextcloud/handlers/main.yml

@@ -0,0 +1,53 @@
+## Register reverse proxy
+- name: Ensure upstream directory exists
+  ansible.builtin.file:
+    path: "{{ nextcloud_upstream_file_dir }}"
+    state: directory
+    mode: '0755'
+  become: yes
+  listen: "update nextcloud upstream"
+
+- name: Update upstream template
+  ansible.builtin.template:
+    src: upstream.json.j2
+    dest: "{{ nextcloud_upstream_file_dir }}/upstream.json"
+  become: yes
+  listen: "update nextcloud upstream"
+
+# figure out if upstream id exists
+- name: check {{ stack_name }} upstream
+  community.docker.docker_container_exec:
+    container: "{{ caddy_container_id }}"
+    command: >
+      curl localhost:2019/id/{{ stack_name }}_upstream/
+  changed_when: False
+  register: result
+  become: yes
+  listen: "update nextcloud upstream"
+
+# upstream already exists, patch it
+- name: remove old {{ stack_name }} upstream
+  community.docker.docker_container_exec:
+    container: "{{ caddy_container_id }}"
+    command: >
+      curl -X DELETE localhost:2019/id/{{ stack_name }}_upstream/
+  become: yes
+  when: (result.stdout | from_json)['error'] is not defined
+  listen: "update nextcloud upstream"
+
+# upstream has to be created
+- name: add {{ stack_name }} upstream
+  community.docker.docker_container_exec:
+    container: "{{ caddy_container_id }}"
+    command: >
+      curl -X POST -H "Content-Type: application/json" -d @{{ nextcloud_upstream_file_dir }}/upstream.json localhost:2019/config/apps/http/servers/{{ (nextcloud_use_https == True) | ternary(caddy_https_server_name, caddy_http_server_name) }}/routes/0/ 
+  become: yes
+  listen: "update nextcloud upstream"
+
+- name: Ensure upstream directory is gone again
+  ansible.builtin.file:
+    path: "{{ nextcloud_upstream_file_dir }}"
+    state: absent
+  become: yes
+  listen: "update nextcloud upstream"
+

roles/nextcloud/meta/main.yml

@@ -0,0 +1,14 @@
+---
+
+galaxy_info:
+  author: Marty Oehme
+  description: Installs nextcloud as a docker stack service
+  license: GPL-3.0-only
+  min_ansible_version: 2.9
+  galaxy_tags: []
+
+
+dependencies:
+  - docker
+  - docker-swarm
+  - caddy

roles/nextcloud/tasks/main.yml

@@ -0,0 +1,39 @@
+---
+## install container
+- name: Check upstream status
+  community.docker.docker_container_exec:
+    container: "{{ caddy_container_id }}"
+    command: >
+      curl localhost:2019/id/{{ stack_name }}_upstream/
+  register: result
+  changed_when: (result.stdout | from_json) != (lookup('template', 'upstream.json.j2') | from_yaml)
+  become: yes
+  notify: "update nextcloud upstream"
+
+- name: Ensure target directory exists
+  ansible.builtin.file:
+    path: "{{ nextcloud_upstream_file_dir }}"
+    state: directory
+    mode: '0755'
+  become: yes
+  notify: "update nextcloud upstream"
+
+- name: Move webserver Caddyfile to target dir
+  ansible.builtin.copy:
+    src: "Caddyfile"
+    dest: "{{ nextcloud_upstream_file_dir }}/Caddyfile"
+  become: yes
+  notify: "update nextcloud upstream"
+
+- name: Deploy to swarm
+  community.general.docker_stack:
+    name: "{{ stack_name }}"
+    state: present
+    prune: yes
+    compose:
+      - "{{ stack_compose }}"
+  become: yes
+  tags:
+    - docker-swarm
+  notify: "update nextcloud upstream"
+

roles/nextcloud/templates/docker-stack.yml.j2

@@ -0,0 +1,160 @@
+version: '3.7'
+
+services:
+  web:
+    image: caddy
+    networks: 
+      - backend
+      - "{{ docker_swarm_public_network_name }}"
+    healthcheck:
+      test: ["CMD", "wget", "--quiet", "--spider", "--tries=1", "http://localhost:2019/metrics"]
+      interval: 1m
+      timeout: 10s
+      retries: 3
+      start_period: 1m
+    volumes:
+      - data:/var/www/html:ro
+      - "{{ nextcloud_upstream_file_dir }}/Caddyfile:/etc/caddy/Caddyfile:ro"
+      - caddy:/data
+
+  app:
+    image: "{{ stack_image }}:{{ nextcloud_version }}"
+    networks:
+      - backend
+    volumes:
+      - data:/var/www/html
+    healthcheck:
+      test: ["CMD", "nc", "-z", "localhost", "9000"]
+      interval: 1m
+      timeout: 10s
+      retries: 3
+      start_period: 5m
+    # needed for db to be up,
+    # see https://help.nextcloud.com/t/failed-to-install-nextcloud-with-docker-compose/83681/15
+    entrypoint: sh -c "while !(nc -z db 5432); do sleep 30; done; /entrypoint.sh php-fpm"
+    environment:
+      - NEXTCLOUD_ADMIN_USER={{ nextcloud_app_admin_username }}
+      - NEXTCLOUD_ADMIN_PASSWORD={{ nextcloud_app_admin_password }}
+      - REDIS_HOST=redis
+      - REDIS_HOST_PASSWORD={{ nextcloud_redis_password }}
+      - POSTGRES_HOST=db
+      - POSTGRES_DB={{ nextcloud_db_username }}
+      - POSTGRES_USER={{ nextcloud_db_username }}
+      - POSTGRES_PASSWORD={{ nextcloud_db_password }}
+{% if nextcloud_trusted_domains is not undefined and not none %}   
+      - NEXTCLOUD_TRUSTED_DOMAINS={{ nextcloud_trusted_domains }}
+{% endif %}
+{% if nextcloud_smtp_host is not undefined and not none %}   
+      - SMTP_HOST={{ nextcloud_smtp_host }}
+{% endif %}
+{% if nextcloud_smtp_port is not undefined and not none %}   
+      - SMTP_PORT={{ nextcloud_smtp_port }}
+{% endif %}
+{% if nextcloud_smtp_secure is not undefined and not none %}   
+      - SMTP_SECURE={{ nextcloud_smtp_secure }}
+{% endif %}
+{% if nextcloud_smtp_authtype is not undefined and not none %}   
+      - SMTP_AUTHTYPE={{ nextcloud_smtp_authtype }}
+{% endif %}
+{% if nextcloud_smtp_username is not undefined and not none %}   
+      - SMTP_NAME={{ nextcloud_smtp_username }}
+{% endif %}
+{% if nextcloud_smtp_password is not undefined and not none %}   
+      - SMTP_PASSWORD={{ nextcloud_smtp_password }}
+{% endif %}
+{% if nextcloud_smtp_from_address is not undefined and not none %}   
+      - MAIL_FROM_ADDRESS={{ nextcloud_smtp_from_address }}
+{% endif %}
+{% if nextcloud_smtp_from_domain is not undefined and not none %}   
+      - MAIL_DOMAIN={{ nextcloud_smtp_from_domain }}
+{% endif %}
+{% if nextcloud_s3_host is not undefined and not none %}   
+      - OBJECTSTORE_S3_HOST={{ nextcloud_s3_host }}
+{% endif %}
+{% if nextcloud_s3_bucket is not undefined and not none %}   
+      - OBJECTSTORE_S3_BUCKET={{ nextcloud_s3_bucket }}
+{% endif %}
+{% if nextcloud_s3_key is not undefined and not none %}   
+      - OBJECTSTORE_S3_KEY={{ nextcloud_s3_key }}
+{% endif %}
+{% if nextcloud_s3_secret is not undefined and not none %}   
+      - OBJECTSTORE_S3_SECRET={{ nextcloud_s3_secret }} 
+{% endif %}
+{% if nextcloud_s3_port is not undefined and not none %}   
+      - OBJECTSTORE_S3_PORT={{ nextcloud_s3_port }}
+{% endif %}
+{% if nextcloud_s3_ssl is not undefined and not none %}   
+      - OBJECTSTORE_S3_SSL={{ nextcloud_s3_ssl }}
+{% endif %}
+{% if nextcloud_s3_region is not undefined and not none %}   
+      - OBJECTSTORE_S3_REGION={{ nextcloud_s3_region }}
+{% endif %}
+{% if nextcloud_s3_usepath_style is not undefined and not none %}   
+      - OBJECTSTORE_S3_USEPATH_STYLE={{ nextcloud_s3_usepath_style }}
+{% endif %}
+{% if nextcloud_use_https is not undefined and not false %}   
+      - OVERWRITEPROTOCOL=https
+{% endif %}
+
+  cron:
+      image: {{ stack_image }}:{{ nextcloud_version }}
+      volumes:
+        - data:/var/www/html
+      healthcheck:
+        test: ["CMD", "php", "status.php", "|", "grep", "-q", "installed"]
+        interval: 1m
+        timeout: 10s
+        retries: 3
+        start_period: 5m
+      entrypoint: /cron.sh
+      networks:
+        - backend
+
+  db:
+    image: postgres:{{ nextcloud_db_version }}
+    environment:
+      - POSTGRES_USER={{ nextcloud_db_username }}
+      - POSTGRES_PASSWORD={{ nextcloud_db_password }}
+    healthcheck:
+      test: ["CMD", "pg_isready", "-q", "-U", "{{ nextcloud_db_username }}"]
+      interval: 1m
+      timeout: 10s
+      retries: 3
+      start_period: 1m
+    networks:
+      - backend
+    volumes:
+      - db:/var/lib/postgresql/data
+
+  redis:
+    image: redis:alpine
+    command: redis-server --requirepass {{ nextcloud_redis_password }}
+    healthcheck:
+      test: ["CMD", "redis-cli", "--pass", "{{ nextcloud_redis_password }}","ping"]
+      interval: 1m
+      timeout: 10s
+      retries: 3
+      start_period: 1m
+    volumes:
+      - redis:/data
+    networks:
+      - backend
+
+  # metrics:
+  #   image: telegraf
+  #   hostname: "${HOSTNAME:-vmi352583.contaboserver.net}"
+  #   networks:
+  #     - backend
+  #   volumes:
+  #     - ./telegraf:/etc/telegraf/telegraf.conf:ro
+
+volumes:
+  data:
+  db:
+  redis:
+  caddy:
+
+networks:
+  "{{ docker_swarm_public_network_name }}":
+    external: true
+  backend:

roles/nextcloud/templates/upstream.json.j2

@@ -0,0 +1,38 @@
+{
+    "@id": "{{ stack_name }}_upstream",
+{% if server_domain is not undefined and not none %}   
+    "match": [
+        {
+            "host": [
+{% if subdomain_alias is not undefined and not none %}
+                "{{ subdomain_alias }}.{{ server_domain }}"
+{% else %}
+                "{{ stack_name }}.{{ server_domain }}"
+{% endif %}
+            ]
+        }
+    ],
+{% else %}
+    "match": [
+        {
+            "path": [
+{% if subdomain_alias is not undefined and not none %}
+                "/{{ subdomain_alias }}*"
+{% else %}
+                "/{{ stack_name }}*"
+{% endif %}
+            ]
+        }
+    ],
+{% endif %}
+    "handle": [
+        {
+            "handler": "reverse_proxy",
+            "upstreams": [
+                {
+                    "dial": "{{ stack_name }}_web:80"
+                }
+            ]
+        }
+    ]
+}

roles/nextcloud/vars/main.yml

@@ -0,0 +1,7 @@
+---
+
+stack_name: nextcloud
+
+stack_image: "nextcloud"
+
+stack_compose: "{{ lookup('template', 'docker-stack.yml.j2') | from_yaml }}"

site-dev.yml

@@ -8,3 +8,4 @@
     - searx
     - traggo
     - monica
+    - nextcloud

site.yml

@@ -15,3 +15,4 @@
     - searx
     - traggo
     - monica
+    - nextcloud