# -*- coding: utf-8; mode: structured-text -*-

= Deployment =

Wiking applications are WSGI applications and may run on any WSGI compliant web
server.  This chapter describes the currently recommended setup: the packages
installed into a Python virtual environment, the application served by
[https://gunicorn.org/ Gunicorn] under a systemd service and exposed to the
outside world through a reverse proxy (such as Caddy or NGINX).  Alternative
setups (such as uWSGI) are outlined in the final Legacy setups section.

@NodeIndex@

== Installing the packages ==

Create a virtual environment and install Wiking into it.  Installing =wiking=
pulls in LCG (the =lcg-framework= package) automatically:

-----
python3 -m venv /srv/yoursite/venv
/srv/yoursite/venv/bin/pip install wiking
-----

When deploying Wiking CMS (or an application based on it), install the =cms=
extra instead, to add the additional dependencies:

-----
/srv/yoursite/venv/bin/pip install 'wiking[cms]'
-----

Pytis is not distributed through PyPI, so build its wheel from the sources and
install it into the same environment:

-----
git clone https://github.com/cerha/pytis
make -C pytis build
/srv/yoursite/venv/bin/pip install pytis/dist/pytis-*.whl
-----

Install your own application or extension packages the same way.


== Creating a configuration file ==

Create the configuration file, such as =\/srv/yoursite/config.py=:

-----
dbname = 'yoursite'
dbuser = 'www-data'
smtp_server = 'smtp.yourprovider.com'
default_sender_address = 'wiking@yoursite.com'
bug_report_address = 'bugs@yoursite.com'
-----

This example sets only the most important options.  See
[config Configuration Options] for the full list.

== Application modules == modules

The [config#modules modules] option determines what the site actually runs.  It
is a sequence of Python module names, searched in the given order for Wiking
module definitions.  There are three typical cases:

 * *Wiking CMS* -- leave =modules= unset.  It defaults to =('wiking.cms',)=,
   giving you the bare content management system.

 * *A standalone Wiking application* (not based on the CMS) -- set =modules= to
   your application's own modules, such as =('myapp.web',)=.

 * *Wiking CMS with custom extensions* -- combine your extension modules with
   ='wiking.cms'=, such as =('myapp.extension', 'wiking.cms')=.  The order
   matters: modules listed earlier are searched first, so your extensions may
   override CMS modules of the same name.

== Setting up the database ==

Wiking CMS (and applications based on it) keep their content in a PostgreSQL
database.  Create the database and the application's database role, load the
initial schema (application specific) and then keep the schema up to date with
the =cms-migrate= command installed together with the =wiking[cms]= package:

-----
/srv/yoursite/venv/bin/cms-migrate yoursite
-----

The only argument is the database name.  The command connects through psycopg2
using the standard PostgreSQL environment (=PGHOST=, =PGPORT=, peer
authentication, ...), so run it as a user with access to the database (use
=-p= to select a non-default port).  It applies all migration scripts bundled
with the installed Wiking version in a single transaction; run it again after
every package upgrade.

== Running with Gunicorn ==

The WSGI entry point is =wiking.wsgi_interface:application=.  The configuration
file is passed through the =wiking.config_file= environment variable:

-----
/srv/yoursite/venv/bin/gunicorn \
    --env wiking.config_file=/srv/yoursite/config.py \
    --bind unix:/run/wiking-yoursite/gunicorn.sock \
    --workers 3 \
    wiking.wsgi_interface:application
-----

Wiking is currently not thread safe, so always scale by the number of worker
*processes* (the =--workers= option) and never enable worker threads.

You may bind either to a UNIX socket (as above) or to a local TCP port (such as
=--bind 127.0.0.1:8000=); the reverse proxy then connects to the same address.
The socket path =\/run/wiking-<name>\/gunicorn.sock= is a convention expected by
the Caddy configuration below, where =<name>= is a short name of the site.

== Running as a systemd service ==

Run Gunicorn as a systemd service.  Create
=\/etc/systemd/system/yoursite.service=:

-----
[Unit]
Description=yoursite Wiking application
After=network.target postgresql.service

[Service]
User=www-data
Group=www-data
RuntimeDirectory=wiking-yoursite
ExecStart=/srv/yoursite/venv/bin/gunicorn \
    --env wiking.config_file=/srv/yoursite/config.py \
    --bind unix:/run/wiking-yoursite/gunicorn.sock \
    --worker-tmp-dir /run/wiking-yoursite \
    --workers 3 \
    wiking.wsgi_interface:application
Restart=on-failure

[Install]
WantedBy=multi-user.target
-----

The =RuntimeDirectory= directive makes systemd create
=\/run/wiking-yoursite= owned by the service user, which is a convenient place
for the socket (and for Gunicorn's temporary files).

Pass configuration file path through Gunicorn's =--env= option as shown above.

Enable and start the service:

-----
systemctl enable --now yoursite
-----

== Reverse proxy ==

The reverse proxy terminates HTTP(S) and forwards the requests to Gunicorn.
Any reverse proxy will do; the two examples below assume Gunicorn listening on
the UNIX socket =\/run/wiking-yoursite/gunicorn.sock= as configured above.

[https://caddyserver.com/ Caddy] (obtains and renews TLS certificates
automatically) can be configured by a snippet shared by all Wiking sites on the
server.  Put it into =\/etc/caddy/wiking.caddy=:

-----
(wiking) {
    encode zstd gzip
    reverse_proxy unix//run/wiking-{args[0]}/gunicorn.sock {
        header_up X-Sendfile-Type X-Accel-Redirect
        @xaccel header X-Accel-Redirect *
        handle_response @xaccel {
            root * /
            rewrite * {rp.header.X-Accel-Redirect}
            uri strip_prefix /_xaccel
            method * GET
            @outside not path {args[1:]}
            error @outside 403
            copy_response_headers {
                exclude X-Accel-Redirect Content-Length Server
            }
            file_server
        }
    }
}
-----

Import the snippet at the beginning of =\/etc/caddy/Caddyfile= and use it in
the block of each site, giving the site name (the =<name>= of its socket path)
and the patterns of the paths of files which Caddy may send for the
application (see Accelerated file downloads below):

-----
import /etc/caddy/wiking.caddy

yoursite.com {
    import wiking yoursite /srv/yoursite/*
}
-----

The snippet doesn't work with Caddy 2.6 and older, which don't support its
argument placeholders (Caddy 2.6 is shipped for example in Debian 13, where a
newer version is available in backports).

[https://nginx.org/ NGINX]:

-----
server {
    listen 80;
    server_name yoursite.com;
    location / {
        proxy_pass http://unix:/run/wiking-yoursite/gunicorn.sock;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Sendfile-Type X-Accel-Redirect;
    }
    location /_xaccel/srv/yoursite/ {
        internal;
        alias /srv/yoursite/;
    }
}
-----

The =Host= header must include the port, so use =$http_host= rather than
=$host=, as the application uses it to construct its URIs (such as
redirections).  The last two directives set up accelerated file downloads.

== Accelerated file downloads ==

By default, all files (such as resources or files stored by the application)
are read and sent by the Python process, which blocks a Gunicorn worker for the
whole duration of the download.  The reverse proxy can serve the files instead:
the application only responds with the =X-Accel-Redirect= header containing the
internal URI =\/\_xaccel= followed by the file's absolute path and the proxy sends
the file itself (including support for partial downloads through the =Range=
header and caching through =ETag= and =Last-Modified=).

The proxy announces this ability by the request header
=X-Sendfile-Type: X-Accel-Redirect= (both configurations above set it and
replace the header possibly sent by the client).  Without it, such as when
Gunicorn is accessed directly, the files are sent by Python, so no
configuration of the application is needed.

This applies to all resource files (all directories of [config#resource_path])
automatically.  Other files served by the application (such as its data files)
are only served by the proxy when located in one of the directories given by
[config#xaccel_paths], otherwise they are still sent by Python.  The proxy's
user must have read access to the files.  The proxy only serves the files which
the application decides to serve, since the internal URI is never accepted from
a client.

The proxy also relies on the application to decide which files may be served.
If the application was compromised (so that an attacker could set arbitrary
response headers), it could be used to read any file which the proxy's user can
read, including other sites' files or the private keys of the proxy's TLS
certificates.  Thus the proxy is restricted to the directories containing the
application's files: the virtual environment (with the resources of all the
installed components) and the site's own data.  These are the path patterns
given to the Caddy snippet, such as =\/srv/yoursite/*= above, or for sites
sharing one virtual environment:

-----
example.org {
    import wiking example.org /srv/www/wiking/venv/* /srv/www/example.org/*
}
-----

Caddy responds by 403 (Forbidden) for other files.  In NGINX, add an internal
location for each directory (the paths under =\/\_xaccel/= not matching any of
them are passed to the application, which responds by 404).  Don't give
different sites access to each other's directories.

The =copy_response_headers= subdirective of the Caddy snippet passes the
headers set by the application (such as =Content-Type= or
=Content-Disposition=) to the client.  NGINX does that by default.

== Legacy setups == legacy

The following describes the older deployment approach based on running the
components directly from git checkouts (rather than from installed wheels) and
serving the application through uWSGI.  It is kept here for reference; new
deployments should prefer the setup described above.

In this approach the git sources are put into the =src= subdirectory of the
site's root directory (such as =\/var/www/yoursite/src=) and their Python
libraries are linked into a single =lib= directory added to the Python path:

-----
mkdir -p /var/www/yoursite/src
cd /var/www/yoursite/src
git clone https://github.com/cerha/wiking
git clone https://github.com/cerha/pytis
git clone https://github.com/cerha/lcg
cd ..
mkdir lib
ln -s ../src/wiking/lib/wiking lib/wiking
ln -s ../src/pytis/lib/pytis lib/pytis
ln -s ../src/lcg/lib/lcg lib/lcg
-----

Because the data directories are not derived automatically in this layout, the
configuration file must set the paths explicitly:

-----
import os
sitedir = '/var/www/yoursite'
resource_path = [os.path.join(sitedir, 'src', subdir, 'resources')
                 for subdir in ('wiking', 'pytis', 'lcg')]
translation_path = [os.path.join(sitedir, 'src', subdir, 'translations')
                    for subdir in ('wiking', 'pytis', 'lcg')]
doc_dirs = {subdir: os.path.join(sitedir, 'src', subdir, 'doc', 'src')
            for subdir in ('wiking', 'pytis', 'lcg')}
-----

[https://uwsgi-docs.readthedocs.io/en/latest/ uWSGI] runs the application in a
standalone process (or a group of processes) and communicates with the web
server, which plays the role of a thin HTTP frontend.  Both sides need to use
the same socket.

=== Running uWSGI manually === uwsgi-manual

It may be useful to run uWSGI manually for simple testing and debugging, as you
see most problems immediately on its output:

-----
uwsgi --plugin=python3 \
      --chdir=/var/www/yoursite \
      --wsgi-file=lib/wiking/wsgi_interface.py \
      --python-path=lib \
      --virtualenv=venv \
      --socket=yoursite.socket \
      --route-run=addvar:wiking.config_file=config.py \
      --master --processes 3
-----

This creates the communication socket in =\/var/www/yoursite/yoursite.socket=.
Make sure the web server user has read/write access to it.

=== Emperor mode === uwsgi-emperor

For production, uWSGI's Emperor mode runs one configuration file (vassal) per
app, typically in =\/etc/uwsgi/apps-enabled= on Debian and Ubuntu based systems.

Example =\/etc/uwsgi/apps-available/yoursite.ini=:
-----
[uwsgi]
plugin = python3
chdir = /var/www/yoursite
wsgi-file = lib/wiking/wsgi_interface.py
python-path = lib
virtualenv = venv
route-run = addvar:wiking.config_file=config.py
-----

Activate it by linking the file into =\/etc/uwsgi/apps-enabled=:
-----
ln -s /etc/uwsgi/apps-available/yoursite.ini /etc/uwsgi/apps-enabled/
-----

On Debian/Ubuntu the socket is created in =\/run/uwsgi/app/yoursite/socket= and
the log in =\/var/log/uwsgi/app/yoursite.log=.  The following options give more
control when needed:

-----
# User and group of the uWSGI processes of your app.
uid = www-data
gid = www-data

# Where the PID file and log file is created. Uid/gid
# needs read-write access to the directory.
pidfile = /var/www/yoursite/yoursite.pid
logto = /var/log/uwsgi/yoursite.log

# Where the UNIX socket is created. Uid/gid needs read-write
# access to the directory and your web server needs at least
# read-only access to the directory.
socket = /var/www/yoursite/yoursite.socket

# Socket privileges.  Its uid/gid matches uid/gid set above.
# The web server user needs read-write access to the socket.
chmod-socket = 660

# Auto-reload the app on each modification of config.py.
touch-reload = /var/www/yoursite/config.py
-----

The web server is then configured to pass requests to the uWSGI socket, e.g.
NGINX:
-----
server {
    listen 80;
    server_name www.yoursite.com;
    location / {
        include uwsgi_params;
        uwsgi_pass unix://run/uwsgi/app/yoursite/socket;
    }
}
-----

or Apache with =mod_proxy= enabled:
-----
<VirtualHost *>
  ServerName www.yoursite.com
  ProxyPass / unix:/run/uwsgi/app/yoursite/socket;
</VirtualHost>
-----
