Django/Production

From Dev Wiki
< Django
Revision as of 10:43, 14 January 2021 by Brodriguez (talk | contribs) (Correct example)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

The following is an attempt at detailing steps to push a Django project into production.

Note that this page makes references to several system root folders, particularly srv and opt. For details on what root folders are generally meant for, see https://help.ubuntu.com/community/LinuxFilesystemTreeOverview.

However, having said that, these folders are not set in stone and if you have reason to change directory paths, that's okay as long as you change all references to such.

Note: Most of this page will assume you have root account access, as that's required for Apache/Nginx setup anyways. If you don't, then consult the provider of your webhosting for how they recommend to proceed.


Pre-Setup

If not yet done, create an appropriate database instance for your project. Note that SqLite is not sufficient, as it's unlikely to be able to handle throughput necessary for a server. MySQL/ PostgreSQL/Oracle/etc are designed for this.

Also update your project package requirements to include the uWSGI Python package. This is required for Apache/Daphne (aka Nginx) to interface with a Python virtual environment.

Base Project Setup

Project Location

First, create directories and clone your project to the desired location on the server. The recommended location is somewhere like: 

/src/web_serve/django/<project_name>/

Initial Production Settings Values

Then edit your project settings for bare minimum production values to get it up and running. This likely includes things like:

  • Connection settings for the production database.
  • Static file serve locations.
  • Logging setup.
  • Allowed Hosts setting.
  • Secret Key setting.
  • Any possible custom project settings, required to get the project running.

While you're editing the settings file(s), it's recommended to set these as well, but technically it can wait until the end of the project setup.

  • Email settings.
  • Security settings.
  • Any possible custom project settings, required for long-term stability & security
ToDo: Document Django settings file production values.

Installation

If your project has any custom installation scripts, now might be a good time to run them. Otherwise, install system dependencies (such as Apt-Get Packages) and then run the standard manage.py setup commands.

Apt Package Installation

Assuming use of an Ubuntu server, common system packages for production are:

Note that these are the bare minimum for a standard Django install. Depending on your project, you might need other things such as NPM packages to manage Programming/JavaScript files.

Setup Python Environment

After installing system packages, you can create a Python Virtual Environment to serve your project from.

The recommended location is something like: 

/opt/env/<environment_name>/

Once created, remember to load the environment to your console, and then install project packages. (ex: pip install -r requirements.txt from project root).

Manage.py Setup Commands

Note: Before this step, if your project has any static files (CSS or JavaScript) to compile, you should probably do that here.

Once all system packages are installed and the environment is setup, we can finally run our manage.py commands.

Depending on your project, you may have additional custom commands to install. But the standard default ones are (in order):

  • python manage.py migrate
  • python manage.py collectstatic
  • python manage.py check
Warn: Note that we don't run python manage.py makemigrations in production. This is because migration files should always be created and committed in development, and then pushed up after thorough testing.

Project Permissions

Permissions should be set such that both the expected user(s) and apache/nginx can access project files.

Nginx seems to assume the user is Template:Www-data. You should probably also create a group that all users belong to. Thus we set folder/file permissions to www-data:<group_name>.

In both cases, you'll probably want to set permissions such that:

  • Users/groups both have read/write/execute permissions on folders.
  • Users/groups both have read/write permissions on files.
  • Other users have no access on folders or files.

You'll want to make sure to set these permissions on:

  • Project folders
  • Static folders associated with the project
  • The local virtual environment
  • Project logging folders
  • Any other files/folders that your website will need direct access to.

For more details on permissions, see Linux Permissions.

Post-Installation Notes

At this point, your project should be set up enough to run via python manage.py runserver and project UnitTests should pass when run.

While we definitely don't want to serve the project as it is now (and you shouldn't be able to load pages in a web browser at this point), these act as a good check to make sure everything is handling correctly so far.

If you get any errors at this stage, go back and troubleshoot it before proceeding.

ToDo: apache setup?

Nginx Setup

If you're using Django Channels, then you must use Nginx to serve the project. Apache is physically incapable of handling the asynchronous websocket connections that channels uses.

Configuring Volatile Project Files

First, we want to configure our "volatile" project files. This includes our project runtime process, our web-socket files, and other entities pertaining to our project, which only exist during runtime.

It's recommended to put these in the /run/<process_name>/ directory and associated logs in the /var/log/<process_name>/ directory.

This can be done by creating the following files with the respective contents: 

/etc/tmpfiles.d/daphne.conf
d /run/daphne 0755 www-data www-data -
d /var/log/daphne 0755 www-data www-data -

/etc/tmpfiles.d/uwsgi.conf
d /run/uwsgi 0755 www-data www-data -
d /var/log/uwsgi 0755 www-data www-data -

Once the files are created, we can execute them with 

sudo systemd-tmpfiles --create

For more details on these tmpfiles, see https://manpages.ubuntu.com/manpages/bionic/man5/tmpfiles.d.5.html

uWSGI Configurations

uWSGI is essentially a project meant to "provide a common api and configuration style" for hosting services from various different languages.

It's effectively the interface we use to connect Nginx and Django together.

For this, first we create the following folder and set ownership to www-data: 

sudo mkdir -p /etc/uwsgi/sites
sudo touch /etc/uwsgi/sites/<project_name>.ini
sudo chown www-data:www-data /etc/uwsgi -R

Next, we add a configuration file for our project: 

/etc/uwsgi/sites/<project_name>.ini
[uwsgi]
project = <project_name>


# Django Settings.
chdir = /srv/web_serve/django/%(project)
home = /opt/env/<virtual_env_name>
module = <project_settings_folder>.wsgi:application


# Process Settings.
master = true
processes = 5
threads = 2
uid = www-data
gid = www-data
chmod-socket = 664
pidfile = /run/uwsgi/%(project).pid
socket = /run/uwsgi/%(project).sock
wsgi-file = %(chdir)/settings/wsgi.py
manage-script-name = true
vacuum = true
max-requests = 5000
logto = /var/log/uwsgi/%n.log

For this config, note to change:

  • project = <project_name> (the second line) to match your project's root folder name.
  • home = /opt/env/<virtual_env_name> (the seventh line) to match your virtual environment location.
  • module = <project_settings_folder>.wsgi:application (the eigth line) to match your project settings folder.
  • wsgi-file = %(chdir)/settings/wsgi.py (5th to last line) to point to the actual uwsgi file in your project.

While these settings won't work for all projects, they should be a good starting point for the average small Django project to get up and running.

Nginx Configurations

There are many ways to setup a nginx server, with some basics located at Programming/Nginx.

ToDo: Document making server more secure with possible SSL params.

Recommended steps for serving a Django project are as follows:

  • First, it's strongly recommended to set up HTTPS and redirect all server traffic to HTTPS.
  • Set up static file handling for the /media and /static urls (or whatever the equivalent is for your specific Django project).
  • If applicable, set up request redirects for your project, where needed.
  • Define your core UWSGI request handler, preferably via reusable location blocks.
  • Finally, if using something that needs websocket request handling (such as DjangoChannels, then we need an additional handler. The following code snippet passes requests to Daphne for DjangoChannels handling (this assumes websockets are all served under a root url of example.com/ws/):
/etc/nginx/sites-available/<project_name>.conf
    # Send websocket connections to Daphne.
    location /ws {
        proxy_pass http://unix:/run/daphne/<project_name>.sock;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-OP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
    }


Systemd Service File Configurations

The systemd service files are effectively how Linux determines what processes should start on boot. They are located in the /etc/systemd/system/ folder.

For our project, we will create two files here: 

/etc/systemd/system/daphne.service
[Unit]
Description=Daphne server for Django web projects
After=syslog.target network.target uwsgi.service


[Service]
ExecStart=/opt/env/<virtual_env_name>/bin/daphne -u /run/daphne/<project_name>.sock <project_settings_folder>.asgi:application
Restart=always
Type=simple
WorkingDirectory=/srv/web_serve/<project_name>
User=www-data
Group=<group_name>
StandardOutput=syslog
StandardError=syslog
KillSignal=SIGQUIT


[Install]
WantedBy=multi-user.target

/etc/systemd/system/uwsgi.service
[Unit]
Description=uWSGI Application Server in Emperor Mode
After=syslog.target network.target


[Service]
ExecStart=/opt/env/<virtual_env_name>/bin/uwsgi --emperor /etc/uwsgi/sites
Restart=always
Type=notify
User=www-data
Group=<group_name>
StandardOutput=syslog
StandardError=syslog
KillSignal=SIGQUIT
NotifyAccess=all


[Install]
WantedBy=multi-user.target

Once the files are created, tell the system to reload: 

sudo systemctl daemon-reload
sudo systemctl start daphne
sudo systemctl enable daphne
sudo systemctl start uwsgi
sudo systemctl enable uwsgi

Check for possible errors with: 

journalctl -xe

If no errors occur, then everything is good and your project is officially serving in production. Verify by trying to access your site in a browser.

Retrieved from "https://wiki.brandon-rodriguez.com/index.php?title=Django/Production&oldid=583"