Django/Production

From Dev Wiki
< Django
Revision as of 02:43, 1 December 2020 by Brodriguez (talk | contribs) (Add note about required Python package)
Jump to navigation Jump to search

The following is an attempt at detailing steps to push a Django project into production.

Note that this page makes references to several system root folders, particularly srv and opt. For details on what root folders are generally meant for, see https://help.ubuntu.com/community/LinuxFilesystemTreeOverview.

However, having said that, these folders are not set in stone and if you have reason to change directory paths, that's okay as long as you change all references to such.

Note: Most of this page will assume you have root account access, as that's required for Apache/Nginx setup anyways. If you don't, then consult the provider of your webhosting for how they recommend to proceed.


Pre-Setup

If not yet done, create an appropriate database instance for your project. Note that SqLite is not sufficient, as it's unlikely to be able to handle throughput necessary for a server. MySQL/ PostgreSQL/Oracle/etc are designed for this.

Also update your project package requirements to include the uWSGI Python package. This is required for Apache/Daphne (aka Nginx) to interface with a Python virtual environment.

Base Project Setup

Project Location

First, create directories and clone your project to the desired location on the server. The recommended location is somewhere like: 

/src/web_serve/django/<project_name>/

Initial Production Settings Values

Then edit your project settings for bare minimum production values to get it up and running. This likely includes things like:

  • Connection settings for the production database.
  • Static file serve locations.
  • Logging setup.
  • Allowed Hosts setting.
  • Secret Key setting.
  • Any possible custom project settings, required to get the project running.

While you're editing the settings file(s), it's recommended to set these as well, but technically it can wait until the end of the project setup.

  • Email settings.
  • Security settings.
  • Any possible custom project settings, required for long-term stability & security
ToDo: Document Django settings file production values.

Installation

If your project has any custom installation scripts, now might be a good time to run them. Otherwise, install system dependencies (such as Apt-Get Packages) and then run the standard manage.py setup commands.

Apt Package Installation

Assuming use of an Ubuntu server, common system packages for production are:

Note that these are the bare minimum for a standard Django install. Depending on your project, you might need other things such as NPM packages to manage Programming/JavaScript files.

Setup Python Environment

After installing system packages, you can create a Python Virtual Environment to serve your project from.

The recommended location is something like: 

/opt/env/<environment_name>/

Once created, remember to load the environment to your console, and then install project packages. (ex: pip install -r requirements.txt from project root).

Manage.py Setup Commands

Note: Before this step, if your project has any static files (CSS or JavaScript) to compile, you should probably do that here.

Once all system packages are installed and the environment is setup, we can finally run our manage.py commands.

Depending on your project, you may have additional custom commands to install. But the standard default ones are (in order):

  • python manage.py migrate
  • python manage.py collectstatic
  • python manage.py check
Warn: Note that we don't run python manage.py makemigrations in production. This is because migration files should always be created and committed in development, and then pushed up after thorough testing.

Project Permissions

Permissions should be set such that both the expected user(s) and apache/nginx can access project files.

If only one single user account is expected to access the project, then you should probably set folder/file permissions to <username>:www-data.

Alternatively, if multiple user accounts are expected to access the project, then you should probably create a group that all users belong to. Then set folder/file permissions to www-data:<group_name>.

In both cases, you'll probably want to set permissions such that:

  • Users/groups both have read/write/execute permissions on folders.
  • Users/groups both have read/write permissions on files.
  • Other users have no access on folders or files.

You'll want to make sure to set these permissions on:

  • Project folders
  • Static folders associated with the project
  • The local virtual environment
  • Project logging folders

For more details on permissions, see Linux Permissions.

Post-Installation Notes

At this point, your project should be set up enough to run via python manage.py runserver and project UnitTests should pass when run.

While we definitely don't want to serve the project as it is now (and you shouldn't be able to load pages in a web browser at this point), these act as a good check to make sure everything is handling correctly so far.

If you get any errors at this stage, go back and troubleshoot it before proceeding.

ToDo: apache setup?

Nginx Setup

If you're using Django Channels, then you must use Nginx to serve the project. Apache is physically incapable of handling the asynchronous websocket connections that channels uses.

Configuring Volatile Project Files

First, we want to configure our "volatile" project files. This includes our project runtime process, our web-socket files, and other entities pertaining to our project, which only exist during runtime.

It's recommended to put these in the /run/<process_name>/ directory and associated logs in the /var/log/<process_name>/ directory.

This can be done by creating the following files with the respective contents: 

/etc/tmpfiles.d/daphne.conf
d /run/daphne 0755 www-data www-data -
d /var/log/daphne 0755 www-data www-data -

/etc/tmpfiles.d/uwsgi.conf
d /run/uwsgi 0755 www-data www-data -
d /var/log/uwsgi 0755 www-data www-data -

Once the files are created, we can execute them with 

sudo systemd-tmpfiles --create

For more details on these tmpfiles, see https://manpages.ubuntu.com/manpages/bionic/man5/tmpfiles.d.5.html

uWSGI Configurations

uWSGI is essentially a project meant to "provide a common api and configuration style" for hosting services from various different languages.

It's effectively the interface we use to connect Nginx and Django together.

For this, first we create the following folder and set ownership to www-data: 

sudo mkdir -p /etc/uwsgi/sites
sudo chown www-data:www-data /etc/uwsgi -R

Next, we add a configuration file for our project: 

/etc/uwsgi/sites/<project_name>.ini
[uwsgi]
project = <project_name>


# Django Settings.
chdir = /srv/web_serve/django/%(project)
home = /opt/env/<virtual_env_name>
module = settings.wsgi:application


# Process Settings.
master = true
processes = 5
threads = 2
uid = www-data
gid = www-data
chmod-socket = 664
pidfile = /run/uwsgi/%(project).pid
socket = /run/uwsgi/%(project).sock
wsgi-file = %(chdir)/settings/wsgi.py
manage-script-name = true
vacuum = true
max-requests = 5000
logto = /var/log/uwsgi/%n.log

For this config, note to change:

  • project = <project_name> (the second line) to match your project's root folder name.
  • home = /opt/env/<virtual_env_name> (the seventh line) to match your virtual environment location.
  • wsgi-file = %(chdir)/settings/wsgi.py (5th to last line) to point to the actual uwsgi file in your project.

While these settings won't work for all projects, they should be a good starting point for the average small Django project to get up and running.

Nginx Configurations

ToDo: Create separate wiki section for general nginx syntax, which will significantly reduce size of this section and make it far more generic.
Note: For the following section, make sure to replace all values in pointy brackets. Ex: Every instance of <project_name> should be replaced with your project's actual name.

There are many ways to setup a nginx server, so this will document the basics.

All of these configurations are usually located at /etc/nginx/sites-available/.

First, at the top of the file, we set up to automatically redirect all HTTP requests to HTTPS (note, this requires having SSL certs on your server, which is highly recommended): 

/etc/nginx/sites-available/<project_name>.conf
# Insecure connection configuration.
server {

        # Base settings.
        server_name <server_name>;
        listen 80;

        # Redirect settings.
        return 302 https://$host$request_uri;

}

Next, we configure a new section in the file for HTTPS: 

/etc/nginx/sites-available/<project_name>.conf
# Secure connection configuration.
server {

        # Base settings.
        server_name <server_name>;
        listen 443 ssl;

        # SSL settings.
        include snippets/ssl-signed.conf;

}
ToDo: Document making server more secure with possible SSL params.

Create a snippet for our above SSL to pull from: 



ssl_certificate <ssl_certificate_location>;
ssl_certificate_key <ssl_certificate_key_location>;



Both lines should be changed to reflect the absolute path on the server for the SSL certificate, and SSL key, respectively.

With HTTPS taken care of, we can go back to our configuration file to add our Django paths. These values should be included within the 443 SSL section.

We can add paths to serve our static files:



/etc/nginx/sites-available/<project_name>.conf


        # Set Django Static and Media file serving.
        location /static {
                alias <path_to_static_file_location>;
        }
        location /media {
                alias <path_to_media_file_location>;
        }





We can add paths to serve Django Channels web-socket requests (this assumes websockets are all served under a root url of /ws/:



/etc/nginx/sites-available/<project_name>.conf


        # Send websocket connections to Daphne.
        location /ws {
                proxy_pass http://unix:/var/run/daphne/<project_name>.sock;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";

                proxy_redirect off;
                proxy_set_header Host $host;
                proxy_set_header X-Real-OP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Host $server_name;
        }





We can define a reusable value to point all our project urls to:



/etc/nginx/sites-available/<project_name>.conf


        # Define logic for serving project.
        location @<project_name> {
                include uwsgi_params;
                uwsgi_pass unix:/var/run/uwsgi/<project_name>.sock;
        }



And finally, we can serve the urls of our project. For example, if your Django project served urls under both /admin/ and /user/ then we might have:



/etc/nginx/sites-available/<project_name>.conf


        # Grab urls for Django Project.
        location /admin {
                try_files $uri @<project_name>;
        }
        location /user {
                try_files $uri @<project_name>;
        }



Optionally, if you want to set a specific url to redirect, you can also do that here.



/etc/nginx/sites-available/<project_name>.conf


        location /some_url_to_redirect {
                return 302 <full_url_to_redirect_to>;
        }



Once done, save config files and enable the config by creating a symlink from /etc/nginx/sites-available/<project_name>.conf to /etc/nginx/sites-enabled/<project_name>.conf:



ln -s /etc/nginx/sites-available/<project_name>.conf /etc/nginx/sites-enabled/<project_name>.conf



Check for errors with:



sudo nginx -t



Note:  By default, there is probably a "default" config file under /etc/nginx/sites-enabled/. You can safely remove this file, as it's effectively being replaced by your own config. Furthermore, leaving this default may cause issues and override values in any config you create.


If everything checks out, you can run the following to reload Nginx:



sudo systemctl restart nginx



At this point, if everything went well, then your project is now being served and can be accessed from a web browser.



Systemd Service File Configurations


The systemd service files are effectively how Linux determines what processes should start on boot. They are located in the /etc/systemd/system/ folder.

For our project, we will create two files here:



/etc/systemd/system/uwsgi.service


[Unit]
Description=Daphne server for Django web projects
After=syslog.target network.target uwsgi.service


[Service]
ExecStart=/opt/env/<virtual_env_name>/bin/daphne -u /run/daphne/<project_name>.sock settings.asgi:application
Restart=always
Type=simple
WorkingDirectory=/srv/web_serve/<project_name>
User=www-data
Group=<group_name>
StandardOutput=syslog
StandardError=syslog
KillSignal=SIGQUIT


[Install]
WantedBy=multi-user.target



/etc/systemd/system/daphne.service


[Unit]
Description=uWSGI Application Server in Emperor Mode
After=syslog.target network.target


[Service]
ExecStart=/opt/env/<virtual_env_name>/bin/uwsgi --emperor /etc/uwsgi/sites
Restart=always
Type=notify
User=www-data
Group=<group_name>
StandardOutput=syslog
StandardError=syslog
KillSignal=SIGQUIT
NotifyAccess=all


[Install]
WantedBy=multi-user.target



Once the files are created, tell the system to reload:



sudo systemctl daemon-reload



If no errors occur, then everything is good and your project is officially serving in production.









Retrieved from "https://wiki.brandon-rodriguez.com/index.php?title=Django/Production&oldid=551"