Django/Production: Difference between revisions
Brodriguez (talk | contribs) (Update for new nginx page) |
Brodriguez (talk | contribs) (Correct example) |
||
(One intermediate revision by the same user not shown) | |||
Line 69: | Line 69: | ||
Permissions should be set such that both the expected user(s) and apache/nginx can access project files. | Permissions should be set such that both the expected user(s) and apache/nginx can access project files. | ||
Nginx seems to assume the user is {{www-data}}. You should probably also create a group that all users belong to. Thus we set folder/file permissions to {{ ic |www-data:<group_name>}}. | |||
In both cases, you'll probably want to set permissions such that: | In both cases, you'll probably want to set permissions such that: | ||
Line 83: | Line 81: | ||
* The local virtual environment | * The local virtual environment | ||
* Project logging folders | * Project logging folders | ||
* Any other files/folders that your website will need direct access to. | |||
For more details on permissions, see [[Linux/Permissions | Linux Permissions]]. | For more details on permissions, see [[Linux/Permissions | Linux Permissions]]. | ||
Line 175: | Line 174: | ||
* If applicable, set up [[Programming/Nginx#Example 2: Redirecting Specific Requests | request redirects]] for your project, where needed. | * If applicable, set up [[Programming/Nginx#Example 2: Redirecting Specific Requests | request redirects]] for your project, where needed. | ||
* Define your core [[Programming/Nginx#Example 3: Passing Requests to UWSGI | UWSGI request handler]], preferably via [[Programming/Nginx#Reusable Location Blocks | reusable location blocks]]. | * Define your core [[Programming/Nginx#Example 3: Passing Requests to UWSGI | UWSGI request handler]], preferably via [[Programming/Nginx#Reusable Location Blocks | reusable location blocks]]. | ||
* Finally, if using something that needs websocket request handling (such as [https://channels.readthedocs.io/en/stable/ DjangoChannels], then we need an additional handler. The following code snippet passes requests to Daphne for DjangoChannels handling (this assumes websockets are all served under a root url of {{ ic |/ws/}}): | * Finally, if using something that needs websocket request handling (such as [https://channels.readthedocs.io/en/stable/ DjangoChannels], then we need an additional handler. The following code snippet passes requests to Daphne for DjangoChannels handling (this assumes websockets are all served under a root url of {{ ic |example.com/ws/}}): | ||
{{ hc |/etc/nginx/sites-available/<project_name>.conf| | {{ hc |/etc/nginx/sites-available/<project_name>.conf| | ||
<nowiki> # Send websocket connections to Daphne. | <nowiki> # Send websocket connections to Daphne. |
Latest revision as of 10:43, 14 January 2021
The following is an attempt at detailing steps to push a Django project into production.
Note that this page makes references to several system root folders, particularly srv
and opt
. For details on what root folders are generally meant for, see https://help.ubuntu.com/community/LinuxFilesystemTreeOverview.
However, having said that, these folders are not set in stone and if you have reason to change directory paths, that's okay as long as you change all references to such.
Pre-Setup
If not yet done, create an appropriate database instance for your project. Note that SqLite is not sufficient, as it's unlikely to be able to handle throughput necessary for a server. MySQL/ PostgreSQL/Oracle/etc are designed for this.
Also update your project package requirements to include the uWSGI
Python package. This is required for Apache/Daphne (aka Nginx) to interface with a Python virtual environment.
Base Project Setup
Project Location
First, create directories and clone your project to the desired location on the server. The recommended location is somewhere like:
/src/web_serve/django/<project_name>/
Initial Production Settings Values
Then edit your project settings for bare minimum production values to get it up and running. This likely includes things like:
- Connection settings for the production database.
- Static file serve locations.
- Logging setup.
- Allowed Hosts setting.
- Secret Key setting.
- Any possible custom project settings, required to get the project running.
While you're editing the settings file(s), it's recommended to set these as well, but technically it can wait until the end of the project setup.
- Email settings.
- Security settings.
- Any possible custom project settings, required for long-term stability & security
Installation
If your project has any custom installation scripts, now might be a good time to run them. Otherwise, install system dependencies (such as Apt-Get Packages) and then run the standard manage.py
setup commands.
Apt Package Installation
Assuming use of an Ubuntu server, common system packages for production are:
- The desired Python version to serve your project.
- Apache packages, for a standard Django install (alternatively, can use Nginx if you'd prefer).
- Nginx packages, if using Django Channels.
- Redis packages, also if using Django Channels.
Note that these are the bare minimum for a standard Django install. Depending on your project, you might need other things such as NPM packages to manage Programming/JavaScript files.
Setup Python Environment
After installing system packages, you can create a Python Virtual Environment to serve your project from.
The recommended location is something like:
/opt/env/<environment_name>/
Once created, remember to load the environment to your console, and then install project packages. (ex: pip install -r requirements.txt
from project root).
Manage.py Setup Commands
Once all system packages are installed and the environment is setup, we can finally run our manage.py commands.
Depending on your project, you may have additional custom commands to install. But the standard default ones are (in order):
python manage.py migrate
python manage.py collectstatic
python manage.py check
python manage.py makemigrations
in production. This is because migration files should always be created and committed in development, and then pushed up after thorough testing.Project Permissions
Permissions should be set such that both the expected user(s) and apache/nginx can access project files.
Nginx seems to assume the user is Template:Www-data. You should probably also create a group that all users belong to. Thus we set folder/file permissions to www-data:<group_name>
.
In both cases, you'll probably want to set permissions such that:
- Users/groups both have read/write/execute permissions on folders.
- Users/groups both have read/write permissions on files.
- Other users have no access on folders or files.
You'll want to make sure to set these permissions on:
- Project folders
- Static folders associated with the project
- The local virtual environment
- Project logging folders
- Any other files/folders that your website will need direct access to.
For more details on permissions, see Linux Permissions.
Post-Installation Notes
At this point, your project should be set up enough to run via python manage.py runserver
and project UnitTests should pass when run.
While we definitely don't want to serve the project as it is now (and you shouldn't be able to load pages in a web browser at this point), these act as a good check to make sure everything is handling correctly so far.
If you get any errors at this stage, go back and troubleshoot it before proceeding.
Nginx Setup
If you're using Django Channels, then you must use Nginx to serve the project. Apache is physically incapable of handling the asynchronous websocket connections that channels uses.
Configuring Volatile Project Files
First, we want to configure our "volatile" project files. This includes our project runtime process, our web-socket files, and other entities pertaining to our project, which only exist during runtime.
It's recommended to put these in the /run/<process_name>/
directory and associated logs in the /var/log/<process_name>/ directory
.
This can be done by creating the following files with the respective contents:
/etc/tmpfiles.d/daphne.conf
d /run/daphne 0755 www-data www-data - d /var/log/daphne 0755 www-data www-data -
/etc/tmpfiles.d/uwsgi.conf
d /run/uwsgi 0755 www-data www-data - d /var/log/uwsgi 0755 www-data www-data -
Once the files are created, we can execute them with
sudo systemd-tmpfiles --create
For more details on these tmpfiles, see https://manpages.ubuntu.com/manpages/bionic/man5/tmpfiles.d.5.html
uWSGI Configurations
uWSGI is essentially a project meant to "provide a common api and configuration style" for hosting services from various different languages.
It's effectively the interface we use to connect Nginx and Django together.
For this, first we create the following folder and set ownership to www-data
:
sudo mkdir -p /etc/uwsgi/sites sudo touch /etc/uwsgi/sites/<project_name>.ini sudo chown www-data:www-data /etc/uwsgi -R
Next, we add a configuration file for our project:
/etc/uwsgi/sites/<project_name>.ini
[uwsgi] project = <project_name> # Django Settings. chdir = /srv/web_serve/django/%(project) home = /opt/env/<virtual_env_name> module = <project_settings_folder>.wsgi:application # Process Settings. master = true processes = 5 threads = 2 uid = www-data gid = www-data chmod-socket = 664 pidfile = /run/uwsgi/%(project).pid socket = /run/uwsgi/%(project).sock wsgi-file = %(chdir)/settings/wsgi.py manage-script-name = true vacuum = true max-requests = 5000 logto = /var/log/uwsgi/%n.log
For this config, note to change:
project = <project_name>
(the second line) to match your project's root folder name.home = /opt/env/<virtual_env_name>
(the seventh line) to match your virtual environment location.module = <project_settings_folder>.wsgi:application
(the eigth line) to match your project settings folder.wsgi-file = %(chdir)/settings/wsgi.py
(5th to last line) to point to the actual uwsgi file in your project.
While these settings won't work for all projects, they should be a good starting point for the average small Django project to get up and running.
Nginx Configurations
There are many ways to setup a nginx server, with some basics located at Programming/Nginx.
Recommended steps for serving a Django project are as follows:
- First, it's strongly recommended to set up HTTPS and redirect all server traffic to HTTPS.
- Set up static file handling for the
/media
and/static
urls (or whatever the equivalent is for your specific Django project). - If applicable, set up request redirects for your project, where needed.
- Define your core UWSGI request handler, preferably via reusable location blocks.
- Finally, if using something that needs websocket request handling (such as DjangoChannels, then we need an additional handler. The following code snippet passes requests to Daphne for DjangoChannels handling (this assumes websockets are all served under a root url of
example.com/ws/
):
/etc/nginx/sites-available/<project_name>.conf
# Send websocket connections to Daphne. location /ws { proxy_pass http://unix:/run/daphne/<project_name>.sock; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-OP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $server_name; }
Systemd Service File Configurations
The systemd service files are effectively how Linux determines what processes should start on boot. They are located in the /etc/systemd/system/
folder.
For our project, we will create two files here:
/etc/systemd/system/daphne.service
[Unit] Description=Daphne server for Django web projects After=syslog.target network.target uwsgi.service [Service] ExecStart=/opt/env/<virtual_env_name>/bin/daphne -u /run/daphne/<project_name>.sock <project_settings_folder>.asgi:application Restart=always Type=simple WorkingDirectory=/srv/web_serve/<project_name> User=www-data Group=<group_name> StandardOutput=syslog StandardError=syslog KillSignal=SIGQUIT [Install] WantedBy=multi-user.target
/etc/systemd/system/uwsgi.service
[Unit] Description=uWSGI Application Server in Emperor Mode After=syslog.target network.target [Service] ExecStart=/opt/env/<virtual_env_name>/bin/uwsgi --emperor /etc/uwsgi/sites Restart=always Type=notify User=www-data Group=<group_name> StandardOutput=syslog StandardError=syslog KillSignal=SIGQUIT NotifyAccess=all [Install] WantedBy=multi-user.target
Once the files are created, tell the system to reload:
sudo systemctl daemon-reload sudo systemctl start daphne sudo systemctl enable daphne sudo systemctl start uwsgi sudo systemctl enable uwsgi
Check for possible errors with:
journalctl -xe
If no errors occur, then everything is good and your project is officially serving in production. Verify by trying to access your site in a browser.