Hacks by Ruddra

Deploy Django, Gunicorn, NGINX, Postgresql using Docker

Deploy Django, Gunicorn, NGINX, Postgresql using Docker

This post mainly based on this blog: https://docs.docker.com/compose/django/.

I will be extending this post by serving django+gunicorn using Nginx, also I will using Postgresql docker container to use it as database.

Steps

Lets not waste time and go to the following steps.

1. Let’s make an empty directory named myproject and add another folder inside name it src. src should contain the django project. For testing purpose lets put a simple django project inside named mydjango.

2. Let’s create a subdirectory inside myproject and name it config. Lets put a requirement.pip file inside config and write these line in it:

Django==1.10
gunicorn==19.6.0
psycopg2==2.6.2

3. Now let’s make a Dockerfile inside the myproject. This should contain the following lines:

FROM python:3.6 ENV PYTHONUNBUFFERED 1 RUN mkdir /config ADD /config/requirements.pip /config/ RUN pip install -r /config/requirements.pip RUN mkdir /src; WORKDIR /src

So this Dockerfile starts with a Python 3.5 based image. Then the container is modified by adding the requirement.pip file in /config directory within the container and installing the packages from it.

4. Let’s create a file called docker-compose.yml in myproject directory.

The docker-compose.yml file describes the services that make your app. Here we need a web service(Django+Gunicorn), A database(Postgres), and Proxy Server(Nginx). It also describes which Docker images these services will use, how they will link together, any volumes they might need mounted inside the containers. Finally, the docker-compose.yml file describes which ports these services expose. See the docker-compose.yml reference for more information on how this file works. Don’t forget to add docker-compose to your python environment by running pip install docker-compose.

5. Let’s add the following configuration to the docker-compose.yml file:

version: "2"
services:
  nginx:
    image: nginx:latest
    container_name: ng01
    ports:
      - "8000:8000"
    volumes:
      - ./src:/src
      - ./config/nginx:/etc/nginx/conf.d
    depends_on:
      - web
  web:
    build: .
    container_name: dg01
    command: bash -c "python manage.py makemigrations && python manage.py migrate && gunicorn mydjango.wsgi -b 0.0.0.0:8000"
    depends_on:
      - db
    volumes:
      - ./src:/src
    expose:
      - "8000"

  db:
    image: postgres:latest
    container_name: ps01

It says that there are three services for this project: nginx, web, db. nginx depends on web, web depends on db. db container uses postgres’s latest image from dockerhub. Default username for db is postgres and password is postgres web container is build using project’s Dockerfile. It mounts src directory into it and exposes port 8000. version is being used for which format to use to compose the docker file.

nginx uses nginx’s latest image from dockerhub. This proxy server is accessible from port 8000. It mounts src and config directory.

6. Now let’s write a nginx configuration config file named mydjango.conf inside myproject’s config folder and put it in a subdirectory named nginx.

upstream web {
  ip_hash;
  server web:8000;
}

# portal
server {
  location / {
        proxy_pass http://web/;
    }
  listen 8000;
  server_name localhost;
}

So what it does that, nginx acts as a reverse proxy for any connections going to django server and all connections goes through nginx to reach django server.

Project Directory should look like this:

── myproject
    ├── src
    │   ├── mydjango
    │   ├── manage.py
    ├── config
    │   ├── requirements.pip
    │   ├── nginx
    │      ├── mydjango.conf
    ├── Dockerfile
    └── docker-compose.yml

7. To communicate from django to postgres, we need to put database configuration in django applications settings file. It should look like this:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'postgres',
        'USER': 'postgres',
        'HOST': 'db',
        'PORT': 5432,
    }
}

8. All is done. Now lets run docker-compose build in terminal within the project directory. It will build/rebuild(if necessary) all the containers. For first time running the containers, run docker-compose up -d. Lets go to browser and type: localhost:8000. We should see the django application up and running.

9. For stopping the docker, run docker-compose stop. Re-running docker, use docker-compose start.

10. For shell accessing.

#Nginx
docker exec -ti nginx bash

#Web
docker exec -ti web bash

#Database
docker exec -ti db bash

For logs:

#Nginx
docker-compose logs nginx
#Web
docker-compose logs web
#DB
docker-compose logs db

Thats it.

Example Source Code

You can see an working example here in my repo: https://github.com/ruddra/docker-django

Also another deployment example for Ruby on rails here: https://github.com/ruddra/deploy-notebook (Thanks to Akimul Islam for the source)

Cheers!!


Update

Serving django with gunicorn won’t allow you to serve static files with it. You need to serve static files seperately. You can follow this post for how to do serve static files using Nginx from docker.

Share Your Thoughts
M ↓   Markdown
Sidhartha Mahato
Friday, Nov 4, 2016

i have tried the your by cloning it from from repo . on gunicron –bind my i am putting my server ip . its not working .please help

Sidhartha Mahato
Friday, Nov 4, 2016

i have tried the example by cloneing the project directly on server but on (gunicorn –bind) i have replaced the localhost with my server ip .but its showing “invalid address” error . i have attached the screenshot ,please check and help !! https://uploads.disquscdn.c...

Ruddra
Friday, Nov 4, 2016

Hi, thank you for mentioning the issue. I have updated the master branch of the github repo with a fix, it should solve the issue. :) Hope it helps.

Chirag Gupta
Friday, Nov 18, 2016

In your docker file 4th step is giving error.Please change it from ADD requirements.pip /config/ to ADD config/requirements.pip /config/

Ruddra
Monday, Nov 21, 2016

Thanks for noticing. Updated post :) .

Sidhartha Mahato
Wednesday, Nov 23, 2016

I have tried the latest code . I have only replaced the “src” with my django project . As i am trying host it on digital ocean server i have replaced “gunicorn bind ” with my server ip .but it is giving error and i have changed the server name to my digital ocean ip address in nginx configuration file . Please help !! https://uploads.disquscdn.c... https://uploads.disquscdn.c...

Ruddra
Friday, Jan 20, 2017

Hi Sidhartha, Sorry for late response. You should not replace the server and upstream name. You should keep it as it is said in the tutorial. :)

Salun Marvin
Monday, Jan 16, 2017

Awesome Tutorial, but I can’t reach my web page. On the web logs It seems that Gunicorn is working normally. django01 | [2017-01-16 00:40:22 +0000] [9] [INFO] Starting gunicorn 19.6.0 django01 | [2017-01-16 00:40:22 +0000] [9] [INFO] Listening at: http://0.0.0.0:8000 (9) django01 | [2017-01-16 00:40:22 +0000] [9] [INFO] Using worker: sync django01 | [2017-01-16 00:40:23 +0000] [12] [INFO] Booting worker with pid: 12 But my Nginx logs only says: Attaching to nginx01 I didn’t create a /config/ folder, just created a django.conf file and in my docker-compose.yml I changed volumes to: volumes: - ./src:/src - ./nginx:/etc/nginx/conf.d Don’t know whats wrong :(

Salun Marvin
Monday, Jan 16, 2017

Well, just changed my structure just to follow yours, and It worked! Thanks, I’ll try to see what was wrong with my structure. Now I’m getting a 502 Bad Gateway error, I’ll try to figure it out.

Sumit Khanna
Tuesday, Feb 7, 2017

hello, good stuff indeed, but I face problems deploying the very same thing on another server. What changes do you think need to be done to the nginx.conf file for the same?

gamesbook
Saturday, Mar 18, 2017

What parts of the setup need to change so that nGinx is available on port 80 to the “outside world” but gunicorn still keeps using port 8000?

Ruddra
Saturday, Mar 18, 2017

You need to change in `server` section

Sidhartha Mahato
Monday, Mar 27, 2017

My requirement.txt is containing lots of libraries ,So while creating the virtualenv most of the time it gives error,So “RUN pip install -r /config/requirements.pip ” this step fails for me .how to resolve ?

Ruddra
Wednesday, Mar 29, 2017

I haven’t faced this issue yet. For now, I think you can split the requirements.pip into multiple files and run them one after another. Hope that will resolve the issue.

Rohini Chaudhary
Friday, Mar 31, 2017

Thanks alot for you post!

disqus_iJAJ3VREVG
Tuesday, Apr 11, 2017
Wataru Oguchi
Wednesday, Apr 12, 2017

Thank you for the post! The tutorial might be changed(Apr.2017) since you wrote this post: https://docs.docker.com/compose/django/. I have made it work reading your post. Here is my repo: https://github.com/wataruoguchi/setup_app/tree/master/docker_compose … Again, thank you so much!

Jesus
Friday, May 26, 2017

Thanks a lot for this post, it´s much better than the official one, I am totally new on this and I have a question (maybe it´s a silly question). When you setup Django with PostgreSQL each one has to be on it´s own server or virtual machine, I am trying to setup this but I am getting errors: could not connect to server Connection refused web_1 | Is the server running on host 'localhost' (127.0.0.1) and accepting web_1 | TCP/IP connections on port 5432?

Jesus
Sunday, May 28, 2017

I finally made it work!, thanks again!, I have just a little problem, how can I run the migrations? because if I write makemigrations without docker (just python manage.py makemigrations) I am getting an error because `‘HOST’ ‘db’ is not found

Ruddra
Sunday, May 28, 2017

Hi, sorry I couldn’t reply on your earlier comment. Anyways, you have go to the docker’s shell and run this command. Run `docker exec -ti web bash` and go to the directory containing manage.py and run the migration commands. Hope it helps

Jesus
Thursday, Jun 29, 2017

Hello Ruddra, I´ve been working only with Django and Postgress so far, I followed your tutorial before and it helped me a lot, now I am about to integrate gunicorn and Nginx, but I noticed your docker-compose file uses version: 2, is there a difference if I am using version 3? I guess the part that says bash -c 'python manage.py makemigrations && python manage.py migrate && gunicorn mydjango.wsgi -b 0.0.0.0:8000'. I am going to get it from the DOCKERFILE through an image, and I have seen in some tutorials they use a file called entrypoint (which I don´t really understand but can learn if this is the right path). Another question is, I have been looking and comparing different tutorials, and sometimes in the DOCKERFILE I see from Ubuntu 14.02 (or any version) and sometimes I see from Python 3.6 (or any version), why is this? I am really new at this so excuse me if this is a silly question

Salomão Pinheiro Coelho Júnior
Saturday, Aug 5, 2017

Hello there. I kind have this working. In your Dockerfile you just need insert a CMD command, just like this: CMD ["gunicorn", "app.wsgi:application", "--name", "app", "--workers", "3", "--user=www-data", "--group=www-data", "--log-level=debug", "--bind=unix:/var/www/app/run/gunicorn.sock"\] and you have your gunicorn runing inside your app-container. Remember that your Dockerfile can have only one CMD command. The “location” in your nginx is the gunicorn.sock location / { include proxy\_params; proxy\_pass http://unix:/var/www/app/run/gunicorn.sock; }

Ali Yaman
Friday, Aug 4, 2017

DockerFile?

vijay
Tuesday, Aug 8, 2017

I getting this err: python: can’t open file manage.py’: [Errno 2] No such file or directory full output: ttaching to TWVpsql, TWVdjango, TWVnginx WVpsql | LOG: database system was shut down at 2017-08-08 07:29:00 UTC WVdjango | python: can't open file manage.py': [Errno 2] No such file or directory WVpsql | LOG: MultiXact member wraparound protections are now enabled WVpsql | LOG: database system is ready to accept connections WVdjango exited with code 2 WVnginx | 2017/08/08 07:29:15 [emerg] 1#1: host not found in upstream 'web:8000' in /etc/nginx/conf.d/TWV.conf:3 WVnginx | nginx: [emerg] host not found in upstream 'web:8000' in /etc/nginx/conf.d/TWV.conf:3 WVnginx exited with code 1

Rene Dohmen
Wednesday, Nov 22, 2017

do you have a manage.py in src/ If that’s absent you don’t have a correct django app in src/

Yiğit Genç
Wednesday, Aug 16, 2017

Great tutorial, thanks!

Issam Hammad
Monday, Aug 21, 2017

Hi, Thanks for these instructions. I was able to run my web locally using Docker Terminal on Windows with ‘docker-compose build’ & ‘docker-compose-up’, However, After creating the image, (Image created Successfully) I am not able to run the web from the image itself I used docker run -p [newport]:8000 [image], looks to me the container immediately exists after executing, when I try to browse the new port nothing is there .. I did netstat -t , no binding is there to the new port… Any idea whats wrong? I am new to Docker.

bed777
Friday, Oct 13, 2017

Hi there, which is your advice to differentiate the local.py environment with the production.py enviroment? (i.e different credentials for the database?). Thanks!

Ruddra
Friday, Mar 16, 2018

I personally use this: try: import * from .local_settings except ImportError: pass local_settings.py can be put in gitignore. You can put your DB configs and other server related stuff there.

Rovshan Musayev
Tuesday, Nov 28, 2017

Anybody has such problem?

Ruddra
Friday, Mar 16, 2018

Its fixed now. Please pull the latest code.

ncdude
Tuesday, Dec 5, 2017

Thanks for the tutorial! My nginx produces no logs at all and cannot connect to it using a browser. Anyone else having this?

Joost Slijkoort
Tuesday, Jan 16, 2018

Great tutorial! Only thing you missed to explain is setting ALLOWED_HOSTS = [‘web’] to get your app running.

guest
Friday, Mar 16, 2018

I am using docker 3 with django 2.0.3 and I have my static files within a static folder in my app. I also have my html files in a template folder within the app directory. I used your code here as well as in your static file serving tutorial, however, I cannot seem to get the static files to be served using your code. Also using your code, I cannot seem to have nginx run as it says the port has already been allocated. When I change the port, it just says bad gateway. Help please.

Ruddra
Saturday, Mar 17, 2018

Its hard to say anything. Its better if you check the logs and pinpoint the bugs. Also check the code in config>mydjango.conf in given repository with this Blog.

guest
Friday, Mar 23, 2018

Okay, i returned the nginx port to 8000 as you say, however, I keep getting this error: https://pastebin.com/8xwAxzbb. I cannot login to the containers shell for I cannot get past the error.

Carlos
Wednesday, May 16, 2018

If it’s not working as is, check in his git repo the nginx conf file and copy it to your files, that made it work for me.

dayanand124
Tuesday, Jun 5, 2018

instead of command: bash -c "python manage.py makemigrations && python manage.py migrate && gunicorn mydjango.wsgi -b 0.0.0.0:8000" the following line is correct. command: bash -c "python manage.py makemigrations && python manage.py migrate && gunicorn myprojet.wsgi -b 0.0.0.0:8000"

joseaminan
Wednesday, Jun 13, 2018

Congratulations for this tutorial! really interesting

Dan
Friday, Jun 15, 2018

Thanks for the awesome 3x blogs / tutorials. I am trying to get the package from ytour github up and running. “make build” and “make up” appear to work without error. However visiting localhost:8000 gives me an nginx 502 bad gateway. Shell-DB and Shell-Nginx gets me on no problem. “make shell-web” gives “Error response from daemon: Container 3875b0c01522e8736fbb990f9f62c8b3447d5990f013b6969bac5d5a1cca6648 is not running” Any advice where to point me? Apologies for the question… its my first time playing with containers… i am excited to get my django project containorised.

Dan
Friday, Jun 15, 2018

this is my docker ps: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 032e46e131f2 nginx:latest “nginx -g “daemon of…” 16 minutes ago Up 12 minutes 80/tcp, 0.0.0.0:8000->8000/tcp nz01 9d7049646279 redis:latest “docker-entrypoint.s…” About an hour ago Up 12 minutes 0.0.0.0:6379->6379/tcp rz01 2f71f22010e7 postgres:latest “docker-entrypoint.s…” About an hour ago Up 12 minutes 5432/tcp pz01

Adiyat Mubarak
Friday, Jun 22, 2018

Hi, I’ve tried to follow your tutorial and successfully build the docker project. $ docker-compose start Starting db ... done Starting web ... done Starting nginx ... done But in docker ps I only got this: cccd210ee314 postgres:latest ‘docker-entrypoint.s…’ 4 minutes ago Up 8seconds 5432/tcp ps01 Seems like only my postgres db running, and localhost:8000 is not working

Ruddra
Friday, Jun 22, 2018

most probably its for psycopg2. Please use 4.1.1 or greater.

さりん
Friday, Jul 20, 2018

Since this method does not check for the domain is it subseptible to Cross-Site Request Forgery? https://docs.djangoproject.com/en/2.0/topics/security/#host-header-validation

Ruddra
Friday, Jul 20, 2018

You can put your preferred domains in ALLOWED_HOSTS or you can look into this package as well: https://github.com/ottoyiu/django-cors-headers

Виктор Ди
Friday, Aug 3, 2018

the app was not able to connect to db until I renamed the containers: web, nginx and db.

Dan
Monday, Aug 6, 2018

This tutorial is great - was very useful. Has anyone got any resources for deploying this setup to AWS…. I have never done that myself. Would be cool to have a bit of a walk-through if anyone knows of a decent one.