Introduction, or what this is all about
In this article, I will tell and show how you can host a site on servers from Beget (hosting provider). This provider offers a wide range of development platforms/CMS that are used to create a site. But in this article, I will only tell you how to deploy a site written in Django, because I understand this.
In addition to what was said earlier, I also want to show how you can set up your project/site in such a way that if you need to implement new features or fix the old ones, it is way easier than pressing one button. We will do this using such things as Bash, Git, and SSH.
In addition to providing regular hosting, Beget also provides virtual private servers (VPS).
So, on Beget, you can host a site in two ways. The first is simple but limited; the second is complex but also much more interesting. The first is to use a ready-made solution - hosting, plus some magic in Docker. The essence of the second method is to create your own VPS and configure the server manually.
I will demonstrate both, explain the features and nuances of both, and why, in my opinion, VPS is better. If you wish, go straight to the comparison here to choose which suits you better.
Features of this article
For demonstration, I will use a specially made django application. This application has built-in all the features that the site may need in the future, such as:
- working and setting up with the database
- sending email
- working with static files
- uploading various media to the site
- translations for backend and frontend
- tests both functional and modular
I am also a supporter of such an approach to application development as TDD (Test Driven Development).
Why do you need to know this? You should know, because instead of quickly deploying one site to one machine with one domain, I will deploy two sites (one test, as close to the production one as possible - staging.bddt.space, the other production one - bddt.space), to one machine with one domain and one subdomain for the test site.
And also, if you don't need a test component for the site, you can just read the chapter on preliminary preparation + deploying the site on the hosting (or deploying the site on the VPS before the chapter on deploying the live server (exclusively). Also, if the title of the chapter contains - optional, you can safely skip it.
Preliminary preparations
In the next three chapters, you will need to browse the beget website itself and complete some preparatory steps that are common for deploying a website via hosting or via VPS.
Buy a domain and create a subdomain
From the very beginning, you will need to have already paid for Beget hosting, plus to buy a domain. On the page at https://cp.beget.com/domains/register enter any domain name you like and click continue:
Afterwards, you will be redirected to the domain settings page. Here's what you need to know:
- This is your domain name
- We don't redirect you anywhere
- Afterwards, we will need to edit several DNS records for the domain to work correctly
- As a bonus, we will receive a free SSL certificate. This will be required to switch from the http protocol to https.
I got my second domain for free - bddt.space. I will use it to demonstrate how to add a subdomain for it and how to link it to a site (whether it is on a VPS or on the main hosting)
Let's add a subdomain for a test site. If you remember, I am going to have both a production server and a test server hosted on a subdomain. Let's add a staging subdomain:
Now we have a domain and a subdomain for deployment. Remember, if you don't need a test component of the site, don't add a subdomain.
Create a database
Go to the database management page and create one:
- Your database name (For me it will be timachuduk_bddt)
- Password, remember or write down (I won't tell mine)
- Create
- Necessary for connecting sites created for VPS
That's it, the database is ready for use via hosting, but not via VPS. The thing is that you still need to allow the connection of remote addresses. To do this:
Now you can connect to it from a remote address, and not only from the same machine on which this database is located. When we get to the section on writing a configuration file, there, the data that was generated in this chapter will be useful to us.
Creating a mail server
Go to the mailbox management page. There you will see the addresses of your sites, click on yours:
Afterwards, you will need to add a new mailbox.
- Enter the desired name (without @., etc.)
- Generate a password (save it)
- Remember the full name of your mailbox, it will come in handy later
This completes the basic creation of a mailbox. Next, I will show you where and what to write when configuring the site's mail server.
Deploying a site on hosting
Preparation or linking a domain to a site
I want to start with a simple method. First, we need to create a site and link a domain to it. Everything is done on the page https://cp.beget.com/sites:
Also, after creating the site, specify a redirect from HTTP to HTTPS in the settings. After all, we want to have a secure connection. I will create 2 sites, one for bddt.space and another for staging.bddt.space. Again, if you do not need test functionality, you can create only one site.
Connecting to the hosting via SSH
To be able to work with a remote server, we need to connect to it. There are several ways to do this, the easiest is to use the built-in commands and utilities of the hosting itself. But I will use a more old-school method - SSH.
To do this, use the following command in the terminal:
Where instead of USERNAME_PLACEHOLDER use the name of the created user (or login). After that, you will need to log in to the docker:
After we have successfully entered the docker, we will create and enter the tmp directory, where we will install the necessary packages:
Installing OpenSSL
For correct operation of Django applications starting from version 4.4, it is necessary to use OpenSSL version 1.1.1 and higher. Let's install it and unzip the resulting archive:
Now that we have this archive, we will first need to configure the file for compilation (make file), and then directly compile and install (using the previously generated make file):
To check if openssl was installed successfully, run the following command (it should return the openssl version, i.e. 1.1.1l):
Install Python
After openssl is installed, you can proceed to installing python. It is important to note that python version 2.7 is already installed on the system. But, since I personally do not like this version, and it is also very outdated, and almost no one uses it anymore, we will install python version 3.11.0.
Now it's Python's turn. Let's go back to ~/.beget/tmp, download, and then unzip:
We have successfully downloaded and unzipped the Python application, now we will need to generate a make file for subsequent installation on the system:
To check if everything is installed and set up correctly, you can run the following command:
We prepare the file structure of the site, transfer the project to the server
We have everything we need to deploy the site to the hosting. To do this, we will turn to the git repository and download our site to the source directory. But first, let's go to the right directory. At your home (~), a corresponding directory should have appeared under the name of the domain that you assigned to the site at the very beginning. In my case, it will be staging.bddt.space, let's go there and clone it:
Now let's create a virtual environment for this site and install all the necessary packages for the site to work (after activating it of course):
Before installing any packages, it is better to make sure that you have activated the required virtual environment, this is done using the which command, which will indicate the path of the executable command. So, we find out which python we are using by:
And if, for example (/home/t/timachuduk/staging.bddt.space/venv/bin/python), you got a path that contains the name of the virtual environment, then everything is good and you have successfully activated the virtual environment. You can confidently install the necessary packages:
Configuration of server files (.htaccess, passenger_wsgi.py, settings.py)
Now we need to set up some special files for the correct operation (and operation in general) of the site. I'll start with the file passenger_wsgi.py.
The passenger_wsgi.py file
This file will connect your django application with the apache server. This file is required for Passenger - a program for deploying Python applications on the server (the closest analogue is Gunicorn).
It also needs to be created in the root directory of the site, i.e. ~/staging.bddt.space:
And it still needs to be configured:
- Where the first comment specifies the path to the root of your project (this is where the manage.py file is located).
- Where the second comment specifies the path to the installed packages via the virtual environment.
- Where the third comment specifies the path to the settings.py file relative to the first path specified.
The settings.py file
When you have finished creating and configuring the file for Passenger, you can edit the settings.py file. We will only need to add the staging.bddt.space domain to ALLOWED_HOSTS.
I do not change the configuration files in settings.py directly. I do this through the settings.json file, which is then read at the start of the application itself. The values of which are added to the constants I need.
How do I transfer data from settings.json file to Website/settings.py file? In the settings.py file, I open the settings.json file and read the data from there, like this:
I use the JSON format because it is very convenient to work with it via Python. This way I can safely push this file (settings.py) to the commit without worrying that I might accidentally put sensitive data there.
The .htaccess file
We need to tell the Apache web server how and through what to launch our Django application. This is done via the .htaccess file. We will specify that we want to use Passenger and specify which version of Python it will use:
This file should be in the root directory of the site, i.e. where the public_html directory is.
Configuration of static and media files
The last thing left to configure is the places where static and media files will be located. The thing is that the Apache server has access only to the public_html directory, but not to the Django application directory. The opposite is also true: the Django application does not have access to the public_html directory ...
Therefore, when your site processes statics and tries to upload media files to the server, it needs help. This can be easily done by creating soft links, like this:
And of course you need to create the corresponding directories in public_html:
Finishing the site deployment
And the final touches to complete the site deployment are the following commands:
- Database migration [./manage.py migrate]
- Collect static files [./manage.py collectstatic]
- Compile translations (if any) [./manage.py compilemessages]
- Check tests (if any) [./manage.py test]
After successfully completing each of them, you can consider that the site has been successfully deployed.
Extra (optional, though not really)
In the following chapters, I've described extra steps you can take on your new site to improve its performance, security, or simply to update changes you've made faster.
How to restart a Django site if you made changes to it?
The server is almost ready, but first you need to update the server, or rather, restart our Django application. To make Passenger do this, you need to create another tmp directory and create a restart.txt file in there:
Now, to restart Passenger, you only need to recreate the restart.txt file.
How to transfer a local database to a server?
Sometimes, you want to keep the server in production and development synchronized. That is, so that they operate with the same database. And sometimes you want to save your work and protect it from failures and unexpected losses.
To do this, you can make a database dump and load it, already on the server. Django has a built-in function that allows you to do everything described above.
A database dump (data.json) will be created, which can be moved to the server using the scp command (not to be confused with the w(°ｏ°)w fund). The --exclude flag is needed to exclude certain tables from the export.
In order to successfully transfer a file from a local machine to a server, you will need to know the server address and the location where you would like to send the file. Where timachuduk is your login when creating, and timachuduk.beget.tech is the address of your server on beget. All this can be found on the main page of the hosting.
Once on the server, you will need to run the database load command (having previously entered the directory with the manage.py file inside.)
How to transfer a site to HTTPS protocol?
You probably noticed that your site is currently running on HTTP protocol, which is not very encouraged these days - neither by users, nor browsers, nor search engines. Let's fix this.
We have already issued a certificate for the bddt.space domain, but not for the subdomain:
- We select the type of certificate, a free one will do for us.
- We select the subdomain to which we want to extend the certificate.
You will need to wait for some time. On the site itself they say that it can take from 15 minutes to 2 days. So yeah ... I installed everything in 10 minutes, by the way.
After that, you need to set up a redirect from http to https. To do this, on the site management page https://cp.beget.com/sites you need to:
- Select the site for which to set up a redirect
- Enable redirect
This may again take some time, about 15 minutes. But if all the previous steps were done correctly, switching to the https version should not be a problem.
Deployment of a website on VPS
I'm glad you decided to go maybe not the easiest, but definitely the most interesting way. Even more, if you could learn how to deploy a site on a VPS from beget, then you will be able to deploy a site on other VPS, regardless of the hosting provider.
Let me give you a short overview of what you should know or what you will learn while reading this chapter:
- Working in a Linux terminal
- Working through the SSH shell
- Working with services and daemons through systemctl
- Configuring and working with the Nginx server
- Configuring and working with Gunicorn
- Writing and executing Bash scripts
Creating a VPS on beget
First, let's create a VPS and configure it. In your account, on the tab with the cloud, click create VPS.
Next you will see the configuration page where you should select Ubuntu and configure the server parameters (number of cores, disk space, RAM). You can also add your SSH key for password-less access. I wrote a separate article about how to make such a key and how to use it.
We have successfully created a virtual machine, now we need to link the bddt.space domain with created virtual machine. To do this, we need to edit the A-record in DNS. Copy the IP address of your VPS and edit the A-record.
Copy IP-address
Select your domain, then edit A-records in the required domains and subdomains
I found this cool infographic of DNS records that briefly shows what each record is used for. Not all records are shown here, just the most used ones:
The source - https://www.nslookup.io/learning/dns-record-types/
Connecting to VPS
After creating the server, you will need to configure it a little. Just a little. Install the required minimum of packages, plus create a user on behalf of which the site will be launched.
To do all this, you first need to get to it. We will do this via SSH.
I will use PowerShell as a terminal shell through which I will connect to the server. It does not matter at all what you connect to the server through. It can be Git Bash, PowerShell, PuTTy or something else. The main thing is that you have an SSH client installed on your machine.
To connect to a remote server, open your favorite terminal and enter the following command:
Or, if you uploaded an authentication key when creating your VPS, you can use it (you need a private one):
- Where root is the username through which you want to access the server.
- Where 192.168.1.1 is the address of the server you want to connect to.
- Where .ssh/rsa_key is the path to the private key for connecting without a password
All this can be found on the page of the previously created VPS. Well, except for the key, when you created it, you had to remember where you saved it :). Usually this is the .ssh directory.
Installing the necessary packages
Now it's time for Ctrl-C, Ctrl-V. There will be a few commands that will install the necessary packages on your server:
Now let's talk about why I installed all of this, and what each of these commands did:
- apt-get update - Always run it first when installing any package. It syncs the application package index and updates all dependencies if necessary.
- apt-get upgrade - Installs the latest versions of installed packages on the system.
- apt-get install gettext libgettextpo-dev - it installs the dependencies needed to generate translations for your sites (which use the gettext utility)
- apt-get install pkg-config default-libmysqlclient-dev build-essential - necessary dependencies for working with MySQL databases.
- apt-get install nginx - to start the web server
- apt-get install gunicorn - to redirect requests from the nginx server to our django application
- apt-get install python3-dev python3.12-venv - to be able to create a virtual environment when running python applications
My "unnecessary" package is selenium. I need it for my functional tests to work. And for it to work, it will need the Firefox browser, which we will install as a .deb package. Oh boy, this is quite an adventure ヽ(°〇°)ﾉ. I invite everyone interested to visit the official Mozilla website and run one command after another.
In addition to firefox, you will also need to install geckodriver. The latest releases can be found here: https://github.com/mozilla/geckodriver/releases . And now, the command to install this geckodriver:
After downloading (wget) and unzipping geckodriver (tar -xzvf), we then placed it in the /usr/local/bin/ directory so that all applications had access to the driver.
Create a user
This step is mandatory and do not think about making a site with root rights. Now you will need to create a user with root privileges and a home directory.
The first command will create a user named "site", create a home directory for it (flag -m) and set the default shell, i.e. bash. The second command will set a password for the new user. The third will add the user "site" to the sudo group, which is able to execute all commands, only using the sudo command.
Transferring the project to the server
The base for our server is ready, now let's work on the site files themselves. I'm going to create a simple file structure where I'll get my project files using git and copy them to the source directory. I'll also create a virtual environment:
After running all these commands, you should have the following project structure:
- staging.bddt.space <- You are here
- venv
- source
Now, activate the virtual environment and install the necessary packages from source/requirements.txt:
There shouldn't be any problems here, but if they do, read the error logs carefully. Often, these are simply uninstalled dependencies on the server, which can be easily fixed using the apt package manager. Let's try to run the installed Django application (from the directory with the manage.py file):
Setting up mysql database and mail (optional, though not really)
You will get an error that the settings.json file was not found. This is a configuration file that I use as a proxy instead of writing directly to settings.py. I did this so that I could safely add the settings.py file to the git repository and to simplify deployment.
Here is an example of such a file (settings.json), create it in the source/Website directory:
As you can see, I'm already using MySQL as a database. What you need to know to connect to the database from a VPS:
- First, you need to know which backend to connect to - django.db.backends.mysql
- Second, the username and database, they will be the same - timachuduk_bddt
- Third, the database password - xW%exys3ORGS
- Fourth, the database hostname - you can find it on the database setup page
- And fifth, the connection port - it is always the same - 3306
All this data can be obtained when creating your database, the process of which I described in the corresponding chapter at the beginning.
I also have mail configured. Let me tell you what each of these variables is for:
- DEFAULT_FROM_EMAIL is the email address on behalf of which we send messages. Optional
- DEFAULT_TO_EMAIL is the email address to which the message is sent. Optional
- EMAIL_HOST is a constant, smtp.beget.com
- EMAIL_PORT is a constant, 25
- EMAIL_HOST_USER is the name of the previously created mailbox, bddt@bddt.space in my case
- EMAIL_HOST_PASSWORD is the previously set password
How do I transfer data from settings.json to Website/settings.py? In the settings.py file, I open the settings.json file and read the data from there, like this:
I use the JSON format because it is simply very convenient to work with it via Python.
Finishing up the project transfer
By the way, we can already run tests and see what works and what doesn't. Everything should work:
So, we have successfully uploaded our site to the server and even passed all the tests. This means that it is time to configure nginx, after which we will connect our site to the nginx web server via gunicorn.
Configuring Nginx
Along with the project files, you will also receive default configuration files for configuring nginx - nginx-server-http_only.conf, nginx-server-https_301.conf, nginx-server-https.conf. For now, open the nginx-server-http_only.conf file, you will see the following configuration for our site:
- In this configuration, we set up the port on which we will listen for incoming requests.
- The name for our server (domain name). In my case, it should be staging.bddt.space.
- The folders that this server will serve. At a minimum, you need to set up the root directory "/". But this is not all, my site uses various statics in the form of pictures and icons. Plus media files that are uploaded directly by the user.
So the final version of this configuration will look like this:
To make sure we don't forget, let's immediately create the media and static directories in the ~/staging.bddt.space directory and soft links to them. Soft links to them are created to give the server access to them.
All the necessary directories have been created and now we are ready to create soft links to these directories:
The project now has the following structure:
- staging.bddt.space
- venv
- media
- static
- source
- Website
- Backend
- Frontend
- Website
- media -> ../../media
- static -> ../../static
- manage.py
- settings.json
- deploy.sh
- server-setup.sh
- requirements.txt
- gunicorn.service
- nginx-server-http_only.conf
- nginx-server-on_https-301.conf
- nginx-server-on_https.conf
Now, any file that is created in the ~/staging.bddt.space/source/Website/static or ~/staging.bddt.space/source/Website/media directory will be available in the ~/staging.bddt.space/static and ~/staging.bddt.space/media directories.
To check this, you can run the collectstatic command and make sure that all files, although copied to ~/staging.bddt/source/Website/static, are also available in ~/staging.bddt.space/static.
403 Forbiden error - how to solve
Another very common problem is that the server can return a 403 response to a request for some static file. This is often due to the fact that the user www-data does not have permission to read certain files. The nginx configuration file specifies the user on whose behalf the web server is launched.
There are three ways to fix this:
- Replace the user in the /etc/nginx/nginx.conf file
- Give everyone access to read files
- Add the user www-data to the group of the user who launches the application.
I will describe the 3rd because this is, in my opinion, the simplest and most correct way. And you just need to run one command:
- site - the name of the group to which you want to add the user (it will match the user name)
- www-data - the name of the user you want to join the group.
Now, all statics and all media files on the site should be available.
Finish up a Nginx set up
Let's finish setting up the web server and copy the created configuration file to /etc/nginx/sites-available:
Now let's do the same as with the static and media directories, create a soft link to this configuration file. But first, go to the directory where this link should be:
Let's restart the nginx service using systemctl so that the changes take effect:
I think it's worth trying to visit our website now - staging.bddt.space.
And yes, you should see something like this. Why is this happening, what is this "Bad Gateway"? It's all because one line, in the site configuration file /etc/nginx/sites-available/staging-nginx-server.conf.
In fact, the nginx server forwarded my GET request to the specified socket, but nothing is connected to it yet. This is where gunicorn comes into play.
Seting up a Gunicorn
The almost ready configuration file is located in the same place as the nginx server configuration file - ~/staging.bddt.space/source/gunicorn.service. This is a file describing the work of the linux daemon, here are its contents:
This service (daemon), launched by the user site and with its privileges, will be rebooted every time an error occurs (Restart=on-failure). This daemon will work in the directory specified in WorkingDirectory. And it will do what is specified in ExecStart, that is, launch the gunicorn server.
When launching gunicorn, we also specified which socket to connect to and where to redirect requests - to our Django application.
Create the same file in /etc/systemd/system/ and edit it by inserting your user, your domain and your socket. Now let's launch the service and make it start on the server after boot.
Now everything will work. We have created a special service that starts the gunicorn server at startup, and it starts our django application. After refreshing the tab, you should see the following:
Migrate to HTTPS (optional, although not really)
In general, initially, I thought that there was some way to get (move) the generated certificate from Beget to VPS. But I did not find such a way.
Therefore, I decided to get a certificate from LetsEncrypt directly. It is free, but what is even cooler is that everything can be done through the command line, which means automation. Look, in order to configure the nginx web server, you will need a certificate and a key to it.
The LetsEncrypt team made a special utility, which is very cool in terms of supported technology stacks for certification. Here, for example, is a page for generating, signing and issuing certificates for sites on nginx and using pip. It is just dope ;) And you do not need this headache with self-signed certificates that no one trusts, and as a result, no one visits the site.
Activate the virtual environment and install the program:
Now we generate and sign up the certificates:
This command will generate and sign up certificates, which will be placed in /etc/letsencrypt/live/staging.bddt.space. You will see an interactive prompt like this:
- Asks to enter email
- Agree to sell your soul and the soul of your site)
- Registration, optional
- Selecting domains (subdomains) for which this certificate will be valid. If you simply press ENTER, the certificate will be applied to all.
- Path to the certificate (for configuring nginx)
- Path to the key (for configuring nginx)
You can also set a cron task to update the certificate every month:
All that remains is to change the configuration in sites-available, this is done like this:
The difference from the previous configuration is that we changed the port and added ssl_certificate and ssl_certificate_key and the paths for which certbot will tell you. That's all, on how to add https to your site.
But you may have noticed that when trying to get your site via regular http, it will no longer work. You will need to make a separate configuration for port 80 and make a 301 redirect to the https version. This is considered good practice. To do this, create another configuration file in /etc/nginx/sites-available, for example http_301_redirect-staging.bddt.space, with the following:
Where HOST_PLACE_SETUP is your site's domain, for example staging.bddt.space. This configuration makes a 301 redirect from any http pages to https pages.
Deployment of a production server (optional)
The hardest part is over. And if you just wanted to deploy a site without any test versions and subdomains, then you can stop here. After all, I have nothing more to say about this. The site was deployed, the mail service and databases were connected, nginx and gunicorn servers were configured, the HTTPS protocol was connected.
Well, if you, like me, wanted to deploy a test site first, let's continue.
In fact, to deploy a production server, you need to do the same as with the test one, only select the appropriate domain. As you already understood, this is not the most fun activity, so I wrote 2 bash scripts that will help in setting up the server - server-setup.sh and in preparing the file system and the project as a whole - deploy.sh.
I will not describe the features of these scripts, because I wrote about them separately, see the links. In fact, the deploy.sh script describes the chapter "Transferring a project" with some features of git. And the server-setup.sh script describes the chapters: Setting up Nginx, Setting up Gunicorn and Migration to HTTPS.
First, let's run the server-setup.sh script:
- bddt.space is the domain of the production server
- on_https is a flag that tells the script to place the site directly on the HTTPS protocol
After running it, you will see something like this:
Now deploy. The deploy.sh script does just that. It works in tandem with git, so it's a good idea to have it installed. If the specified site isn't on the server yet, it will clone it from the repository and configure it accordingly. And if it already exists, it will update it to the latest commit.
- Where bddt.space is the domain of your production server.
- Where --skip-tests is skipping tests
This is what the deploy.sh script will output when running:
That's it, we've finished deploying the production server. And now our workflow will look like this, at least that's how I see it:
- We make some changes to the project on a local server on a laptop (for example)
- Make a commit
- Connect to the server and git pull updates to the test server - staging.bddt.space
- Run tests (Functional, Modular). In the end, we check ourselves to see if we broke anything
- If everything is ok, we run deploy.sh and the changes that were made on the local server are applied to the production server - bddt.space.
Comparison of launching a website on a Hosting and on a VPS
I decided to write a comparison in the form of a table, where I will clearly show what and where is better to do.
| Testing | Not available or limited | No limit |
| Custom project structure | Limited | No limit |
| Automation of deploy, setting up etc. | Limited | No limit |
| WSGI server | Passenger | No limit |
| Web server | Apache | No limit |
| Migration to HTTPS | Seamless | Hard |
| Database setup | Easy | Less easy |
| Email setup | Easy | Easy |
| Linking domain | Seamless | Hard |
| Price | Cheap | Less cheap |
| Metrics | Beget Hosting | Beget VPS |
|---|
I can say that my choice is still to deploy the site on a VPS. It gives me more freedom, but also responsibility. For me personally, a very big cons is the lack of testing as such on the hosting. They didn't install something properly and now you can't do TDD here, and I really want to (╥_╥)
Conclusion
I hope that I'm just wrong and just don't know something and it's still possible to run tests, even functional ones. But if you don't need a test layer or just want to install a site so that it can be seen, then beget is well suited for this. Simple, clear, visual.
And so, I hope that this article helped you figure out how and where to deploy your site written in Django. I also hope for your kind comment and that you might want to share this article with a friend. In any case, see you in the next article (⌒‿⌒)
