Continuous deploy .NET Core website to Linux server with Nginx

February 10, 2017
.NET Core Continuous deployment Ubuntu Nginx Git

One of the great benefits with .NET Core besides its modular design and performance improvements is that it can run on other platforms than Windows. We can finally use C# in over personal projects since it can be hosted in Linux with more reasonable pricing. No more PHP!

I am here going to describe how to host an ASP.NET Core application in Ubuntu 16.04 with auto deployment from Gitlab but it will work with any Git host.

I am later going to explore Docker and build agents in Gitlab and have therefore decided not to take npm and bower into account during the deployment process and those dependencies needs to be added to Git for now.

Table of Contents

• Install .Net Core in Ubuntu 16.04
• Setup the website as an service
• Setup continuous deployment from Git
  • Setup SSH
  • Allow build script to restart services
  • Configure Git-Auto-Deploy
  • Update Git-Auto-Deploy
• Setup http server (Nginx)
• Setup Let's Encrypt with auto renewal
• Ports
• Troubleshooting
  • MailKit
  • Environment variables
  • Environments

Enter your site name to replace hello.world.site with your site name.

Replace

Install .Net Core in Ubuntu 16.04

The official installation instructions are easy to follow. Microsoft has created an apt-get feed with the different versions of .NET Core. All you need to do is to add the feed and install the version you want to use. Run the following commands to install .NET Core 1.1 on Ubuntu 16.04

sudo sh -c 'echo "deb [arch=amd64] https://apt-mo.trafficmanager.net/repos/dotnet-release/ xenial main" > /etc/apt/sources.list.d/dotnetdev.list'
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 417A0893
sudo apt-get update
sudo apt-get install dotnet-dev-1.0.1

Setup the website as an service

Another great thing with .NET Core is that the documentation is well written and easy to follow. Microsoft has even provided best practice instructions on how to publish a website to a Linux Production Environment. I am here only going to show setup my website as a service and setup Nginx after I have setup git auto deployment.

We are going to setup the website as a service so that it will start on boot and having monitoring that restarts the site if it goes down. Create the service file and add the following content:

sudo vi /etc/systemd/system/aspnetcore-hello.world.site.service

[Unit]
    Description=hello.world.site website

    [Service]
    WorkingDirectory=/var/aspnetcore/hello.world.site/
    ExecStart=/usr/bin/dotnet /var/aspnetcore/hello.world.site/hello.world.site.dll
    Restart=always
    RestartSec=1
    SyslogIdentifier=hello.world.site
    User=www-data
    Environment=ASPNETCORE_ENVIRONMENT=Production

    [Install]
    WantedBy=multi-user.target

Enable the service

sudo systemctl enable aspnetcore-hello.world.site.service

You can handle the service with the following commands:

sudo service aspnetcore-hello.world.site start
sudo service aspnetcore-hello.world.site stop
sudo service aspnetcore-hello.world.site restart
sudo service aspnetcore-hello.world.site status

Check the service logs with:

sudo journalctl -fu aspnetcore-hello.world.site

Setup continuous deployment from Git

The preferred way of setting up continuous deployment from Git is to use a build server like Team City or Gitlab that runs all the tests and publishes artifacts of the published site with its Nuget/Npm/Bower dependencies.

I am here going to create a poor man’s build server directly on the production server. Npm and Bower decencies are going to be checked in order to simplify the publish process but it is straightforward to install all dependencies with the build script if you prefer that.

Git-Auto-Deploy is a small HTTP server that handles web hooks from Git servers. We are going to setup Git-Auto-Deploy with Gitlab but any Git server that supports web hooks will work. Gitlab will send a HTTP request to the server on every push that tells the server to pull the latest code and run a build script in order publish the site.

Remember to remove bower install from the prepublish section in project.json

Add the Git-Auto-Deploy repository feed and install it.

sudo apt-get install software-properties-common
sudo add-apt-repository ppa:olipo186/git-auto-deploy
sudo apt-get update
sudo apt-get install git-auto-deploy

Setup SSH

You can skip this part if you are using a public repository or if you access the server with HTTP(S) and are okay with storing your username and password in the repository URL. I prefer to use SSH since the connection is encrypted and since I don’t need to store the username and password anywhere on the production server.

We are going to create the key and test them as the logged in user and then move the keys to Git-Auto-Deploy. You can later remove the keys from your account in order to improve security. Move your own private keys temporarily if you already have keys setup for your account.

You can read more about Git and SSH on Github.

Generate your keys. Create the keys without a passphrase if you don’t know how the keyring works.

ssh-keygen -t rsa -b 4096 -C "deploy@yavtech.se"

Go to Settings -> Deploy Keys in your repository and add your public key. Type the following command to see your public key:

cat .ssh/id_rsa.pub

Test that you can access the server with your private key without having to type your password and save the Git server fingerprint.

ssh git@example.com -p 22

Copy the keys and set the owner as git-auto-deploy

sudo cp ~/.ssh/* /etc/git-auto-deploy/.ssh/
sudo chown -R git-auto-deploy:git-auto-deploy /etc/git-auto-deploy

Allow build script to restart services

The git-auto-deploy user is going to execute our deploy scripts and will need to restart the website during the deployment. The services can however normally only be controlled as a privileged user. One way of accomplishing this is to specify specific a whitelist of commands that the user can execute as a privileged user.

You need to be careful when performing the following commands since you can lose your own sudo rights if the files get any syntax errors. I recommend that you open another shell and switch user to root so that you can revert the changes without having to use sudo if anything goes wrong.

Open another command prompt and switch user to root (keep this shell open)

sudo su root

In the first prompt type and the following content:

sudo visudo -f /etc/sudoers.d/git-auto-deploy
git-auto-deploy ALL = (root) NOPASSWD: /usr/sbin/service aspnetcore-*

Configure Git-Auto-Deploy

Add your repository

sudo vi /etc/git-auto-deploy.conf.json
{
  "http-enabled": true,
  "http-host": "0.0.0.0",
  "http-port": 8001,
  "log-file": "/var/log/git-auto-deploy.log",
  "repositories": [
    {
      "url": "ssh://git@example.com:22/payam/hello.world.site.git",
      "path": "/var/lib/git-auto-deploy/hello.world.site",
      "branch": "master",
      "remote": "origin",
      "deploy": "/var/aspnetcore/publish/hello.world.site.sh",
      "filters": [
        {
          "object_kind": "push",
          "ref": "refs/heads/master"
        }
      ]
    }
  ]
}

Start Git-Auto-Deploy and check that your code is checked out. You can also check the status of the service

sudo service git-auto-deploy start
sudo service git-auto-deploy status
sudo ls /var/lib/git-auto-deploy/hello.world.site

Setup the folders and the build script. My websites are storing their applications logs in the logs folder using Nlog. This is how my build script looks like.

sudo mkdir /var/aspnetcore
sudo mkdir /var/aspnetcore/hello.world.site
sudo mkdir /var/aspnetcore/publish
sudo vi /var/aspnetcore/publish/hello.world.site.sh

{
    cd /var/lib/git-auto-deploy/hello.world.site/src/hello.world.site.Web/
    dotnet restore
    dotnet build
    rm -rf /var/aspnetcore/publish/hello.world.site/*
    dotnet publish -o /var/aspnetcore/publish/hello.world.site
    sudo service aspnetcore-hello.world.site stop
    rm -rf /var/aspnetcore/hello.world.site/*
    cp -rf /var/aspnetcore/publish/hello.world.site/* /var/aspnetcore/hello.world.site/
    sudo service aspnetcore-hello.world.site start
} > /var/aspnetcore/publish/hello.world.site.build.logs 2>/var/aspnetcore/publish/hello.world.site.error.logs

The build script will log the build outputs to hello.world.site.build.logs and the build error outputs to hello.world.site.error.logs

Create publish and logs folder and setup correct permissions. My applications write their log files to logs folder as the www-data user:

sudo mkdir /var/aspnetcore/logs
sudo chmod 775 /var/aspnetcore/logs
sudo chmod +x /var/aspnetcore/publish/hello.world.site.sh
sudo chown -R git-auto-deploy:git-auto-deploy /var/aspnetcore/
sudo chown -R www-data:www-data /var/aspnetcore/logs/

Add user www-data to git-auto-deploy group

sudo usermod -a -G git-auto-deploy www-data

Go to Integrations and add a webhook in your repository in Gitlab and add a web hook to you server. In my case: http://192.168.1.46:8001. Test that eveything is working by clicking on the test button in Gitlab.

Update Git-Auto-Deploy

sudo chown -R git-auto-deploy:git-auto-deploy /etc/git-auto-deploy
sudo chmod 700 /etc/git-auto-deploy/.ssh
sudo chmod 600 /etc/git-auto-deploy/.ssh/id_rsa

Setup http server (Nginx)

Install Nginx

sudo apt-get install nginx

Configure you website

sudo vi /etc/nginx/sites-available/hello.world.site
server {
    listen 80;
    server_name hello.world.site;
    access_log /var/log/nginx/hello.world.site_access.log;
    error_log /var/log/nginx/hello.world.site_error.log;
    location / {
        proxy_pass http://localhost:5000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection keep-alive;
        proxy_cache_bypass $http_upgrade;
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        client_max_body_size 10m;
        client_body_buffer_size 128k;
        proxy_connect_timeout 90;
        proxy_send_timeout 90;
        proxy_read_timeout 90;
        proxy_buffers 32 4k;

    }
}

Enable the configuration

sudo ln -s /etc/nginx/sites-available/hello.world.site /etc/nginx/sites-enabled/

Test configuration and reload the configuration files if everything is okey

sudo nginx -t
sudo nginx -s reload

Setup Let's Encrypt with auto renewal

Ubuntu 16.04 comes with a old version of letsencrypt which doesn’t support Nginx well. I have therefor followed this blog on how to set it up. Clone the latest bleeding edge version from Github

sudo git clone https://github.com/certbot/certbot /opt/certbot

Run certbot and follow the guided steps

sudo /opt/certbot/certbot-auto --nginx

Improve the SSL security by generating a stronger DHE parameter.

sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096

Add the file to all configurations that use SSL

sudo vi /etc/nginx/sites-available/hello.world.site
server {
    ...
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    ...
}

Reload the configurations

sudo nginx -s reload

Setup auto renewal of the certificates with cron

sudo crontab -e
15 2 * * * /opt/certbot/certbot-auto --nginx renew --quiet --non-interactive >> /var/log/letsencrypt.log

Ports

See Change default Kestrel port in ASP.NET Core to setup .net core applications using different ports in order to be able to host more sites.

Troubleshooting

The different log files:

sudo service aspnetcore-hello.world.site status
sudo journalctl -fu aspnetcore-hello.world.site
sudo vi /var/log/git-auto-deploy.log
sudo service git-auto-deploy status
vi /var/aspnetcore/publish/hello.world.site.build.logs
vi /var/aspnetcore/publish/hello.world.site.error.logs
vi /var/log/letsencry

MailKit

MailKit creates cryptography files in /var/www/.dotnet. Create the folder and give www-data ownership

sudo mkdir /var/www/.dotnet
sudo chown www-data:www-data /var/www/.dotnet

Environment variables

You can pass environment in the service file. We did pass ASPNETCORE_ENVIRONMENT in etc/systemd/system/aspnetcore-hello.world.site.service in the example above. Nested settings are normally separated with : but you need to repalce : with __ (double underscore) in Linux.

Environments

See Environment settings in Linux if your environment settings aren't replaced for the different environments. It could be due to that file names are case sensitive in Linux.

• .NET Core (3)
• Ubuntu (3)
• Dropbox (1)
• Continuous deployment (1)
• Nginx (1)
• Git (1)