The complete guide for hosting and optimizing the performance of asp.net core

List of content

• Prepare you solution with self hosting option
• Order your Ubuntu hosting and the best provider if you have a limited budget
• Connect to your Ubuntu machine for the first time
• Installing And securing MySQL on Ubuntu
• Optional: Allow MySQL Remote connection and enable support for character set like Hebrew and Arabic
• Uploading project files to the server
• Running your asp.net core project for the first time
• Running your application as background service
• Using Nginx as revers proxy to host your application
• Attach domain or sub-domain to your website
• Getting a free SSL certificate
• Enabling https within Nginx reverse proxy
• Speed up your application and improve json requests and API performance
• offloading kestrel server and serving static content using Nginx and enable cashing
• Update your application on fly
• Author notes
• About the Author

In this Guide I will show you how to host .net core application on Linux servers and how to speed up application and optimize its performance fist I will assumes that your application uses MySQL as database later I will show how to use MsSQL on Linux based servers

Prepare you solution with self hosting option

Asp .net core application can run under kestrel web server thats required that the server have asp.net core runtime installed, as we try to make the required packages to be installed on server minimum, I choose to use self hosting option to host our application , please make sure that your program.cs file have the following line .UseUrls("http://*:5000") which instruct the kestrel server to listen to port 5000 from any IP source so later you can access your application hosted on the server from within your local machine so follow the following steps to prepare your solution for self hosting

• 1- Open your solution directory
• 2- Create file and Name it [publish-ubantu-18.04.bat]
• 3- Open and edit this file with Notepad++ (I assume that every developer have Notepad++ installed )
• 4- Within the file type the following lines assume that you are going to use ubantu-18.04 cd /d %0\.. dotnet restore dotnet publish -c Release --self-contained true -r ubuntu.18.04-x64 --output "%cd%\bin\YourProjectName" copy con
• 5- Right click on the [publish-ubantu-18.04.bat] file and choose run as administrator and wait until it finish compiling and building of your application

Now your solution is ready for self hosting on ubantu-18 linux

Order your ubuntu hosting and the best provider if you have a limited budget

Hosting on Amazon or Google cloud is not the best choice if you have limited budget and need to have full control on your overall cost here I will provide you with three of vps and cloud provider I personally try their services and recommend one of them

• Digitalocean they have SLA and free firewall and provide 99% uptime
• Linode you will get the best performance and the lowest price with very stable service
• Liquidweb they may be not the cheapest one but you will get the best support

Also you can go with Amzone or Google cloud no its your option and your budget

After you order your VPS or setup your local virtual machine you will get user name and password for root user and for none root user my recommendation for security reasons not to use root user for running or installing

Connect to your ubuntu machine for the first time

Most of cloud companies provide you a free web terminal you can access your machine directly from within your account but its better to use native ssh client installed in your machine so, if its the first time for you to connect to Linux machine all people will recommend you to install PuTTY please don't and download Mobaxterm
after downloading and installing Mobaxterm open new tap in Mobaxterm and type the following command ssh your-ip-address -l your-user-name

Installing And securing MySQL on Ubuntu

Use the following commands to install MySQL server
sudo apt-get update sudo apt-get install mysql-server Use the following command to secure MySQL instillation and set the MySQL root password please keep this password save mysql_secure_installation This will ask you whether to allow root user to connect remotely or not don't allow root user to login to MySQL remotely and when prompt choose yes delete the test database finally choose reload privilege
Ubuntu comes with the firewall enabled by default and if you want to connect to MySQL from other machine or from your local machine you have to allow incoming connection to MySQL default port which is 3306, before doing so you have to note that allowing this globally is a security risk and you have to allow the connection to and from specific IP or IP rang, for simplicity and testing purpose we will allow the connection from any IP so use the following command to allow remote connection to MySQL
sudo ufw allow mysql
To ensure that the database server keep running after reboot use the following command sudo systemctl enable mysql Now its time to connect to MySQL shell use the following command sudo mysql -u root -p you will be prompt to enter MySQL root password and you will see the MySQL shell prompt
Lets create MySQL user will full permissions so we can use it for testing and its strongly recommend in the production environment to create very limited MySQL user with only the minimum required permissions for the application to run correctly so on MySQL prompt type the following command > create user 'SomeUserName'@'%' IDENTIFIED BY 'SomePassword'; GRANT ALL PRIVILEGES ON *.* TO 'SomeUserName'@'%'; > flush privileges;

Optional: Allow MySQL Remote connection and enable support for character set like Hebrew and Arabic

if your application is multilingual its better to change MySQL default character set by default and i don't knew why MySQL comes with latin1 as the default character set so you need to change this to utf8 or some thing that support your language character set so we will use nano editor to edit MySQL configuration file so use the following command to edit MySQL configuration sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf this command will open mysqld.cnf for editing to enable remote connection change bind-address= 127.0.0.1 to bind-address= 0.0.0.0 that means MySQL will accept connection from any host then add the following lines after [mysqld] collation-server = utf8_unicode_ci init-connect='SET NAMES utf8' character-set-server = utf8 Optionally you can tweak MySQL performance by adjusting the following variables you need to tweak them regarding your server hardware query_cache_type=1 query_cache_size = 100M query_cache_limit=10M max_connections = 500 innodb_buffer_pool_size = 2GB innodb_io_capacity = 10000 finally lets stop and start MySQL for these changes to take place enter the following commands sudo systemctl stop mysql sudo systemctl start mysql

Uploading project files to the server

Before uploading our project files to server lets create a directory for our files with the following path /var/www/YourProjectName so use the following commands to create the path sudo mkdir -p /var/www/YourProjectName this will create the path if not exists
Before we upload our project files lets create a limited user for running our web server enter the following commands to create limited user with the user name "webuser" and password "YourPawwsord" useradd -g www-data webuser passwd webuser you will be prompted to enter the desired password enter a password of your choice and keep it in save place for later use
Lets give this user full permission on www directory, and please note its good security practice to give the user only the minimal permissions need to run the service and read only permission in our case is OK and enough, so enter the following command to give the user the required permissions setfacl -m u:webuser:rwx /var/www now we are ready to upload our files to the server so open Filezilla FTP client if you don't have Filezilla please install it you will use it a lot every time you have a new version or update, go to site manage and add new site choice the protocol to be SFTP then enter webuser in the user name box and your password you just set

Then click connect and transfer you build directory which in the bin directory to /var/www/YourProjectName please make the necessary changes to your appsettings JSON file to use the new MySQL user and password

Running your asp.net core project for the first time

Before running your website first you need to install libgdiplus if you are using system.drawing its required by some library in .net so enter the following command sudo apt-get install libgdiplus for our self-contained project to run we have to give the executable file inside our project directory execution permission so enter the following command to change your executable permission sudo chmod 0755 /var/www/YourProjectName/YourProjectName yes its /YourProjectName/YourProjectName as your folder name is the same as YourProjectName.
If your project is listening to port 5000 or any other port lets temporary allow this port through our firewall so enter the following command sudo ufw allow 5000 this ensure you can test and access your project from within any machine not only the server.
Its time to run and test our project enter the following commands to start run your asp.net core application cd /var/www/YourProjectName ./YourProjectName you will see a screen like the following screen open your internet browser and in the address bar type
http://your_server_ip:5000
you will see your site running

Running your application as background service

We need our application to run as service so it will start automatically after server reboot or process crash or killed by the server so create a new text file with the name [YourProjectName.service] edit this file with Notepad++ and copy the following text to it and save your file, note you have to change YourProjectName to your current project name [Unit] Description= YourProjectName running on Ubuntu [Service] WorkingDirectory=/var/www/YourProjectName ExecStart=/var/www/YourProjectName/YourProjectName Restart=always # Restart service after 10 seconds if the service crashes RestartSec=10 SyslogIdentifier=dotnet-YourProjectName User=webuser Environment=ASPNETCORE_ENVIRONMENT=Production [Install] WantedBy=multi-user.target Inorder for our service to run we need to copy this file [YourProjectName.service] to the following path /etc/systemd/system/ with the limited user you can't directly copy the file to this system path so first open your filezilla and upload the file to /var/www/YourProjectName then run the following commands to more the file , install the service and start the service sudo cp /var/www/YourProjectName/YourProjectName.service /etc/systemd/system/ sudo rm -f /var/www/YourProjectName/YourProjectName.service sudo systemctl enable YourProjectName.service sudo systemctl start YourProjectName.service Now our dont net service installed and running to see the service status use the following command sudo systemctl status YourProjectName.service any time you want to stop the service use the following command sudo systemctl stop YourProjectName.service to see service log and what happening in back ground use the following command sudo journalctl -fu YourProjectName.service

Using Nginx as revers proxy to host your application

As am typing this article kestrel web server still lack many featchers fonded in professional web server like Nginx, Apahce , IIS so using reverse proxy on front of kestrel is a must and not an optional so lets go throught the process of installing and configuring Nginx as reverse proxy for our project , run the following commands to install Nginx sudo apt update sudo apt install nginx then use the following command to allow http and https sudo ufw allow 80 sudo ufw allow 443 then navigate to the url http://your_server_ip you will see some thing like the following image after verifying that Nginx is running create text file and name it [YourProjectName.nginx.conf] edit this file with notepad++ and add the following lines and make sure that your project is listen to port 5000 or change it to your project port server { client_max_body_size 350M; listen 80; fastcgi_buffers 16 16k; fastcgi_buffer_size 32k; proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $remote_addr; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection keep-alive; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; proxy_pass http://localhost:5000; } } save the file and using filezilla upload the file to /var/www/YourProjectName then use the following command to copy this configuration file to Nginx configuration directory path /etc/nginx/sites-enabled/ sudo cp /var/www/YourProjectName/YourProjectName.nginx.conf /etc/nginx/sites-enabled/ sudo cp /var/www/YourProjectName/YourProjectName.nginx.conf /etc/nginx/sites-available/ sudo rm -f /etc/nginx/sites-enabled/default sudo rm -f /etc/nginx/sites-available/default sudo rm -f /var/www/YourProjectName/YourProjectName.nginx.conf sudo systemctl restart nginx then navigate to the url http://your_server_ip you will see your project been serverd by Nginx reverse proxy

Attach domain or sub-domain to your website

As you are serving the project from dedicated IP address the only requirement to link your domain to your server is an [A] recored go to your domain provider and choose create new [A] recored and when prompt to enter IP address enter your server IP address and wait may be 10 minutes to 4 hours and vist your domain and ensure it linked to your server
If you prefer to use a sub-domain go to your hosting and open Cpanel or plesk control panel , then go to the zone editor and add [A] recored and when prompt to enter IP address enter your server IP address

Getting a free SSL certificate

In-order to run https your have to buy SSL certificate from a certified certificate authority and you have to pay for this certificate per year, fortunately there is a project called Let's Encrypt supported by community which provide you with free SSL certificate, so You'll need to add the Certbot PPA to your list of repositories. To do so, run the following commands on the command line on the machine: sudo apt-get update sudo apt-get install software-properties-common sudo add-apt-repository universe sudo add-apt-repository ppa:certbot/certbot sudo apt-get update Run this command on the command line on the machine to install Certbot. sudo apt-get install certbot python-certbot-nginx then run the following command to create your certificate sudo certbot certonly --nginx then you will be prompt to enter your domain name and it will output the certification to path like /etc/letsencrypt/live/your_domain_name/cert.pem /etc/letsencrypt/live/your_domain_name/privkey.pem The Certbot packages on your system come with a cron job or systemd timer that will renew your certificates automatically before they expire. You will not need to run Certbot again, unless you change your configuration. You can test automatic renewal for your certificates by running this command: sudo certbot renew --dry-run

Enabling https within Nginx reverse proxy

In order to enable Https we will going to use the free certificate we just created with Certbot so edit your file [YourProjectName.nginx.conf] make it exactly look like the following server { client_max_body_size 350M; listen 80; fastcgi_buffers 16 16k; fastcgi_buffer_size 32k; proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $remote_addr; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection keep-alive; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; proxy_pass http://localhost:5000; } } server { client_max_body_size 350M; listen 443; ssl on; ssl_certificate /etc/letsencrypt/live/your_domain_name/cert.pem; # path to your cacert.pem ssl_certificate_key /etc/letsencrypt/live/your_domain_name/privkey.pem; # path to your privkey.pem server_name seafile.example.com; server_tokens off; # ...... fastcgi_param HTTPS on; fastcgi_param HTTP_SCHEME https; fastcgi_buffers 16 16k; fastcgi_buffer_size 32k; proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $remote_addr; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection keep-alive; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; proxy_pass http://localhost:5000; } } save the file and using filezilla upload the file to /var/www/YourProjectName then use the following command to copy this configuration file to Nginx configuration directory path /etc/nginx/sites-enabled/ sudo cp /var/www/YourProjectName/YourProjectName.nginx.conf /etc/nginx/sites-enabled/ sudo cp /var/www/YourProjectName/YourProjectName.nginx.conf /etc/nginx/sites-available/ sudo rm -f /var/www/YourProjectName/YourProjectName.nginx.conf sudo systemctl restart nginx then navigate to the url https://your_server_ip you will see your project been served by Nginx reverse proxy with https and free SSL certificate

Speed up your application and improve JSON requests and API performance

we have real case where we have point of sales which need to cash about 25000 item on its start up those items comes from the server as raw JSON response it was about 14MB, the propose solution to speed up the request is to compress the JSON file on the server with gzip and decompress the response on the client this involves change in the client and server code, and is not practical solution an idea comes out to me to use Nginx gzip module to compress the JSON file on the server and the browser will handle the decompression on the client side, all major browsers support this feature, after using this module the JSON response size becomes less than 1MB and the point of sales can cash all the 25K item in less than 3 seconds instead of 43 seconds needed before compression, actually this module improve the overall performance of the application, but you have to use this feature with caution if you have single virtual core its better not to use gzip module but if you have more than 1 virtual core its ok, and don't enable this fetcher for all MIME types just enable it to request which have large amount of text, if you for example enable it for JPG you will not benefit from it that JPG is compressed already, so to enable the gzip compression open your file [YourProjectName.nginx.conf] edit this file with notepad++ and change its content to the following server { gzip on; gzip_types text/json application/json text/plain application/xml; gzip_proxied no-cache no-store private expired auth; gzip_min_length 1000; gzip_proxied no-cache no-store private expired auth; client_max_body_size 350M; listen 80; fastcgi_buffers 16 16k; fastcgi_buffer_size 32k; proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $remote_addr; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection keep-alive; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; proxy_pass http://localhost:5000; } } server { gzip on; gzip_types text/json application/json text/plain application/xml; gzip_proxied no-cache no-store private expired auth; gzip_min_length 1000; gzip_proxied no-cache no-store private expired auth; client_max_body_size 350M; listen 443; ssl on; ssl_certificate /etc/letsencrypt/live/your_domain_name/cert.pem; # path to your cacert.pem ssl_certificate_key /etc/letsencrypt/live/your_domain_name/privkey.pem; # path to your privkey.pem server_name seafile.example.com; server_tokens off; # ...... fastcgi_param HTTPS on; fastcgi_param HTTP_SCHEME https; fastcgi_buffers 16 16k; fastcgi_buffer_size 32k; proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $remote_addr; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection keep-alive; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; proxy_pass http://localhost:5000; } } save the file and using filezilla upload the file to /var/www/YourProjectName then use the following command to copy this configuration file to Nginx configuration directory path /etc/nginx/sites-enabled/ sudo cp /var/www/YourProjectName/YourProjectName.nginx.conf /etc/nginx/sites-enabled/ sudo cp /var/www/YourProjectName/YourProjectName.nginx.conf /etc/nginx/sites-available/ sudo rm -f /var/www/YourProjectName/YourProjectName.nginx.conf sudo systemctl restart nginx Test your API and JSON response speed and size and see how much this module gona improve your server overal response

offloading kestrel server and serving static content using Nginx and enable cashing

service static content using kestrel is a waste of valuable time of kestrel server so we can serve static content from within Nginx and leave the dynamic for kestrel to serve static content edit [YourProjectName.nginx.conf] and add change it to look like the following proxy_cache_path /tmp/cache levels=1:2 keys_zone=aspnetcache:1000m max_size=1000m inactive=600m; proxy_temp_path /tmp/cache/temp; server { gzip on; gzip_types text/json application/json text/plain application/xml; gzip_proxied no-cache no-store private expired auth; gzip_min_length 1000; gzip_proxied no-cache no-store private expired auth; client_max_body_size 350M; listen 80; fastcgi_buffers 16 16k; fastcgi_buffer_size 32k; proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; location ~ ^/(images|lib|js|css)/ { proxy_cache aspnetcache; proxy_cache_valid 200 302 60m; proxy_cache_valid 404 1m; root /var/www/YourProjectName/wwwroot; } location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $remote_addr; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection keep-alive; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; proxy_pass http://localhost:5000; } } server { gzip on; gzip_types text/json application/json text/plain application/xml; gzip_proxied no-cache no-store private expired auth; gzip_min_length 1000; gzip_proxied no-cache no-store private expired auth; client_max_body_size 350M; listen 443; ssl on; ssl_certificate /etc/letsencrypt/live/your_domain_name/cert.pem; # path to your cacert.pem ssl_certificate_key /etc/letsencrypt/live/your_domain_name/privkey.pem; # path to your privkey.pem server_name seafile.example.com; server_tokens off; # ...... fastcgi_param HTTPS on; fastcgi_param HTTP_SCHEME https; fastcgi_buffers 16 16k; fastcgi_buffer_size 32k; proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; location ~ ^/(images|lib|js|css)/ { proxy_cache aspnetcache; proxy_cache_valid 200 302 60m; proxy_cache_valid 404 1m; root /var/www/YourProjectName/wwwroot; } location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $remote_addr; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection keep-alive; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; proxy_pass http://localhost:5000; } } Do the same thing you do in the previous step upload the file and ... and restart Nginx, to ensure that your work is correct stop your project service with the following command sudo systemctl stop YourProjectName.service and see if you can still access the static content saved in wwwroot directory
start YourProjectName.service with the following command sudo systemctl stop YourProjectName.service

Update your application on fly

if you made some changes in your code its not necessary to upload the hole bulid directory to the server, instead every time you add or remove files to wwwroot upload it to the server wwwroot and every time you have changes in your code just save the changes and publish your project,by running the bat file we made for self-contained publishing, after it finish just upload the two files YourProjectName.dll YourProjectName.Views.dll and if you are using internationalization upload your language directory then restart your project service with the following command sudo systemctl restart YourProjectName.service

Author notes

• This manual is for educational purpose and prove of concept and is not intended to be for production use
• For simplicity I ignore a lot of security optimization and recommendation so if you intended to use this manual for production you have to check and review in-deep your permissions and ensure to use best practices
• This manual is open source and you allowed to modify it and distribute it but don't remove the Author section or
• If you are using this manual you are using it at your own risk

About the Author

Name : Mohamed ALi El-Sayed
Education : Master's degree of Computer Science Optics, image, vision & multimedia from University Paris-Est Créteil (UPEC)
Email: phptop@gmail.com
Linked-In profile: https://www.linkedin.com/in/mohamed-elsayed-9034916/
Github page : https://github.com/codejq
An update version of this file may be found on http://faceprintlab.com