How To Install and Configure Zabbix to Securely Monitor Remote Servers on Ubuntu 20.04

The author selected the Computer History Museum to receive a donation as part of the Write for DOnations program.

Introduction

Zabbix is open-source monitoring software for networks and applications. It offers real-time monitoring of thousands of metrics collected from servers, virtual machines, network devices, and web applications. These metrics can help you determine the current health of your IT infrastructure and detect problems with hardware or software components before customers complain. Useful information is stored in a database so you can analyze data over time and improve the quality of provided services or plan upgrades of your equipment.

Zabbix uses several options for collecting metrics, including agentless monitoring of user services and client-server architecture. To collect server metrics, it uses a small agent on the monitored client to gather data and send it to the Zabbix server. Zabbix supports encrypted communication between the server and connected clients, so your data is protected while it travels over insecure networks.

The Zabbix server stores its data in a relational database powered by MySQL or PostgreSQL. You can also store historical data in NoSQL databases like Elasticsearch and TimescaleDB. Zabbix provides a web interface so you can view data and configure system settings.

In this tutorial, you will configure Zabbix on two Ubuntu 20.04 machines. One will be configured as the Zabbix server, and the other as a client that you’ll monitor. The Zabbix server will use a MySQL database to record monitoring data and use Nginx to serve the web interface.

Prerequisites

To follow this tutorial, you will need:

  • Two Ubuntu 20.04 servers set up by following the Initial Server Setup Guide for Ubuntu 20.04, including a non-root user with sudo privileges and a firewall configured with ufw. On one server, you will install Zabbix; this tutorial will refer to this as the Zabbix server. It will monitor your second server; this second server will be referred to as the second Ubuntu server.
  • The server that will run the Zabbix server needs Nginx, MySQL, and PHP installed. Follow Steps 1–3 of our Ubuntu 20.04 LEMP Stack guide to configure those on your Zabbix server.
  • A registered domain name. This tutorial will use your_domain throughout. You can purchase a domain name from Namecheap, get one for free with Freenom, or use the domain registrar of your choice.
  • Both of the following DNS records set up for your Zabbix server. If you are using DigitalOcean, please see our DNS documentation for details on how to add them.
    • An A record with your_domain pointing to your Zabbix server’s public IP address.
    • An A record with www.your_domain pointing to your Zabbix server’s public IP address.

Additionally, because the Zabbix Server is used to access valuable information about your infrastructure that you would not want unauthorized users to access, it’s important that you keep your server secure by installing a TLS/SSL certificate. This is optional but strongly encouraged. If you would like to secure your server, follow the Let’s Encrypt on Ubuntu 20.04 guide after Step 3 of this tutorial.

Step 1 — Installing the Zabbix Server

First, you need to install Zabbix on the server where you installed MySQL, Nginx, and PHP. Log in to this machine as your non-root user:

  • ssh sammy@zabbix_server_ip_address

Zabbix is available in Ubuntu’s package manager, but it’s outdated, so use the official Zabbix repository to install the latest stable version. Download and install the repository configuration package:

  • wget https://repo.zabbix.com/zabbix/5.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.0-1+focal_all.deb
  • sudo dpkg -i zabbix-release_5.0-1+focal_all.deb

You will see the following output:

Output
Selecting previously unselected package zabbix-release.
(Reading database ... 64058 files and directories currently installed.)
Preparing to unpack zabbix-release_5.0-1+focal_all.deb ...
Unpacking zabbix-release (1:5.0-1+focal) ...
Setting up zabbix-release (1:5.0-1+focal) ...

Update the package index so the new repository is included:

  • sudo apt update

Then install the Zabbix server and web frontend with MySQL database support:

  • sudo apt install zabbix-server-mysql zabbix-frontend-php

Also, install the Zabbix agent, which will let you collect data about the Zabbix server status itself.

  • sudo apt install zabbix-agent

Before you can use Zabbix, you have to set up a database to hold the data that the Zabbix server will collect from its agents. You can do this in the next step.

Step 2 — Configuring the MySQL Database for Zabbix

You need to create a new MySQL database and populate it with some basic information in order to make it suitable for Zabbix. You’ll also create a specific user for this database so Zabbix isn’t logging in to MySQL with the root account.

Log in to MySQL as the root user:

  • sudo mysql

Create the Zabbix database with UTF-8 character support:

  • create database zabbix character set utf8 collate utf8_bin;

Then create a user that the Zabbix server will use, give it access to the new database, and set the password for the user:

  • create user zabbix@localhost identified by ‘your_zabbix_mysql_password‘;
  • grant all privileges on zabbix.* to zabbix@localhost;

That takes care of the user and the database. Exit out of the database console.

  • quit;

Next you have to import the initial schema and data. The Zabbix installation provided you with a file that sets this up.

Run the following command to set up the schema and import the data into the zabbix database. Use zcat since the data in the file is compressed:

  • zcat /usr/share/doc/zabbix-server-mysql*/create.sql.gz | mysql -uzabbix -p zabbix

Enter the password for the zabbix MySQL user that you configured when prompted.

This command may take a minute or two to execute. If you see the error ERROR 1045 (28000): Access denied for userzabbix@'localhost' (using password: YES) then make sure you used the right password for the zabbix user.

In order for the Zabbix server to use this database, you need to set the database password in the Zabbix server configuration file. Open the configuration file in your preferred text editor. This tutorial will use nano:

  • sudo nano /etc/zabbix/zabbix_server.conf

Look for the following section of the file:

/etc/zabbix/zabbix_server.conf
...
### Option: DBPassword                           
#       Database password. Ignored for SQLite.   
#       Comment this line if no password is used.
#                                                
# Mandatory: no                                  
# Default:                                       
# DBPassword=
...

These comments in the file explain how to connect to the database. You need to set the DBPassword value in the file to the password for your database user. Add this line after those comments to configure the database:

/etc/zabbix/zabbix_server.conf
...
DBPassword=your_zabbix_mysql_password
...

Save and close zabbix_server.conf by pressing CTRL+X, followed by Y and then ENTER if you’re using nano.

You’ve now configured the Zabbix server to connect to the database. Next, you will configure the Nginx web server to serve the Zabbix frontend.

Step 3 — Configuring Nginx for Zabbix

To configure Nginx automatically, install the automatic configuration package:

  • sudo apt install zabbix-nginx-conf

As a result, you will get the configuration file /etc/zabbix/nginx.conf, as well as a link to it in the Nginx configuration directory /etc/nginx/conf.d/zabbix.conf.

Next, you need to make changes to this file. Open the configuration file:

  • sudo nano /etc/zabbix/nginx.conf

The file contains an automatically generated Nginx server block configuration. It contains two lines that determine the server name and what port it is listening on:

/etc/zabbix/nginx.conf
server {
#        listen          80;
#        server_name     example.com;
...

Uncomment the two lines, and replace example.com with your domain name. Your settings will look like this:

/etc/zabbix/nginx.conf
server {
        listen          80;
        server_name     your_domain;
...

Save and close the file. Next, test to make sure that there are no syntax errors in any of your Nginx files and reload the configuration:

  • sudo nginx -t
  • sudo nginx -s reload

Now that Nginx is set up to serve the Zabbix frontend, you will make some modifications to your PHP setup in order for the Zabbix web interface to work properly.

Note: As mentioned in the Prerequisites section, it is recommended that you enable SSL/TLS on your server. If you would like to do this, follow our Ubuntu 20.04 Let’s Encrypt tutorial before you move on to Step 4 to obtain a free SSL certificate for Nginx. This process will automatically detect your Zabbix server block and configure it for HTTPS. After obtaining your SSL/TLS certificates, you can come back and complete this tutorial.

Step 4 — Configuring PHP for Zabbix

The Zabbix web interface is written in PHP and requires some special PHP server settings. The Zabbix installation process created a PHP-FPM configuration file that contains these settings. It is located in the directory /etc/zabbix and is loaded automatically by PHP-FPM. You need to make a small change to this file, so open it up with the following:

  • sudo nano /etc/zabbix/php-fpm.conf

The file contains PHP settings that meet the necessary requirements for the Zabbix web interface. However, the timezone setting is commented out by default. To make sure that Zabbix uses the correct time, you need to set the appropriate timezone:

/etc/zabbix/php-fpm.conf
...
php_value[max_execution_time] = 300
php_value[memory_limit] = 128M
php_value[post_max_size] = 16M
php_value[upload_max_filesize] = 2M
php_value[max_input_time] = 300
php_value[max_input_vars] = 10000
; php_value[date.timezone] = Europe/Riga

Uncomment the timezone line highlighted in the preceding code block and change it to your timezone. You can use this list of supported time zones to find the right one for you. Then save and close the file.

Now restart PHP-FPM to apply these new settings:

  • sudo systemctl restart php7.4-fpm.service

You can now start the Zabbix server:

  • sudo systemctl start zabbix-server

Then check whether the Zabbix server is running properly:

  • sudo systemctl status zabbix-server

You will see the following status:

Output
● zabbix-server.service - Zabbix Server
     Loaded: loaded (/lib/systemd/system/zabbix-server.service; disabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-06-12 05:59:32 UTC; 36s ago
    Process: 27026 ExecStart=/usr/sbin/zabbix_server -c $CONFFILE (code=exited, status=0/SUCCESS)
  ...

Finally, enable the server to start at boot time:

  • sudo systemctl enable zabbix-server

The server is set up and connected to the database. Next, set up the web frontend.

Step 5 — Configuring Settings for the Zabbix Web Interface

The web interface lets you see reports and add hosts that you want to monitor, but it needs some initial setup before you can use it. Launch your browser and go to the address http://zabbix_server_name or https://zabbix_server_name if you set up Let’s Encrypt. On the first screen, you will see a welcome message. Click Next step to continue.

On the next screen, you will see the table that lists all of the prerequisites to run Zabbix.

Prerequisites

All of the values in this table must be OK, so verify that they are. Be sure to scroll down and look at all of the prerequisites. Once you’ve verified that everything is ready to go, click Next step to proceed.

The next screen asks for database connection information.

DB Connection

You told the Zabbix server about your database, but the Zabbix web interface also needs access to the database to manage hosts and read data. Therefore enter the MySQL credentials you configured in Step 2. Click Next step to proceed.

On the next screen, you can leave the options at their default values.

Zabbix Server Details

The Name is optional; it is used in the web interface to distinguish one server from another in case you have several monitoring servers. Click Next step to proceed.

The next screen will show the pre-installation summary so you can confirm everything is correct.

Summary

Click Next step to proceed to the final screen.

The web interface setup is now complete. This process creates the configuration file /usr/share/zabbix/conf/zabbix.conf.php, which you could back up and use in the future. Click Finish to proceed to the login screen. The default user is Admin and the password is zabbix.

Before you log in, set up the Zabbix agent on your second Ubuntu server.

Step 6 — Installing and Configuring the Zabbix Agent

Now you need to configure the agent software that will send monitoring data to the Zabbix server.

Log in to the second Ubuntu server:

  • ssh sammy@second_ubuntu_server_ip_address

Just like on the Zabbix server, run the following commands to install the repository configuration package:

  • wget https://repo.zabbix.com/zabbix/5.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.0-1+focal_all.deb
  • sudo dpkg -i zabbix-release_5.0-1+focal_all.deb

Next, update the package index:

  • sudo apt update

Then install the Zabbix agent:

  • sudo apt install zabbix-agent

While Zabbix supports certificate-based encryption, setting up a certificate authority is beyond the scope of this tutorial. But you can use pre-shared keys (PSK) to secure the connection between the server and agent.

First, generate a PSK:

  • sudo sh -c “openssl rand -hex 32 > /etc/zabbix/zabbix_agentd.psk”

Show the key by using cat so you can copy it somewhere:

  • cat /etc/zabbix/zabbix_agentd.psk

The key will look something like this:

Output
75ad6cb5e17d244ac8c00c96a1b074d0550b8e7b15d0ab3cde60cd79af280fca

Save this for later; you will need it to configure the host.

Now edit the Zabbix agent settings to set up its secure connection to the Zabbix server. Open the agent configuration file in your text editor:

  • sudo nano /etc/zabbix/zabbix_agentd.conf

Each setting within this file is documented via informative comments throughout the file, but you only need to edit some of them.

First you have to edit the IP address of the Zabbix server. Find the following section:

/etc/zabbix/zabbix_agentd.conf
...
### Option: Server
#       List of comma delimited IP addresses, optionally in CIDR notation, or DNS names of Zabbix servers and Zabbix proxies.
#       Incoming connections will be accepted only from the hosts listed here.
#       If IPv6 support is enabled then '127.0.0.1', '::127.0.0.1', '::ffff:127.0.0.1' are treated equally
#       and '::/0' will allow any IPv4 or IPv6 address.
#       '0.0.0.0/0' can be used to allow any IPv4 address.
#       Example: Server=127.0.0.1,192.168.1.0/24,::1,2001:db8::/32,zabbix.example.com
#
# Mandatory: yes, if StartAgents is not explicitly set to 0
# Default:
# Server=

Server=127.0.0.1
...

Change the default value to the IP of your Zabbix server:

/etc/zabbix/zabbix_agentd.conf
...
Server=zabbix_server_ip_address
...

By default, Zabbix server connects to the agent. But for some checks (for example, monitoring the logs), a reverse connection is required. For correct operation, you need to specify the Zabbix server address and a unique host name.

Find the section that configures the active checks and change the default values:

/etc/zabbix/zabbix_agentd.conf
...
##### Active checks related

### Option: ServerActive
#       List of comma delimited IP:port (or DNS name:port) pairs of Zabbix servers and Zabbix proxies for active checks.
#       If port is not specified, default port is used.
#       IPv6 addresses must be enclosed in square brackets if port for that host is specified.
#       If port is not specified, square brackets for IPv6 addresses are optional.
#       If this parameter is not specified, active checks are disabled.
#       Example: ServerActive=127.0.0.1:20051,zabbix.domain,[::1]:30051,::1,[12fc::1]
#
# Mandatory: no
# Default:
# ServerActive=

ServerActive=zabbix_server_ip_address

### Option: Hostname
#       Unique, case sensitive hostname.
#       Required for active checks and must match hostname as configured on the server.
#       Value is acquired from HostnameItem if undefined.
#
# Mandatory: no
# Default:
# Hostname=

Hostname=Second Ubuntu Server
...

Next, find the section that configures the secure connection to the Zabbix server and enable pre-shared key support. Find the TLSConnect section, which looks like this:

/etc/zabbix/zabbix_agentd.conf
...
### Option: TLSConnect
#       How the agent should connect to server or proxy. Used for active checks.
#       Only one value can be specified:
#               unencrypted - connect without encryption
#               psk         - connect using TLS and a pre-shared key
#               cert        - connect using TLS and a certificate
#
# Mandatory: yes, if TLS certificate or PSK parameters are defined (even for 'unencrypted' connection)
# Default:
# TLSConnect=unencrypted
...

Then add this line to configure pre-shared key support:

/etc/zabbix/zabbix_agentd.conf
...
TLSConnect=psk
...

Next, locate the TLSAccept section, which looks like this:

/etc/zabbix/zabbix_agentd.conf
...
### Option: TLSAccept
#       What incoming connections to accept.
#       Multiple values can be specified, separated by comma:
#               unencrypted - accept connections without encryption
#               psk         - accept connections secured with TLS and a pre-shared key
#               cert        - accept connections secured with TLS and a certificate
#
# Mandatory: yes, if TLS certificate or PSK parameters are defined (even for 'unencrypted' connection)
# Default:
# TLSAccept=unencrypted
...

Configure incoming connections to support pre-shared keys by adding this line:

/etc/zabbix/zabbix_agentd.conf
...
TLSAccept=psk
...

Next, find the TLSPSKIdentity section, which looks like this:

/etc/zabbix/zabbix_agentd.conf
...
### Option: TLSPSKIdentity
#       Unique, case sensitive string used to identify the pre-shared key.
#
# Mandatory: no
# Default:
# TLSPSKIdentity=
...

Choose a unique name to identify your pre-shared key by adding this line:

/etc/zabbix/zabbix_agentd.conf
...
TLSPSKIdentity=PSK 001
...

You’ll use this as the PSK ID when you add your host through the Zabbix web interface.

Then set the option that points to your previously created pre-shared key. Locate the TLSPSKFileoption:

/etc/zabbix/zabbix_agentd.conf
...
### Option: TLSPSKFile
#       Full pathname of a file containing the pre-shared key.
#
# Mandatory: no
# Default:
# TLSPSKFile=
...

Add this line to point the Zabbix agent to your PSK file you created:

/etc/zabbix/zabbix_agentd.conf
...
TLSPSKFile=/etc/zabbix/zabbix_agentd.psk
...

Save and close the file. Now you can restart the Zabbix agent and set it to start at boot time:

  • sudo systemctl restart zabbix-agent
  • sudo systemctl enable zabbix-agent

For good measure, check that the Zabbix agent is running properly:

  • sudo systemctl status zabbix-agent

You will see the following status, indicating the agent is running:

Output
● zabbix-agent.service - Zabbix Agent
   Loaded: loaded (/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-06-12 08:19:54 UTC; 25s ago
  ...

The agent will listen on port 10050 for connections from the server. Configure UFW to allow connections to this port:

  • sudo ufw allow 10050/tcp

You can learn more about UFW in How To Set Up a Firewall with UFW on Ubuntu 20.04.

Your agent is now ready to send data to the Zabbix server. But in order to use it, you have to link to it from the server’s web console. In the next step, you will complete the configuration.

Step 7 — Adding the New Host to the Zabbix Server

Installing an agent on a server you want to monitor is only half of the process. Each host you want to monitor needs to be registered on the Zabbix server, which you can do through the web interface.

Log in to the Zabbix Server web interface by navigating to the address http://zabbix_server_name or https://zabbix_server_name:

The Zabbix login screen

When you have logged in, click on Configuration and then Hosts in the left navigation bar. Then click the Create host button in the top right corner of the screen. This will open the host configuration page.

Creating a host

Adjust the Host name and IP address to reflect the host name and IP address of your second Ubuntu server, then add the host to a group. You can select an existing group, for example Linux servers, or create your own group. The host can be in multiple groups. To do this, enter the name of an existing or new group in the Groups field and select the desired value from the proposed list.

Before adding the group, click the Templates tab.

Adding a template to the host

Type Template OS Linux by Zabbix agent in the Search field and then select it from the list to add this template to the host.

Next, navigate to the Encryption tab. Select PSK for both Connections to host and Connections from host. Then set PSK identity to PSK 001, which is the value of the TLSPSKIdentity setting of the Zabbix agent you configured previously. Then set PSK value to the key you generated for the Zabbix agent. It’s the one stored in the file /etc/zabbix/zabbix_agentd.psk on the agent machine.

Setting up the encryption

Finally, click the Add button at the bottom of the form to create the host.

You will see your new host in the list. Wait for a minute and reload the page to see green labels indicating that everything is working fine and the connection is encrypted.

Zabbix shows your new host

If you have additional servers you need to monitor, log in to each host, install the Zabbix agent, generate a PSK, configure the agent, and add the host to the web interface following the same steps you followed to add your first host.

The Zabbix server is now monitoring your second Ubuntu server. Now, set up email notifications to be notified about problems.

Step 8 — Configuring Email Notifications

Zabbix automatically supports many types of notifications: email, OTRSSlackTelegram, SMS, etc. You can see the full list of integrations at the Zabbix website.

As an example, this tutorial will configure notifications for the Email media type.

Click on Administration, and then Media types in the left navigation bar. You will see the list of all media types. There are two preconfigured options for emails: for the plain text notification and for the HTML notifications. In this tutorial you will use plain text notification. Click on Email.

Adjust the SMTP options according to the settings provided by your email service. This tutorial uses Gmail’s SMTP capabilities to set up email notifications; if you would like more information about setting this up, see How To Use Google’s SMTP Server.


Note: If you use 2-Step Verification with Gmail, you need to generate an App Password for Zabbix. You’ll only have to enter an App password once during setup. You will find instructions on how to generate this password in the Google Help Center.

If you are using Gmail, type in smtp.gmail.com for the SMTP server field, 465 for the SMTP server port field, gmail.com for SMTP helo, and your email for SMTP email. Then choose SSL/TLS for Connection security and Username and password for Authentication. Enter your Gmail address as the Username, and the App Password you generated from your Google account as the Password.

Setting up email media type

On the Message templates tab you can see the list of predefined messages for various types of notifications. Finally, click the Update button at the bottom of the form to update the email parameters.

Now you can test sending notifications. To do this, click the Test underlined link in the corresponding line.

You will see a pop-up window. Enter your email address in the Send to field and click the Testbutton. You will see a message about the successful sending and you will receive a test message.

Testing email

Close the pop-up by clicking the Cancel button.

Now, create a new user. Click on Administration, and then Users in the left navigation bar. You will see the list of users. Then click the Create user button in the top right corner of the screen. This will open the user configuration page:

Creating a user

Enter the new username in the Alias field and set up a new password. Next, add the user to the administrator’s group. Type Zabbix administrators in the Groups field and select it from the proposed list.

Once you’ve added the group, click the Media tab and click on the Add underlined link (not the Add button below it). You will see a pop-up window.

Adding an email

Select the Email option from the Type drop down. Enter your email address in the Send to field. You can leave the rest of the options at the default values. Click the Add button at the bottom to submit.

Now navigate to the Permissions tab. Select Zabbix Super Admin from the User type drop-down menu.

Finally, click the Add button at the bottom of the form to create the user.

Note: Using the default password is not safe. In order to change the password of the built-in user Admin click on the alias in the list of users. Then click Change password, enter a new password, and confirm the changes by clicking Update button.

Now you need to enable notifications. Click on the Configuration tab and then Actions in the left navigation bar. You will see a pre-configured action, which is responsible for sending notifications to all Zabbix administrators. You can review and change the settings by clicking on its name. For the purposes of this tutorial, use the default parameters. To enable the action, click on the red Disabled link in the Status column.

Now you are ready to receive alerts. In the next step, you will generate one to test your notification setup.

Step 9 — Generating a Test Alert

In this step, you will generate a test alert to ensure everything is connected. By default, Zabbix keeps track of the amount of free disk space on your server. It automatically detects all disk mounts and adds the corresponding checks. This discovery is executed every hour, so you need to wait a while for the notification to be triggered.

Create a temporary file that’s large enough to trigger Zabbix’s file system usage alert. To do this, log in to your second Ubuntu server if you’re not already connected:

  • ssh sammy@second_ubuntu_server_ip_address

Next, determine how much free space you have on the server. You can use the df command to find out:

  • df -h

The command df will report the disk space usage of your file system, and the -h will make the output human-readable. You’ll see output like the following:

Output
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        78G  1.4G   77G    2% /

In this case, the free space is 77G. Your free space may differ.

Use the fallocate command, which allows you to pre-allocate or de-allocate space to a file, to create a file that takes up more than 80% of the available disk space. This will be enough to trigger the alert:

  • fallocate -l 70G /tmp/temp.img

After around an hour, Zabbix will trigger an alert about the amount of free disk space and will run the action you configured, sending the notification message. You can check your inbox for the message from the Zabbix server. You will see a message like:

Problem started at 09:49:08 on 2020.06.12
Problem name: /: Disk space is low (used > 80%)
Host: Second Ubuntu Server
Severity: Warning
Operational data: Space used: 71.34 GB of 77.36 GB (92.23 %)
Original problem ID: 106

You can also navigate to the Monitoring tab and then Dashboard to see the notification and its details.

Main dashboard

Now that you know the alerts are working, delete the temporary file you created so you can reclaim your disk space:

  • rm -f /tmp/temp.img

After a minute Zabbix will send the recovery message and the alert will disappear from the main dashboard.

Conclusion

In this tutorial, you learned how to set up a simple and secure monitoring solution that will help you monitor the state of your servers. It can now warn you of problems, and you have the opportunity to analyze the processes occurring in your IT infrastructure.

To learn more about setting up monitoring infrastructure, check out our Monitoring topic page.

How To Install Jenkins on Ubuntu 20.04

Introduction
When faced with repetitive technical tasks, finding automation solutions that work can be a chore. With Jenkins, an open-source automation server, you can efficiently manage tasks from building to deploying software. Jenkins is Java-based, installed from Ubuntu packages or by downloading and running its web application archive (WAR) file — a collection of files that make up a complete web application to run on a server.

In this tutorial we’ll install Jenkins on Ubuntu 20.04, start the development server and create an administrative user to get you started in exploring what Jenkins can do. While you’ll have a development-level server ready for use at the conclusion of this tutorial, to secure this installation for production, follow the guide How to Configure Jenkins with SSL Using an Nginx Reverse Proxy on Ubuntu 18.04.

Prerequisites
To follow this tutorial, you will need:

One Ubuntu 20.04 server configured with a non-root sudo user and firewall by following the Ubuntu 20.04 initial server setup guide. We recommend starting with at least 1 GB of RAM. Visit Jenkins’s “Hardware Recommendations” for guidance in planning the capacity of a production-level Jenkins installation.
Oracle JDK 11 installed, following our guidelines on installing specific versions of OpenJDK on Ubuntu 20.04.

Step 1 — Installing Jenkins
The version of Jenkins included with the default Ubuntu packages is often behind the latest available version from the project itself. To ensure you have the latest fixes and features, use the project-maintained packages to install Jenkins.

First, add the repository key to the system:

$ wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -

After the key is added the system will return with OK.

Next, let’s append the Debian package repository address to the server’s sources.list:

$ sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
After both commands have been entered, we’ll run update so that apt will use the new repository.
$ sudo apt update

Finally, we’ll install Jenkins and its dependencies.

$ sudo apt install jenkins

Now that Jenkins and its dependencies are in place, we’ll start the Jenkins server.

Step 2 — Starting Jenkins
Let’s start Jenkins by using systemctl:

$ sudo systemctl start jenkins

Since systemctl doesn’t display status output, we’ll use the status command to verify that Jenkins started successfully:

$ sudo systemctl status jenkins

If everything went well, the beginning of the status output shows that the service is active and configured to start at boot:

Output
● jenkins.service - LSB: Start Jenkins at boot time
   Loaded: loaded (/etc/init.d/jenkins; generated)
   Active: active (exited) since Fri 2020-06-05 21:21:46 UTC; 45s ago
     Docs: man:systemd-sysv-generator(8)
    Tasks: 0 (limit: 1137)
   CGroup: /system.slice/jenkins.service
Now that Jenkins is up and running, let’s adjust our firewall rules so that we can reach it from a web browser to complete the initial setup.

Step 3 — Opening the Firewall
To set up a UFW firewall, visit Initial Server Setup with Ubuntu 20.04, Step 4- Setting up a Basic Firewall. By default, Jenkins runs on port 8080. We’ll open that port using ufw:

$ sudo ufw allow 8080

Note: If the firewall is inactive, the following commands will allow OpenSSH and enable the firewall:

$ sudo ufw allow OpenSSH
$ sudo ufw enable

Check ufw’s status to confirm the new rules:

$ sudo ufw status

You’ll notice that traffic is allowed to port 8080 from anywhere:

Output
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
8080                       ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)
8080 (v6)                  ALLOW       Anywhere (v6)

With Jenkins installed and our firewall configured, we can complete the installation stage and dive into Jenkins setup.

Step 4 — Setting Up Jenkins
To set up your installation, visit Jenkins on its default port, 8080, using your server domain name or IP address: http://your_server_ip_or_domain:8080

You should receive the Unlock Jenkins screen, which displays the location of the initial password:

Unlock Jenkins screen

In the terminal window, use the cat command to display the password:

$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the 32-character alphanumeric password from the terminal and paste it into the Administrator password field, then click Continue.

The next screen presents the option of installing suggested plugins or selecting specific plugins:

Customize Jenkins Screen

We’ll click the Install suggested plugins option, which will immediately begin the installation process.

Jenkins Getting Started Install Plugins Screen

When the installation is complete, you’ll be prompted to set up the first administrative user. It’s possible to skip this step and continue as admin using the initial password we used above, but we’ll take a moment to create the user.

Note: The default Jenkins server is NOT encrypted, so the data submitted with this form is not protected. Refer to How to Configure Jenkins with SSL Using an Nginx Reverse Proxy on Ubuntu 20.04 to protect user credentials and information about builds that are transmitted via the web interface.

Jenkins Create First Admin User Screen

Enter the name and password for your user:

Jenkins Create User

You’ll receive an Instance Configuration page that will ask you to confirm the preferred URL for your Jenkins instance. Confirm either the domain name for your server or your server’s IP address:

Jenkins Instance Configuration

After confirming the appropriate information, click Save and Finish. You’ll receive a confirmation page confirming that “Jenkins is Ready!”:

Jenkins is ready screen

Click Start using Jenkins to visit the main Jenkins dashboard:

Welcome to Jenkins Screen

At this point, you have completed a successful installation of Jenkins.

Conclusion
In this tutorial, you installed Jenkins using the project-provided packages, started the server, opened the firewall, and created an administrative user. At this point, you can start exploring Jenkins.

When you’ve completed your exploration, follow the guide How to Configure Jenkins with SSL Using an Nginx Reverse Proxy on Ubuntu 20.04 to protect your passwords, as well as any sensitive system or product information that will be sent between your machine and the server in plain text to continue using Jenkins.

To learn more about what you can do using Jenkins, check out other tutorials on the subject:

How to Build Android Apps with Jenkins
How To Set Up Continuous Integration Pipelines in Jenkins on Ubuntu 16.04

How To Create a Self-Signed SSL Certificate for Apache in Ubuntu 20.04

Introduction
TLS, or “transport layer security” — and its predecessor SSL — are protocols used to wrap normal traffic in a protected, encrypted wrapper. Using this technology, servers can safely send information to their clients without their messages being intercepted or read by an outside party.

In this guide, we will show you how to create and use a self-signed SSL certificate with the Apache web server on Ubuntu 20.04.

Note: A self-signed certificate will encrypt communication between your server and any clients. However, because it is not signed by any of the trusted certificate authorities included with web browsers and operating systems, users cannot use the certificate to validate the identity of your server automatically. As a result, your users will see a security error when visiting your site.

Because of this limitation, self-signed certificates are not appropriate for a production environment serving the public. They are typically used for testing, or for securing non-critical services used by a single user or a small group of users that can establish trust in the certificate’s validity through alternate communication channels.

For a more production-ready certificate solution, check out Let’s Encrypt, a free certificate authority. You can learn how to download and configure a Let’s Encrypt certificate in our How To Secure Apache with Let’s Encrypt on Ubuntu 20.04 tutorial.

Prerequisites
Before starting this tutorial, you’ll need the following:

Access to a Ubuntu 20.04 server with a non-root, sudo-enabled user. Our Initial Server Setup with Ubuntu 20.04 guide can show you how to create this account.
You will also need to have Apache installed. You can install Apache using apt. First, update the local package index to reflect the latest upstream changes:

 $ sudo apt update

Then, install the apache2 package:

 $ sudo apt install apache2

And finally, if you have a ufw firewall set up, open up the http and https ports:

 $ sudo ufw allow "Apache Full"

After these steps are complete, be sure you are logged in as your non-root user and continue with the tutorial.

Step 1 — Enabling mod_ssl
Before we can use any SSL certificates, we first have to enable mod_ssl, an Apache module that provides support for SSL encryption.

Enable mod_ssl with the a2enmod command:

$ sudo a2enmod ssl
Restart Apache to activate the module:

$ sudo systemctl restart apache2
The mod_ssl module is now enabled and ready for use.

Step 2 – Creating the SSL Certificate
Now that Apache is ready to use encryption, we can move on to generating a new SSL certificate. The certificate will store some basic information about your site, and will be accompanied by a key file that allows the server to securely handle encrypted data.

We can create the SSL key and certificate files with the openssl command:

 $ sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/apache-selfsigned.key -out /etc/ssl/certs/apache-selfsigned.crt

After you enter the command, you will be taken to a prompt where you can enter information about your website. Before we go over that, let’s take a look at what is happening in the command we are issuing:

openssl: This is the command line tool for creating and managing OpenSSL certificates, keys, and other files.
req -x509: This specifies that we want to use X.509 certificate signing request (CSR) management. X.509 is a public key infrastructure standard that SSL and TLS adhere to for key and certificate management.
-nodes: This tells OpenSSL to skip the option to secure our certificate with a passphrase. We need Apache to be able to read the file, without user intervention, when the server starts up. A passphrase would prevent this from happening, since we would have to enter it after every restart.
-days 365: This option sets the length of time that the certificate will be considered valid. We set it for one year here. Many modern browsers will reject any certificates that are valid for longer than one year.
-newkey rsa:2048: This specifies that we want to generate a new certificate and a new key at the same time. We did not create the key that is required to sign the certificate in a previous step, so we need to create it along with the certificate. The rsa:2048 portion tells it to make an RSA key that is 2048 bits long.
-keyout: This line tells OpenSSL where to place the generated private key file that we are creating.
-out: This tells OpenSSL where to place the certificate that we are creating.
Fill out the prompts appropriately. The most important line is the one that requests the Common Name. You need to enter either the hostname you’ll use to access the server by, or the public IP of the server. It’s important that this field matches whatever you’ll put into your browser’s address bar to access the site, as a mismatch will cause more security errors.

The full list of prompts will look something like this:

Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:Example
Locality Name (eg, city) [Default City]:Example 
Organization Name (eg, company) [Default Company Ltd]:Example Inc
Organizational Unit Name (eg, section) []:Example Dept
Common Name (eg, your name or your server's hostname) []:your_domain_or_ip
Email Address []:[email protected]
Both of the files you created will be placed in the appropriate subdirectories under /etc/ssl.

Next we will update our Apache configuration to use the new certificate and key.

Step 3 – Configuring Apache to Use SSL
Now that we have our self-signed certificate and key available, we need to update our Apache configuration to use them. On Ubuntu, you can place new Apache configuration files (they must end in .conf) into /etc/apache2/sites-available/and they will be loaded the next time the Apache process is reloaded or restarted.

For this tutorial we will create a new minimal configuration file. (If you already have an Apache set up and just need to add SSL to it, you will likely need to copy over the configuration lines that start with SSL, and switch the VirtualHost port from 80 to 443. We will take care of port 80 in the next step.)

Open a new file in the /etc/apache2/sites-available directory:

 $ sudo nano /etc/apache2/sites-available/your_domain_or_ip.conf

Paste in the following minimal VirtualHost configuration:

/etc/apache2/sites-available/your_domain_or_ip.conf

   ServerName your_domain_or_ip
   DocumentRoot /var/www/your_domain_or_ip

   SSLEngine on
   SSLCertificateFile /etc/ssl/certs/apache-selfsigned.crt
   SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key

Be sure to update the ServerName line to however you intend to address your server. This can be a hostname, full domain name, or an IP address. Make sure whatever you choose matches the Common Name you chose when making the certificate.

The remaining lines specify a DocumentRoot directory to serve files from, and the SSL options needed to point Apache to our newly-created certificate and key.

Now let’s create our DocumentRoot and put an HTML file in it just for testing purposes:

$ sudo mkdir /var/www/your_domain_or_ip

Open a new index.html file with your text editor:

$ sudo nano /var/www/your_domain_or_ip/index.html

Paste the following into the blank file:

/var/www/your_domain_or_ip/index.html

it worked!

This is not a full HTML file, of course, but browsers are lenient and it will be enough to verify our configuration.

Save and close the file
Next, we need to enable the configuration file with the a2ensite tool:

$ sudo a2ensite your_domain_or_ip.conf

Next, let’s test for configuration errors:

$ sudo apache2ctl configtest

If everything is successful, you will get a result that looks like this:

Output
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
Syntax OK

The first line is a message telling you that the ServerName directive is not set globally. If you want to get rid of that message, you can set ServerName to your server’s domain name or IP address in /etc/apache2/apache2.conf. This is optional as the message will do no harm.

If your output has Syntax OK in it, your configuration file has no syntax errors. We can safely reload Apache to implement our changes:

$ sudo systemctl reload apache2

Now load your site in a browser, being sure to use https:// at the beginning.

You should see an error. This is normal for a self-signed certificate! The browser is warning you that it can’t verify the identity of the server, because our certificate is not signed by any of its known certificate authorities. For testing purposes and personal use this can be fine. You should be able to click through to advanced or more information and choose to proceed.

After you do so, your browser will load the it worked! message.

Note: if your browser doesn’t connect at all to the server, make sure your connection isn’t being blocked by a firewall. If you are using ufw, the following commands will open ports 80 and 443:

$ sudo ufw allow "Apache Full"

Next we will add another VirtualHost section to our configuration to serve plain HTTP requests and redirect them to HTTPS.

Step 4 — Redirecting HTTP to HTTPS
Currently, our configuration will only respond to HTTPS requests on port 443. It is good practice to also respond on port 80, even if you want to force all traffic to be encrypted. Let’s set up a VirtualHost to respond to these unencrypted requests and redirect them to HTTPS.

Open the same Apache configuration file we started in previous steps:

$ sudo nano /etc/apache2/sites-available/your_domain_or_ip.conf

At the bottom, create another VirtualHost block to match requests on port 80. Use the ServerName directive to again match your domain name or IP address. Then, use Redirect to match any requests and send them to the SSL VirtualHost. Make sure to include the trailing slash:

/etc/apache2/sites-available/your_domain_or_ip.conf

    ServerName your_domain_or_ip
    Redirect / https://your_domain_or_ip/

Save and close this file when you are finished, then test your configuration syntax again, and reload Apache:

$ sudo apachectl configtest
$ sudo systemctl reload apache2

You can test the new redirect functionality by visiting your site with plain http:// in front of the address. You should be redirected to https:// automatically.

Conclusion
You have now configured Apache to serve encrypted requests using a self-signed SSL certificate, and to redirect unencrypted HTTP requests to HTTPS.

If you are planning on using SSL for a public website, you should look into purchasing a domain name and using a widely supported certificate authority such as Let’s Encrypt.

For more information on using Let’s Encrypt with Apache, please read our How To Secure Apache with Let’s Encrypt on Ubuntu 20.04 tutorial.

How To Install and Use GoAccess Web Log Analyzer on Ubuntu 20.04

The author selected the Internet Archive to receive a donation as part of the Write for DOnations program.

Introduction
GoAccess is a tool for monitoring web server logs in realtime. It’s written in C and uses the popular ncurses library for its dashboard interface, which can be accessed directly from the command-line.

This is great because you’re able to SSH into any web server you control and view or analyze relevant statistics quickly and securely. Apart from the command-line dashboard interface, it’s also capable of displaying the statistics in other formats such as HTML, JSON, and CSV, which you can use in other contexts or share with others.

GoAccess could also be a great alternative to client-side analytics tools depending on your needs. It analyzes your server logs directly, so you don’t need to load any additional scripts, and your data is completely under your control.

In this tutorial, you’ll install and configure GoAccess for Apache on an Ubuntu 20.04 web server. You’ll access the Apache log files with GoAccess before reviewing the modules available and navigation shortcuts on the command-line interface.

Prerequisites
For this tutorial, you’ll need the following:

One Ubuntu 20.04 server. You can set it by following this initial server setup for Ubuntu 20.04 tutorial, including a non-root user with sudo privileges and a firewall.
Apache installed by following How To Install Apache on Ubuntu 20.04.
Step 1 — Installing GoAccess
In this step you’ll install the GoAccess tool and its dependencies.

Start by ensuring that the package database and system are up to date:

$ sudo apt update
$ sudo apt full-upgrade

Now it’s time to install GoAccess. A version of the tool is available in the Ubuntu repos, but this is not usually the latest stable version. For example, the latest version of GoAccess at the time of writing is 1.4, while the version available from the Ubuntu 20.04 repos is 1.3.

To ensure that you have the latest stable version of GoAccess installed on your server, you can compile from source or use the official GoAccess repository on Ubuntu.

Method 1 — Compiling from source
First, install the dependencies required to compile GoAccess from source:

$ sudo apt install libncursesw5-dev libgeoip-dev libtokyocabinet-dev build-essential

You install the following dependencies:

build-essential: installs many packages, which includes gcc compilers for C, C+, and other programming languages, and make for building the GoAccess makefile.
libncursesw5-dev: installs the ncurses library that GoAccess uses for its command-line interface.
libgeoip-dev: includes the necessary files for the GeoIP library.
libtokyocabinet-dev: provides database dependencies for higher performance.
Next, download the latest version of the GoAccess from their official website with the following command:

$ wget http://tar.goaccess.io/goaccess-1.4.tar.gz

Once the download completes, extract the archive with:

$ tar -xzvf goaccess-1.4.tar.gz

Change into the newly unpacked directory like this:

$ cd goaccess-1.4/

Run the configure script found inside this directory:

$ ./configure --enable-utf8 --enable-geoip=legacy

The –enable-utf8 flag ensures GoAccess compiles with wide character support, while –enable-geoip enables GeoLocation support with the original GeoIP databases. You can replace legacy with mmdb to use the enhanced GeoIP2 databases instead. You can find other configuration options on the GoAccess website.

You’ll receive output similar to the following:

Output
. . .
Your build configuration:

Prefix : /usr/local
Package : goaccess
Version : 1.4
Compiler flags : -pthread
Linker flags : -lnsl -lncursesw -lGeoIP -lpthread
UTF-8 support : yes
Dynamic buffer : no
Geolocation : GeoIP Legacy
Storage method : In-Memory with On-Disk Persitance Storage
TLS/SSL : no
Bugs : [email protected]

Run the make command to build the makefile required for installing GoAccess:

$ make

Finally, install GoAccess using the previously created makefile to the system:

$ sudo make install

Ensure that the program was installed successfully by running:

goaccess –version
You will receive the following output:

$ Output
GoAccess - 1.4.
For more details visit: http://goaccess.io
Copyright (C) 2009-2020 by Gerardo Orellana

Build configure arguments:
--enable-utf8
--enable-geoip=legacy

Method 2 — Using the Official GoAccess Repos
Another way to install GoAccess is by using the official Ubuntu repository for the program. This method is preferable if you’d like it to be updated to a newer version automatically during system upgrades without having to compile from source for each new release. You need to add the repository to your server first:

$ echo "deb http://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list

First you get the release name of the distribution and then pipe that to tee, which appends to the file /etc/apt/sources.list.d/goaccess.list.

With the repository in your sources list, you can now download the GPG key to verify the signature:

$ wget -O - https://deb.goaccess.io/gnugpg.key | sudo apt-key --keyring /etc/apt/trusted.gpg.d/goaccess.gpg add -

Next, update the package database with the following command:

$ sudo apt update

Finally, install GoAccess:

$ 

sudo apt install goaccess
GoAccess is now installed on your Ubuntu server. In the next step, you’ll access and edit its configuration file so that you can make changes to how the program runs.

Step 2 — Editing the GoAccess Configuration
GoAccess comes with a configuration file where you can make permanent changes to the behavior of the program. You’ll edit this file to specify the time, date, and log format so that GoAccess knows how to parse the server logs.

The configuration file may be located at ~/.goaccessrc or %sysconfdir%/goaccess.conf where %sysconfdir% is either /etc/, /usr/etc/, or /usr/local/etc/. To find out where the config file is located on your server, run the following command:

goaccess --dcf

Sample output
/etc/goaccess/goaccess.conf
Edit this config file using nano:
$ sudo nano /etc/goaccess/goaccess.conf

Note: If this file does not exist on the server, ensure to create it first and populate it with the contents of the goaccess.conf file on GitHub.

Many of the lines in the file are commented out. To enable an option, remove the first # character in front of it. Let’s enable the time-format setting for Apache first. This setting specifies the log-format time and allows GoAccess to parse any plain-text Apache log files that meet the supported formatting criteria.

/etc/goaccess/goaccess.conf
# The following time format works with any of the
# Apache/NGINX’s log formats below.
#
time-format %H:%M:%S
Next, you’ll uncomment the Apache date-format setting that specifies the log-format date:

/etc/goaccess/goaccess.conf
# The following date format works with any of the
# Apache/NGINX’s log formats below.
#
date-format %d/%b/%Y
Finally, uncomment the log-format setting. Several lines change this setting and the exact one to uncomment depends on the way your web server is set up. If you have a non-virtual hosts setup, uncomment the following log-format line:

/etc/goaccess/goaccess.conf
# NCSA Combined Log Format
log-format %h %^[%d:%t %^] “%r” %s %b “%R” “%u”
Otherwise, if you have virtual hosts set up, uncomment the following line instead:

/etc/goaccess/goaccess.conf
# NCSA Combined Log Format with Virtual Host
log-format %v:%^ %h %^[%d:%t %^] “%r” %s %b “%R” “%u”
At this point, you can save the file and exit the editor. You are now ready to run the GoAccess program and analyze some Apache plain-text log files.

Step 3 — Accessing Apache’s Log Files with GoAccess
The Apache server grants access to your website and keeps an access log for all incoming HTTP traffic. These records, or log files, are stored on the system and can be a valuable source of information about your website’s usage and audience.

On Ubuntu, the Apache log files are stored in the /var/log/apache2 directory by default. To inspect the contents of this directory, run the following command:

$ sudo ls /var/log/apache2

Sample output
access.log error.log other_vhosts_access.log
If your server has been running for a long time, you may find compressed .gz files in this directory containing past log files as a result of log rotation. The most recent logs are placed in an access.log file. For web servers with virtual hosts, you may have to cd into sub-directories from within the /apache2 directory to locate each host’s log files.

Let’s go ahead and run GoAccess against the Apache access logs to gain insight into what type of traffic is being handled by the web server. Run the following command to analyze your access.log file with GoAccess:

$ sudo goaccess /var/log/apache2/access.log

This will launch the GoAccess command-line dashboard.

GoAccess command-line dashboard interface

Note: If you see a Log Format Configuration prompt instead, it means that the changes you made to the GoAccess config file in the previous step are not taking effect. Ensure that your the config file is in the right place and that you have uncommented the necessary settings.

As mentioned previously, you will sometimes have several compressed log files on a long-running web server. To run GoAccess on all these files without extracting them first, you can pipe the output of the zcat command to goaccess:

zcat /var/log/apache2/access.log.*.gz | goaccess -a
Next you’ll learn how to quickly navigate through the dashboard interface with keyboard shortcuts.

Step 4 — Navigating the Terminal Dashboard
At the top of the dashboard is a summary of several key metrics. This includes total requests for the reporting period, unique visitors, log size, 404 not found errors, requested files, size of the parsed log file, HTTP referrers, name of the log source, time taken to process the log file, and more.

Summary of dashboard metrics

Below the top panel, you will find all the available modules which provide more details on the aforementioned metrics and other data points supported by GoAccess. To navigate the interface, use the following keyboard shortcuts:

TAB to move forward through the available modules and SHIFT+TAB to move backwards.
F5 to refresh the dashboard.
g to move to the top of the dashboard screen and G to move to the last item in the dashboard.
o or ENTER to expand the selected module.
j and k to scroll down and up within the active module.
s to display the sort options for the active module.
/ to search across all modules and n to move to the next match.
0-9 and SHIFT+0 to quickly activate the respective numbered module.
? to view the quick help dialog.
q to quit the program.
Let’s examine each of the available modules on the dashboard next. Each one has a number and a title, and an indication of the total number of lines present. The > character indicates the active panel, which is also reflected at the top of the dashboard.

Active GoAccess panel demonstration

Here’s a brief explanation of each of the panels. Each section below correspond to the panel number and title in the program.

1 — Unique Visitors per Day
This panel displays the hits, unique visitors, and cumulative bandwidth for each reported date. A unique visitor is considered to be one with the same IP address, date, and user-agent. It includes web crawlers and spiders by default.

Unique visitors per day panel

2 – Requested Files (URLs)
This panel provides the statistics concerning the most highly requested non-static files on your web server. It displays the request path, HTTP protocol and method, unique visitors, number of hits, and cumulative bandwidth.

Requested files

3 — Static Requests
This panel provides the same metrics as the previous one, but for static files such as images, CSS, JavaScript, or other file types.

4 — Not Found URLs (404s)
This panel also displays the same metrics discussed in 2 and 3, but for paths that were not found on the server (404s).

5 — Visitor Hostnames and IPs
This panel provides detailed information on the hosts that connect to your web server. You can find their IP address, the number of visits, and the amount of bandwidth consumed. This is a great way to identify who is eating up all your bandwidth and block them if necessary.

Visitor hostnames and IPs

If you expand this panel by pressing o, you will see more info about each host such as its country of origin, city, and reverse DNS lookup result.

Vistor hostnames and IPs expanded

6 — Operating Systems
This panel reports the different operating systems used by the hosts to connect to your web server. Expanding this panel will display specific versions of each operating system.

Operating systems

7 — Browsers
Similar to the previous panel, this reports the browsers used by each unique visitor to your web server and lists specific versions for each browser once expanded.

Browsers

8 — Time distribution
Here, you will find an hourly report for the number of hits, unique visitors, and bandwidth consumed. This is a great way to spot periods of peak traffic on your server.

Time distribution panel

9 — Virtual Hosts
This panel displays the virtual hosts parsed from the log file. It becomes active only if %v is included in the log-format configuration.

10 — Referrer URLs
The URLs that referred the visiting hosts to your web server are reflected here. This panel is disabled by default and can only be enabled by commenting out the REFERRERS line highlighted following in the GoAccess config file:

/etc/goaccess/goaccess.conf
#ignore-panel VISIT_TIMES
#ignore-panel VIRTUAL_HOSTS
#ignore-panel REFERRERS
#ignore-panel REFERRING_SITES
Referrer URLs panel

11 — Referring Sites
This panel displays the IP address of the referring hosts, but not the whole URL.

12 — Keyphrases
Here, the keywords used on Google search, Google cache, and Google translate that led to your website are reported. This panel is also disabled by default and must be enabled in the settings:

/etc/goaccess/goaccess.conf
#ignore-panel REFERRERS
#ignore-panel REFERRING_SITES
#ignore-panel KEYPHRASES
#ignore-panel STATUS_CODES

13 — HTTP Status Codes
This panel reflects the overall statistics for HTTP status codes returned by your web server when responding to a request. Expanding the panel will display the aggregated stats for each status code.

HTTP status codes panel

14 — Remote User (HTTP Authentication)
This panel displays the user ID of the person requesting a document on your server as determined by HTTP authentication. For documents that are not password protected, this part will be -. Note that this panel is only enabled if %e is part of the log-format configuration.

15 — Cache status
This panel allows you to determine if a request is being cached and served from the cache. It is enabled if %C is part of the log-format variable, and the status could be MISS, BYPASS, EXPIRED, STALE, UPDATING, REVALIDATED, or HIT.

16 — Geo Location
This panel provides a summary of the geographical locations derived from visiting IP addresses. Expanding this panel will display the aggregated stats for each country of origin.

Geo location panel

You’ve reviewed the panels available in the dashboard, now you’ll generate reports in different formats.

Step 5 — Generating Reports
Aside from displaying the data in the terminal, GoAccess also allows you to generate HTML, JSON, or CSV reports. Make sure that you’re in the home directory before running any of the commands in this section:

cd ~
To output the report as static HTML, specify an HTML file as the argument to the -o flag. This flag also accepts filenames that end in .json or .csv.

$ sudo goaccess /var/log/apache2/access.log -o stats.html

A stats.html file should appear in your user directory.

ls

Output
goaccess-1.4 goaccess-1.4.tar.gz snap stats.html

You can copy this file to the user directory on your local machine using scp. Run this command from your local machine, and not the remote server:

scp user@your_server_ip:stats.html ~/stats.html

Once the file has been copied over, you can open it in your browser with the open command on macOS:

open ~/stats.html

Or if you’re using a Linux distribution on your local machine:

xdg-open ~/stats.html
HTML report in Firefox<

You’ve generated a HTML report and viewed this in your browser.

Conclusion
In this article, we covered the GoAccess command-line tool and discussed how to use it for analyzing server logs. Although we only considered how GoAccess may be used with Apache logs, the tool also supports other log formats such as Nginx, Amazon S3, Elastic Load Balancing, and CloudFront.

You can check the full GoAccess documentation or run man goaccess in your terminal.

How to Install PyCharm on Ubuntu 18.04

In this article we will learn How to Install PyCharm on Ubuntu 18.04.

PyCharm is a Python IDE for Professional Developers. You can use Professional or Free Community version.

install pycharm on ubuntu 18.04

PyCharm Features

Intelligent Coding Assistance
PyCharm provides smart code completion, code inspections, on-the-fly error highlighting and quick-fixes, along with automated code refactorings and rich navigation capabilities.

Built-in Developer Tools
PyCharm’s huge collection of tools out of the box includes an integrated debugger and test runner; Python profiler; a built-in terminal; integration with major VCS and built-in database tools; remote development capabilities with remote interpreters; an integrated ssh terminal; and integration with Docker and Vagrant.

Web Development
In addition to Python, PyCharm provides first-class support for various Python web development frameworks, specific template languages, JavaScript, CoffeeScript, TypeScript, HTML/CSS, AngularJS, Node.js, and more.

Scientific Tools
PyCharm integrates with IPython Notebook, has an interactive Python console, and supports Anaconda as well as multiple scientific packages including Matplotlib and NumPy.

System requirements:

    • Any 64-bit Linux distribution with Gnome, KDE, or Unity
    • 4 GB RAM minimum, 8 GB RAM recommended
    • 1.5 GB hard disk space + at least 1 GB for caches
    • 1024×768 minimum screen resolution
    • Python 2.7, or Python 3.5 or newer

Watch this video to learn how to install PyCharm on Ubuntu

Update all your system packages:

sudo apt-get update
sudo apt-get upgrade

Method-1: Direct Download

Go to PyCharm link to download for Ubuntu 18.04

Installation Instructions:
1. Copy the pycharm-2019.3.tar.gz to the desired installation location
(make sure you have rw permissions for that directory)

2. Unpack the pycharm-2019.3.tar.gz file to an empty directory using the following command: tar -xzf pycharm-2019.3.tar.gz

Note: A new instance MUST NOT be extracted over an existing one. The target folder must be empty

3. Run pycharm.sh from the bin subdirectory

Method-2: Install with Command line

sudo snap install [pycharm-professional|pycharm-community] --classic

If you want to use free version:

sudo snap install pycharm-community --classic

Successful Installation Message:

pycharm-community 2019.3 from jetbrains installed

Method-3: Install from Ubuntu Software

Step -1: Open Ubuntu software.

Step-2: Search “PyCharm”

Step-3: Click on “Install”

install pycharm on ubuntu 18.04

Learn how to install Atom on Ubuntu 18.04

How To Install and Configure the OpenLiteSpeed Web Server on Ubuntu 18.04

install openlitespeed server on ubuntu 18.04

In this article, we’ll learn how to install and configure OpenLiteSpeed on an Ubuntu 18.04 server.

install openlitespped server ubuntu 18OpenLiteSpeed is the Open Source edition of LiteSpeed Web Server Enterprise. OpenLiteSpeed contains all of the essential features found in LiteSpeed Enterprise, and represents our commitment to support the Open Source community. It features Apache-compatible rewrite rules, a built-in web-based administration interface, and customized PHP processing optimized for the server.

OpenLiteSpeed Features:

Event-Driven Architecture
Fewer processes, less overhead, and enormous scalability. Keep your existing hardware.

Understands Apache Rewrite Rules
OpenLiteSpeed is mod_rewrite compatible, with no new syntax to learn. Continue to use your existing rewrite rules.

Friendly Admin Interfaces
OLS comes with a built-in WebAdmin GUI. Control panel support is available with CyberPanel.

Built for Speed and Security
Features Anti-DDoS connection and bandwidth throttling, ModSecurity v3 integration, and more.

Intelligent Cache Acceleration
Built-in full-page cache module is highly-customizable and efficient for an exceptional user experience.

PageSpeed Optimization
Automatically implement Google’s PageSpeed optimization system with the mod_pagespeed module.

PHP LiteSpeed SAPI
Native SAPI for PHP allows external applications written in PHP to run up to 50% faster.

WordPress Acceleration
Experience a measurable performance boost with OpenLiteSpeed and LSCache for WordPress.

Step 1 – Installing OpenLiteSpeed on Ubuntu 18.04

OpenLiteSpeed provides a software repository we can use to download and install the server with Ubuntu’s standard apt command.

Update all your system packages:

sudo apt-get update
sudo apt-get upgrade

Download and add the developer’s software signing key:

$ wget -qO - https://rpms.litespeedtech.com/debian/lst_repo.gpg | sudo apt-key add -

Now we will add the repository information to our system:

$ sudo add-apt-repository 'deb http://rpms.litespeedtech.com/debian/ bionic main'

Install the OpenLiteSpeed server and its PHP processor using apt install:

$ sudo apt install openlitespeed lsphp73

Finally, create a soft link to the PHP processor we just installed. This directs the OpenLiteSpeed server to use the correct version:

$ sudo ln -sf /usr/local/lsws/lsphp73/bin/lsphp /usr/local/lsws/fcgi-bin/lsphp5

 

OpenLiteSpeed server is installed, we’ll secure it by updating the default admin account.

Step 2 – Configure OpenLiteSpeed & Setting the Admin Password

Configure the administrative password for OpenLiteSpeed web server. By default, the password is set to 123456, so we should change this immediately. We can do this by running a script provided by OpenLiteSpeed:

sudo /usr/local/lsws/admin/misc/admpass.sh

Provide a username for the administrative user as below:

Please specify the user name of administrator.
This is the user name required to login the administration Web interface.
 
User name [admin]: wpcademyadmin
 
Please specify the administrator's password. This is the password required to login the administration Web interface.
 
Password:
Retype password:

Administrator's username/password is updated successfully!

Step 3 – Accessing the OpenLiteSpeed Web Server

OpenLiteSpeed should have started automatically. We can verify this with the following command:

$ sudo /usr/local/lsws/bin/lswsctrl status

Output:

litespeed is running with PID 990.
sudo /usr/local/lsws/bin/lswsctrl start

Output:

[OK] litespeed: pid=5137.

We need to open up some ports on our firewall. Configure Firewall for Port Access. Add the firewall rules:

$ sudo ufw allow http
$ sudo ufw allow https
$ sudo ufw allow 8088
$ sudo ufw allow 7080

ReloAd ufw to effect the changes:

sudo ufw reload

In your web browser, navigate to your server’s domain name or IP address, followed by :8088 to specify the port:

http://server_domain_or_IP:8088

Browser should load the default OpenLiteSpeed web page as seen below:
install openlitespeed ubuntu 18.04

To configure the administrative interface. Got to your web browser, using HTTPS, navigate to your server’s domain name or IP address followed by :7080 to specify the port:

https://server_domain_or_IP:7080

install openlitespeed ubuntu 18.04 and configure admin interface

Enter the admin logins you had created during the OpenLiteSpeed configuration. Once you correctly authenticate, you will be presented with the OpenLiteSpeed administration interface:

openlitespeed-admin-dashboard

Step 4 – Configuring the Port

In the list of listeners, click the “View/Edit” button for the Default listener:

litesped listeners summary port config

Click the edit button in the top-right corner of the “Address Settings” table to modify its values:
modify listener value port

On the next page, then click the floppy disk icon, Save.

change port 8088 to port 80

You’ll need to now open up port 80 on your firewall:

$ sudo ufw allow 80

The default web page should now be accessible in your browser on port 80 instead of port 8088.

Congratulations! You should have OpenLiteSpeed and PHP installed and running on an Ubuntu 18.04 server.

You are running Ubuntu 16.04 : Install OpenLiteSpeed on Ubuntu 16.04

Visit OpenLiteSpeed Official Website fore more details.