How to set up monitoring with Zabbix

Created by Cyvan Oort at 01-01-2018 11:19:49 +0100

Zabbix is an open source Enterprise-Class monitoring system. We have used Zabbix for our monitoring solutions for the past years and in this article we will explain how to set up your own monitoring solution.

Steps

  1. Creating the container
  2. Installing Zabbix server
  3. Configuring front end
    1. Change Admin password
    2. Disable the guest user
  4. Configuring server
  5. Installing the agent
    1. PSK authentication
  6. Adding the agent to Zabbix server

1. Creating the container

Create an Medium ubuntu package on https://www.cloudcontainers.net/ for an basic installation of Zabbix server. If you are planning on monitoring a larger infrastructure I would advise to take a look at the Zabbix official manual to calculate the required resources:
https://www.zabbix.com/documentation/3.4/manual/installation/requirements

Log in to your new container with the SSH protocol via the command line. When creating a new LX container, the username will always be 'root'. If you have a KVM container, the username you need to log in with will be 'ubuntu'.

ssh username@ip

It is important with a new container to always update and upgrade before doing anything else. If prompted to continue, type 'y' or 'yes' and press enter.

apt-get update
apt-get upgrade

Before continuing with the following steps, make sure to set the hostname and reboot the zone. This will make sure we are running on the latest packages.

hostname 
reboot

2. Installing Zabbix server

First of download the newest version from the official repo:

wget http://repo.zabbix.com/zabbix/3.4/ubuntu/pool/main/z/zabbix-release/zabbix-release_3.4-1+xenial_all.deb
dpkg -i zabbix-release_3.4-1+xenial_all.deb

Now we have added the latest production version (on date of writing) to our server.
By default we will install zabbix with a MySQL server.

sudo apt-get update
sudo -E apt-get -q -y install zabbix-server-mysql zabbix-frontend-php

As this process has been completed you'll still need to configure the mysql database, replace PASS within the example with a password of your own liking.

echo "create database zabbix character set utf8 collate utf8_bin;" > /var/tmp/zabbix.sql
echo "grant all privileges on zabbix.* to zabbix@localhost identified by 'PASS';" >> /var/tmp/zabbix.sql
echo "flush privileges;" >> /var/tmp/zabbix.sql
mysql -u root < /var/tmp/zabbix.sql

zcat /usr/share/doc/zabbix-server-mysql/create.sql.gz | mysql -uzabbix -p"PASS" zabbix

Now the database is configured and filled with the required Zabbix data. it's time to let Zabbix know.

Within the Zabbix server config file uncomment and update the following lines:

DBHost=localhost
DBPassword=PASS
FpingLocation=/usr/bin/fping

That should be enough to get the zabbix server up and running

service zabbix-server start
update-rc.d zabbix-server enable

3. Configure Front End

As you could see in the first chapter we have installed the zabbix-frontend-php package. this package contains the Zabbix dashboard. but if you visit your ip address in your browser you might not see this, because it's not configured correctly.

First of all lets set the timezone.

sed -i "s:# php_value date.timezone Europe/Riga:php_value date.timezone Europe/Amsterdam:" /etc/apache2/conf-enabled/zabbix.conf
service apache2 restart

Next we need to tell the Zabbix dashboard where to find the database settings, create a file in /etc/zabbix/web/zabbix.conf.php with the following content, only replace PASS with the same password as you have used in chapter one and place it between php tags.

now if you would open the url: http://your_ip/zabbix in your browser it should show you an login screen like this:

The Zabbix login screen

The default credentials are:

Username: Admin
Password: zabbix

Congratulations you now have a working Zabbix installation! before we are going to add hosts to monitor it, we might want to change some settings to secure your Zabbix installation.

3.1 Changing the Admin password

within the Zabbix dashboard click on Administration and then Users.

If you do not see the Administration button it means that you are not logged in correctly! click on the upper right corner on the login button to go back to the login screen. 

login button

If you have clicked on the Users menu item you should see an overview of all users


Click on the username Admin and the details screen will show

Simply press the Change password button, fill in the new password twice and click update to save.

3.2 Disable the Guest account

You might want to disable the guest account because even with little permissions it could show some data to the world that you want to keep private. So again repeat the steps of 3.1 to get to the user details screen only this time pick the account name Guest.

To disable the user click on the Select button next to groups, an popup window will open with all available groups.

Select the groups 'Disabled' and 'No access to the frontend'

Click select within the popup and update within the user details screen.

4. Configuring the server

Within the first chapter we have edited the Zabbix server configuration with the database settings, now we are going to take a look at some other settings, these will have an impact on the Zabbix server performance and features.

the first setting is an poller for unreachable hosts, uncomment the following value:

StartPollersUnreachable=1

One of the main functionalities of Zabbix is sending alerts. These Alerts requires there is an Alerter running so uncomment the line:

StartAlerters=3

These are the basic changes required to run a Zabbix server with alerting options.

5. Installing the Agent

Just like for the Zabbix server the default agent within the ubuntu repo's is outdated, so we are going to add the zabbix repo to get the latest version

wget http://repo.zabbix.com/zabbix/3.4/ubuntu/pool/main/z/zabbix-release/zabbix-release_3.4-1+xenial_all.deb
dpkg -i zabbix-release_3.4-1+xenial_all.deb

Now it's time to install Zabbix agent

apt-get -y update
apt-get -y install zabbix-agent

After the install has been completed we still need to change a few settings to get the agent ready to send data to the server.

first lets add the Zabbix server to the hosts file:

echo "ZABBIX_IP_HERE zabbix-server" >> /etc/hosts

Next it's time to configure the agent, within /etc/zabbix/zabbix_agentd.conf edit the following lines:

LogFile=/var/log/zabbix/zabbix_agentd.log
EnableRemoteCommands=1
Server=zabbix-server
StartAgents=5
ServerActive=zabbix-server
Hostname=YOUR_HOSTNAME
Timeout=20

The logfile location should exists and be owned by Zabbix, if this does not exists you may create it with:

mkdir /var/log/zabbix && chown zabbix /var/log/zabbix

The EnableRemoteCommands option will allow the Zabbix server to execute commands on the client. This will enable more possible checks and commands on failure of services.

The Server and ServerActive will whitelist these hosts for the Zabbix agent. To complete the changes within the Zabbix agent we need to restart the zabbix agent

systemctl restart zabbix-agent.service


5.1 PSK authentication

These days authentication is an key component of every IT infrastructure, and out of the box Zabbix supports multiple authentication methods, we are going to enable PSK authentication within the agent.

First we need to generate a key, this key will be used for the authentication and will be saved within the Zabbix agent and needs to be added to the Zabbix dashboard later on.

openssl rand -hex 32
99823d6c5ac69d5082ffc4cbe56ddcdebb93947df3cc2e6100d726079f14f081

Now we have the key we need to save it for the zabbix agent to use

echo "99823d6c5ac69d5082ffc4cbe56ddcdebb93947df3cc2e6100d726079f14f081" > /etc/zabbix/zabbix_agentd.psk

Right, now that the key has been saved it's time to let the Zabbix agent know, again edit the Zabbix agent config file /etc/zabbix/zabbix_agentd.conf

TLSConnect=psk
TLSAccept=psk
TLSPSKIdentity=YOUR_HOSTNAME
TLSPSKFile=/etc/zabbix/zabbix_agentd.psk

Again restart the Zabbix agent to complete the changes

systemctl restart zabbix-agent.service

6. Adding the agent to the Zabbix server

Now that we have a fully functional Zabbix server and agent it's time to make sure we get the some data into Zabbix! First login to your Zabbix dashboard ( http://YOUR_IP/zabbix ), this time click on configuration then hosts.

In the upper left corner you should see a button with 'Create host' if we click this we will be redirected to the following screen.

A host must have a Group and at least one interface. Since we have installed an Zabbix agent before we will use that one.

Fill in the form with as much information as you have, you may add multiple groups to the servers. In just below the title you see an tabbed menu bar, here we can add additional information. The templates will add monitoring templates to your server, as default Zabbix has multiple useful base templates but we recommend you to check them and edit them to your liking.

Since we have added PSK authentication to our host we need to make sure the Zabbix server knows that too. click within the tabbed menu on Encryption and you will see the following screen

As said before we have configured PSK encryption so we will need to also add PSK encryption in the Zabbix server. Select both PSK encryption for the "Connections to host" and "Connections from host" and disable the "No encryption". You will now see two additional fields with information about the PSK encryption. Give them the same values as you have done for step 5.1 and it should look something like

Next you can click on Add to add the host to the Zabbix Server.

If everything went correctly you should see the host overview with an entry you have just added. this entry should look something along the line of:

Within a couple of minutes the ZBX flag and the PSK flags should get Green if all is well. This means the host has been found by Zabbix and is being monitored. If something went wrong the flags should get Red. In this case you can hover your mouse over it the red flag and it should give you an error message. These message should help you debug what went wrong.

Comments