Tagged: Monitoring

Monit is a Linux Daemon designed to monitor systems internally, can email you when they occur, and provides a nice web interface on port 2812. Additionally, it checks on an interval (every 2 minutes by default) that helps keep your server awake.

It also has built-in functionality to mitigate problems automatically, such as restarting services.

Important Note: The config file that gets installed by default is quite out of date and requires some tinkering, which I will talk about in detail below.

Written: March 12, 2012
Applies to: Ubuntu 11.10 Server

1. Install Monit
   
   sudo apt-get install monit

   1	sudo apt-get install monit

2. Edit the Monit Config File
   
   sudo nano /etc/monit/monitrc

   1	sudo nano /etc/monit/monitrc

3. Enable the web interface
   set httpd port 2812
# use address localhost # only accept connection from localhost
allow localhost # allow localhost to connect to the server and
allow 192.168.1.0/255.255.255.0 # allow any host on 192.168.1.* subnet
allow admin:pa$$w0rd # require user 'admin' with password 'pa$$w0rd'
4. Optional: Enable Monitoring of General System Resources
check system hostname.domain.com
if loadavg (1min) > 4 then alert
if loadavg (5min) > 2 then alert
if memory usage > 75% then alert
if swap usage > 25% then alert
if cpu usage (user) > 70% then alert
if cpu usage (system) > 30% then alert
if cpu usage (wait) > 20% then alert
5. Optional: Enable MySQL Monitoring
   Monit uses .pid files to monitor running processes. If you plan to monitor MySQL, this file may not exist. To check, do
   
   sudo cat /var/run/mysqld/mysqld.pid

   1	sudo cat /var/run/mysqld/mysqld.pid

   
   If you see a message like this: cat: /var/run/mysqld/mysqld.pid: No such file or directory, modify your mysql my.cnf file as follows:
   
   sudo nano /etc/mysql/my.cnf

   1	sudo nano /etc/mysql/my.cnf

   
   Under the mysqld heading, add the line
   pid-file = /var/run/mysqld/mysqld.pid
   then do
   sudo restart mysql

   1	sudo restart mysql

   Add a new section to the Monit config file for MySQL.
   
   #mysql
check process mysql with pidfile /var/run/mysqld/mysqld.pid
group database
start program = "/etc/init.d/mysql start" with timeout 60 seconds
stop program = "/etc/init.d/mysql stop"
if failed host 127.0.0.1 port 3306 then restart
if 5 restarts within 5 cycles then timeout
6. Optional: Enable NGINX Web Server Monitoring
   #nginx
check process nginx with pidfile /opt/nginx/logs/nginx.pid

7. Optional: Enable Apache 2 Web Server Monitoring
   Find the Apache process section, and modify it as follows, paying special attention to pidfile, start program and stop programcheck process apache2 with pidfile /var/run/apache2.pid
start program = "/etc/init.d/apache2 start" with timeout 60 seconds
stop program = "/etc/init.d/apache2 stop"
if cpu > 60% for 2 cycles then alert
if cpu > 80% for 5 cycles then restart
if totalmem > 200.0 MB for 5 cycles then restart
if children > 250 then restart
if loadavg(5min) greater than 10 for 8 cycles then stop
# if failed host www.tildeslash.com port 80 protocol http
# and request "/somefile.html"
# then restart
# if failed port 443 type tcpssl protocol http
# with timeout 15 seconds
# then restart
if 3 restarts within 5 cycles then timeout
# depends on apache_bin
group server
8. Optional: Set up SSH Monitoring
   
   #ssh
check process sshd with pidfile /var/run/sshd.pid
start program "/etc/init.d/ssh start"
stop program "/etc/init.d/ssh stop"
if failed port 22 protocol ssh then restart
if 5 restarts within 5 cycles then timeout
9. Optional: Set up your mail server settings to get email alerts
   set mailserver smtp.domain.com
username "smtp" password "pa$$w0rd"
10. Comment out the Includes directive
    #   include /etc/monit/conf.d/*
11. Save the File
12. Check your syntax
    Fix any problems found – it’s not too tough to figure out what’s going on.
    
    sudo monit -t

    1	sudo monit -t

    
    When you see Control file syntax OK, or if you see no errors, you’re good to proceed.
13. Start (or restart) Monit
    
    sudo service monit start

    1	sudo service monit start

    or 
    sudo service monit restart

    1	sudo service monit restart

14. Visit the web interface
    http://localhost:2812 if you’re running Ubuntu Desktop, or
    http://hostname.domain.com:2812
    from a host on the configured network
15. Sign in with your admin:pa$$w0rd credentials