# System monitoring



## trumee (Jul 8, 2016)

Hello
I have a FreeBSD server running a bunch of ezjails. I would like to monitor the cpu and ram usage of the server including jail processes. I have intentionally kept the host clean and all the services run in the jails.
Where should I host the montoring server; on the host or another jail?
Installing it on the host will bring in dependencies like a webserver and PHP which I was hoping to avoid.

Any recommendations?

Thanks


----------



## wblock@ (Jul 9, 2016)

net-mgmt/icinga2 does not use PHP, as long as you don't install their fancy web client.  It's a kind of reimplementation of Nagios.

As far as where to run it, I consider it just another service and run it in a jail.


----------



## trumee (Jul 9, 2016)

I went with zabbix-server/frontend running in the jail and zabbix-agent on the host. I also needed to monitor my pfsense router which had a zabbix agent.


----------



## fred974 (Dec 7, 2016)

trumee did you had to do anything to the jail to have zabbix running?
wblock@ I saw you are a great advocate of net-mgmt/icinga2 on the forum, could you please tell me why?
How is  net-mgmt/icinga2 better then zabbix?

I am deciding which monitoring tool to use and I have never used any before so whatever i choose , I'll have to learn from basic
Could you please help me decide? I am running ZFS and have around 18 jails to monitor


----------



## wblock@ (Dec 7, 2016)

fred974 said:


> wblock@ I saw you are a great advocate of net-mgmt/icinga2 on the forum, could you please tell me why?


I wouldn't say I'm a great advocate, but I use it for the reasons mentioned in post #2: no PHP, and scripts and modules meant for Nagios can usually be used with it.


----------



## fred974 (Dec 7, 2016)

how do you get to the user interface without php?


----------



## Phishfry (Dec 7, 2016)

I am going to guess- no web interface is what he means. Simple command prompt programs showing status.

Icinga2 is popular because it uses Nagios plugins. Nagios went corporate Icinga split, Nagios took plugins site from community ect.ect...


----------



## Oko (Dec 7, 2016)

fred974 said:


> how do you get to the user interface without php?


There is a nagios server with ncurses interface but obviously most people use nagios with web interface.  Icinga2 is complete rewrite of icinga which was forked from nagios. I doubted Icinga2 has ncurses interface but the idea is that you run web server and web interface on your designated monitoring server/your desktop not on the servers you are trying to monitor.


----------



## wblock@ (Dec 7, 2016)

fred974 said:


> how do you get to the user interface without php?


Contrary to popular belief, HTML can be generated from any programming language.  Icinga2 is written in C++ (mostly).  This is what the non-PHP site looks like: https://www.icinga.com/products/screenshots/icinga-classic/.

Now, they went and added a "web client" type thing, which does use PHP.  I'm very disappointed in that, but not enough to switch to something else.  The classic web interface still works fine, and as long as I don't have to install PHP, I'll keep using it.

Careful readers might detect a theme here.  I really, strongly go out of my way to avoid PHP.  It seems to be the tool of choice for people who make accidental whoopsies that compromise websites.


----------



## lme@ (Dec 8, 2016)

You can use net-mgmt/icinga2 together with net-mgmt/icinga-classicweb (no PHP, plain C CGIs).


----------



## SirDice (Dec 8, 2016)

I like M/Monit for its simplicity. One of my clients has it running. You'll need a license for the server but it's a one-time payment and you get a lifetime of updates. I've had a few minor issues in the past and their support was reasonably fast. Issues were handled quickly, patches followed not long after that. Definitely worth the money.

The server part is pretty much stand-alone, doesn't require PHP or even a web server. Executables are built for FreeBSD 9 but it runs fine on 10 if you install misc/compat9x. Backend can be PostgreSQL or MySQL. The client part is in the ports tree: sysutils/monit and is updated quite frequently.


----------



## fred974 (Dec 8, 2016)

Hi

Is net-mgmt/icinga2 a push or pull model?
I have 1 physical server with many jails all behind PF.
My web web reverse proxy jail, mail server jail and ftp jail have their own plublic ip, so pull or push model is no problem here..
But  from what I gather I need a push client on the jails that has internal IPs 10.8.20.0/8 but I also understand that if I go for Zabbix for example, pf could co some zabix redirection.. (not sure how)

I looked at the license for M/Monit but I am not sure which license apply to me.. are they interpreting a jail as a host or physical box?
I have 18 jails and 1 box.. would I need a 20hosts license?

At the moment I am thinking of using something that I can easyli get help with as it is my first time in the monitoring world and have a lot to learn...So was looking at Zabbix and cinga2 ..maybe M/Monit

trumee, sorry if I 'hijack' your tread.. I'll start my own tread if I need for answer


----------



## Oko (Dec 8, 2016)

SirDice said:


> I like M/Monit for its simplicity. One of my clients has it running. You'll need a license for the server but it's a one-time payment and you get a lifetime of updates. I've had a few minor issues in the past and their support was reasonably fast. Issues were handled quickly, patches followed not long after that. Definitely worth the money.
> 
> The server part is pretty much stand-alone, doesn't require PHP or even a web server. Executables are built for FreeBSD 9 but it runs fine on 10 if you install misc/compat9x. Backend can be PostgreSQL or MySQL. The client part is in the ports tree: sysutils/monit and is updated quite frequently.


+1

I run it in production.  I used to run M/Monit server on OpenBSD but now it runs in FreeBSD jail due to infrastructure consolidation.  You can get evaluation version which will run 3 months for free by directly downloading things no questions asked.  To make things easy just use SQLite. No server configuration is necessary. The truth of the matter is that Monit client sends its output in plain XML which could be captured and displayed.  There were several attempts to create free server version (client is open source and runs on everything I ever tried to run with exception of the DragonFly BSD on which is broken).  You can run as many clients as you want. You have to pay for server unless you want to reinstall every three months. Check my posts on StackOverflow for the whole story. The free server versions are not usable (one is Ruby and another one I forgot).

Personally I prescribe to the opinion that one should separate monitoring physical servers and virtuaal (Jail instances). For two servers you can just install Monit client on both of them and monitor using two different tabs in your browser because client does come with the web interface. The problem is when you want to monitor 100 hosts like me and aggregate results.

Beat the pants out icinga in terms of simplicity but icinga scales better if you have more than 100 servers.


----------



## wblock@ (Dec 8, 2016)

fred974 said:


> Is net-mgmt/icinga2 a push or pull model?


Depends on what is being checked and also your perspective, but I would say it's mostly pull, as in the server collects data from the monitored systems at intervals.  For basic connectivity like responding to pings, it polls the monitored systems.  If you install a package on the monitored systems, they can report things like disk space used, CPU busy, that kind of thing.  This kind of information is collected from the host running the Icinga2 system without any add-ons, and it will do that also for jails running on that system.  Of course, a lot of that is the same for a jail as for the host, like CPU busy.

My current usage is mostly to detect network outages.  I get email alerts when one of the ISP's nameservers goes down, or when the ISP loses connectivity to a large part of the web (as happens all too frequently), or if these forums cannot be reached.  I wrote a small Perl add-on to verify that the online FreeBSD documentation has been rebuilt within the last week.  Icinga2 does this fine, although their configuration examples are very sparse.  The ideal thing would be a non-trivial real installation that can be used for examples.


----------



## lme@ (Dec 8, 2016)

Nagios / Icinga 1/2 support all kinds of checks. Very simplified it's just a scheduler which executes plugins at a given time and evaluates the plugin's return value and stdout.

Some use case examples:

check_icmp pings the target host, if the ping succeeds, the host is treated "up"
check_http does a http(s) request on a host's web server
check_snmp queries a remote machine via snmp
check_by_ssh executes plugins on remote machines. You login via ssh, execute e.g. check_load and get back the remote system's load
If you don't want to use SSH to execute remote commands you can use check_nrpe for Nagios / Icinga 1.

With Icinga 2 you can install an instance of Icinga 2 on the target host and easily configure it so that it communicates securely with the monitoring host.
Checks are scheduled and executed by the Icinga 2 instance on the remote host and sent back to the monitoring host.

See https://docs.icinga.com/icinga2/latest/doc/module/icinga2/chapter/distributed-monitoring for details on that.

https://www.monitoring-plugins.org/doc/man/index.html has the manpages of the official Monitoring Plugins for Icinga / Nagios.

Writing a plugin is a piece of cake: You can use any language, just provide an executable that returns 0 for "Ok", 1 for "Warning", 2 for "Criticial", 3 for "Unknown" and some status Text on stdout.

This is a valid plugin:


```
#!/bin/sh

file="/etc/master.passwd"
output="${file} is"
if test -r "${file}"; then
  output="${output} readable"
  rc=0
else
  output="${output} not readable"
  rc=2
fi

echo ${output}
exit ${rc}
```


----------



## Oko (Dec 8, 2016)

wblock@ said:


> Depends on what is being checked and also your perspective,


 No it doesn't. It is polling model. Icinga server sends the signal to nrpe client which executes a script (plugin) on the remote host and send the result to the Icinga server. Rraditionally those plugins were Perl scripts. IIRC Icinga unlike remote telemetry tool Obsevium mentioned in several other monitoring posts has very good proxy so polling devices behind the corporate firewalls is not a problem. As I mentioned earlier it scales far better than M/Monit and should be the first choice for any organization with more than 100 devices.


For two computers he is beating a dead horse. He can literally have tmux on his desktop and run simultaneously top on both machines. That is the point of functional monitoring. We want to see if the things are up or down and current load. Anything more than that is called remote telemetry and requires another solution.


----------



## wblock@ (Dec 9, 2016)

Oko said:


> No it doesn't.


Actually, it can be several different ways.  For hosts that might or might not be up or accessible at any time (remote Windows systems, in my case), there is a way to configure it so those systems report their status rather than being polled.

Very few modern setups are really just one or two computers.  A monitoring system can check for access to the host ISP's DNS servers, mail server, the webserver in the cable modem, the router, a UPS, and access to any of a couple of dozen online sites that might be important to a particular user.  Once you have the basics of a monitoring system set up, adding monitoring of other things is pretty easy.  Since it's automated, there is not much work after it is set up, and it only pipes up when there is something you should know.


----------

