Monitoring and support

Our monitoring tool, Do IT Systems DHMS (Do IT HPC Monitoring Solution),
provides indications and information on the status and effectiveness of your HPC system, monitoring specific metrics in real-time and sending warnings or notifications when the “readings” deviate from the set thresholds. The collected data can be stored, analysed and visualised in dashboard and in intelligence and analysis business reports to help you making decisions concerning the management of the HPC infrastructure. 

The software monitors the company’s IT infrastructure, with particular focus on HPC systems also of different nature (PBS, SGE, LSF,…) in order to:

• Promptly notify the operators in case of anomalies (down nodes, temperature alarms, etc) by sending emails and/or SMS
• Undertake automatic corrective actions based on the issues found
• Monitor the use of resources in order to identify “bottle necks” or under-exploited resources  

DHMS exploits Check_MK potentials, installed (if requested by the client) together with WATO (Web Administration Tool).

Check_MK is an open-source extension of Nagios’ monitoring system that allows creating configurations based on rules written in Python and streamline the workload from Nagios Core, improving performances. It is supplied with a set of about 300 system check, a web interface based on JavaScript and a module that allows quick access to Nagios’ core. An automatic acknowledgement of the services and configuration generator replace the regular Nagios’ process.

WATO makes it possible to fully configure a system based on Check_MK via browser. Not only to set hosts and services, but also for example, for users, roles, groups and timeframes.

Supplied on a specific “appliance”, it can be of software type providing a pre-installed virtual machine, or of hardware type, offering a properly dimensioned server.

On the machines to monitor, you can choose whether to install a specific “client” or notify the data to monitor to a main server, or use standard monitoring protocols such as SNMP, MPI, WMI, based on the data to be monitored.

What is Nagios?

Nagios is a program to monitor systems, networks and services. This software consists of a scheduler that, at regular intervals, performs checks on the IT infrastructure and collects results. Based on the status of a certain check, it can notify the administrator about the presence of issues such as a non-responsive host or service by email or SMS, or execute certain corrective actions (reboot a service, reboot a machine, reschedule a job, etc.).

All the checks are entrusted to so called plugins, that interrogate a specific service on the local and/or remote machines. Nagios features a web interface from which it is possible to check the status of all monitoring sessions.

All monitored values are shown on graphs that visualize the history.

The status of a service or host may be of soft or hard type. The soft status is considered “uncertain” or “transitional” and is checked more frequently until a counter reaches a maximum defined level that turns the status into “hard”.

When the status of a service or host changes too often, it is defined as “flapping”. The hosts in “flapping” status do not send notifications in order not to clog the email box. A configurable threshold is present in order to determine if a host is “flapping” and the flapping check can be completely disabled; however, the pre-set defaults are generally sufficient.

In addition to remote monitoring through network services or remotely through “Agent”, monitoring is more effective through management interface. Indeed, the systems described above, have the disadvantage to employ (even if irrelevantly) the resources of the remote machine (cpu, memory, network) because they rely on the host’s operating system. 

Advantages of DHMS solutions

The monitoring system proposed by Do IT Systems combines the stability of Nagios Core to the performances of Check_MK plugin, visualizing the data of the values monitored through PNP4nagios addon. Moreover, Check_MK provides a more friendly and customizable web interface compared to the standard one offered by Nagios. 

The monitoring system proposed by Do IT requires:

• A dedicated server
• To be able to access the management interfaces of the machines to monitor
• Potentially install an “Agent” on some (or all) machines to monitor (according to the values to monitor)

All monitored values can be visualized through graphs in order to display the trend over time; the graphs can be exported in PDF or XML format and it is possible to configure (with WATO) all parameters of the monitoring system through a convenient web interface, without having to access the command line and create users that can access the system with different privileges.

The system features a notification system simple to configure and flexible and it promptly notifies users and administrators in case of problems detected on the monitored systems. Different rules can be set for different users and also multiple rules for the same user and can be viewed from mobile.

By choosing the software solution proposed by Do IT Systems, it is possible to monitor the entire HPC infrastructure with just a few clicks.