Server Monitor IVR for Admins

LinuxScrew, Artem Nosulchik’s Linux and Open Source blog, had a nice review of Linux server web-based monitoring tools earlier this year.  There is some nice stuff out there designed to help the overtasked sysadmin stay on top of a server herd.  These tools monitor CPU load, memory usage, disk space, etc.  Here’s how to get the server to tell you this stuff with a phone call using simple IVR scripting in conjunction with very basic UNIX shell scripting.  I’m using FacetPhone IVR scripting for this, but it should be doable in any IVR system that can run simple server commands.

First, let’s define what information we want to get and figure out how to query the server for the information we want to access via the PBX… CPU utilization and disk space are good things to monitor.  This sort of IVR system could be expanded upon to provide just about any info that can be gleaned from a server command including real-world sensing using monitoring devices like those of EnviroTrack.  In our example here, let’s use the ”iostat” command to check on CPU utilization.  ”iostat -c” provides a CPU report that is easy to parse:

"iostat -c":
Linux 2.6.20-1.2962.fc6 (facetphone2.sssi.com) 	10/07/2012

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.24    0.48    0.77    0.12    0.00   98.39

All we care about here is the “%idle” figure. Snatching the number is easy:

iostat -c | grep . | tail -1 | tr -s ' ' \
      | cut -d ' ' -f7 | cut -d '.' -f1
98

Call it 98% to keep it simple… So we know how to get our number… CPU is 98% idle.  Do this from inside an IVR script, tell the caller the results, and you’ve got a nice Server Monitor IVR which can easily be enhanced to report on all sorts of things.

FacetPhone’s IVR scripting language has a powerful “popen” type of functionality perfect for this sort of thing.  There are three related functions we’ll use:  ”Popen”, “Pread”, and “Pclose”.  The “Popen” function opens a command pipe and runs the command, the “Pread” function reads the results of running that command, and the “Pclose” function closes the command pipe.  Don’t get hung up on complicated technicalities… It is easy to do and here is how:

cmd  = "iostat -c | grep . | tail -1 | tr -s ' ' ";
cmd += "| cut -d ' ' -f7 | cut -d ' ' -f1";
cpuidle = "";
fd = Popen( cmd );
Pread( cpuidle, fd );
Pclose( fd );

Build the command string, initialize a variable to hold results, open the command pipe, read the command pipe, close the command pipe. The end result is that our “cpuidle” variable will contain our “98″ %idle figure.  Not so bad, right?  The rest of this is just a matter of having a menu option that our dear sysadmin can get to which gets the “cpuidle” value and then tells him what it is.  FacetPhone has a pretty good collection of default system prompts — it can handle saying numbers okay.  Specialized stuff like we need here can either be done with TTS (Text to Speech) or by pre-recording any prompts you’ll need.  I’m sure we’ll be talking about TTS later, so for now, let’s assume we’ve already got pre-recorded helper prompts (“cpu_is” and “percent_idle”) we’ll use as we present our “98″ %idle number to the sysadmin.

Here is an IVR script snippet for doing this which assumes taking menu option 3 to find out about CPU utilization:

if ( ret == E_TT_3 ) {

    cmd  = "iostat -c | grep . | tail -1 | tr -s ' ' ";
    cmd += "| cut -d ' ' -f7 | cut -d ' ' -f1";
    cpuidle = "";
    fd = Popen( cmd );
    Pread( cpuidle, fd );
    Pclose( fd );

    Play_list_of_prompts( "cpu_is;" + cpuidle + ";percent_idle" );
}

Nada más! That’s it! The very same sort of thing could be done for reporting disk usage or job status or flight arrival time or order status, etc.  This is a very powerful technique…  Please share cool things you are doing like this so we can all be inspired!

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>