Here is a first list of downloadable web monitoring packages, which I define as anything that you have to set up on your own machine, as opposed to Hosted Services, which live on the web and require only a browser (and usually, some money, too) to operate.
Laughing Man is an application that detects if a Web page has changed. Uses web server's last-modified date if available, otherwise, generates a checksum based on the contents of the page.
http://sourceforge.net/projects/jopeneyes
Jawa Open Eyes is a project to build network monitoring tool by SNMP with the power of Java. The features support data collection of MIB-II, graphical view, multi layer network topology, email alert, trap receiver, JDBC, logging, http and port checking.
http://freshmeat.net/projects/gospy-applet/
gospy-applet is a GNOME 2 applet monitoring Web pages and servers. It detects changes in page content, page status, page size, page loading time, IP address, and so on.
* Nice configurability choices for monitoring.
http://freshmeat.net/redir/visualpulse/11183/url_homepage/index.html
VisualPulse is a Web-based monitoring tool designed for easy reporting of Internet service availability. It enables monitoring of specific websites, servers, and network nodes anywhere over the Internet, corporate Intranet, or Extranet, visually providing real-time and historical reports for ping, HTTP, and TCP statistics. It provides automated e-mail notification of problems.
* This is a proprietary solution with free trial. Nice looking screenshots, tho.
http://webperf.org/
WebPerf is a system for measuring response time of specified URLs from multiple locations on the internet. The project is founded on the premise that there are lot of other companies who also require such a monitoring service. If the other companies are willing to monitor our URLs, we will montior theirs (a free co-peering arrangement).
* Volunteer effort, information cannot be charged for.
http://www.jhealthcheck.com/
JHealthCheck? is a tool for monitoring the availability of databases, Web servers and custom applications from a Web interface. When it notices a problem, it can log the error, send an email, or call a custom logger.
* Crippleware trial, per-CPU license fee.
http://www.horsburgh.com/h_npulse.html
nPULSE is a web-based network monitoring package for Unix-like operating systems. It can quickly monitor tens, hundreds, even thousands of sites/devices at a time on multiple ports. nPULSE is written in Perl and comes with its own (SSL optional) web server for extra security.
* Freeware, perl-based with own http server for 'security', awful looking interface with potentially great information.
http://argus.tcp4me.com/
Argus is a system and network monitoring application. It will monitor nearly anything you ask it to monitor (TCP + UDP applications, IP connectivity, SNMP OIDS, Programs, Databases, etc). It presents a nice clean, easy to view web interface.
* Open source freeware. Web interface.
http://spong.sourceforge.net/
Spong is a simple systems and network monitoring package. It does not compete with Tivoli, OpenView?, UniCenter?, or any other commercial packages. It is not SNMP based, it communcates via simple TCP based messages. It is written in Perl. It can currently run on every major Unix and Unix-like operating systems.
* Not changed in a long time, still looks interesting.
http://freshmeat.net/projects/sitecheck/
Sitecheck is a Perl-based Web site monitoring tool. It monitors a list of Web sites for availability, and checks for regex-based errors. It will also send emails to a list of recipients upon failure or error.
Features of an Alert System
Alerts, or notifications, are the attempts of a monitoring program to notify someone that something anomalous is happening to the monitored system. There is no point in having an automated test of any page or system if there is no way to remedy problems that occur. For the most part, our systems' best response is to try to bring the potential problem to the attention of a human being.
Two features are crucial to making a notification system effective. The first, obviously, is to be able to get through to someone who can address and fix the problem, as soon after it is detected as possible. The second is to make some judgment about how serious the problem is, so that the humans who are being notified are not flooded with alerts that do NOT represent problems, a situation in which real notices will be ignored.
Email is the most common way of attempting to fulfill the first requirement. Most if not all modern web monitoring systems allow the webmaster to select particular email addresses who should be sent an automated message when anomalies of a particular type or severity are detected by the monitoring system. Obviously, this relies on the fact that the email service is working for that user, which can be a bit tricky if email service is being handled by the same machine(s) that are currently exhibiting problems. It is frustrating to wade through the smoking wreckage of a web server or database, finally restore it to some working function, only THEN to finally receive a bunch of messages that would have kept the wreckage from smoking so furiously in the first place.
At a minimum, the system that handles email for anyone who is in charge of dealing with a notification should be hosted on a different system from whatever web services are being monitored. A better solution would be to have problems escalate through several different types of messaging solutions, in order to maximize the chance of the alert getting through. The more sophisticated (also more expensive) web monitoring solutions offer several means beyond email to get in touch with responsible parties ... Instant Messenger, ICQ, Skype, SMS text messaging, pager or a call on the traditional telephone network are all options that I've seen advertised by these services.
The second requirement, that of intelligent filtering of notifications, is generally more complicated, and often requires some trial and error by the person responsible for configuring web monitoring before it works correctly. Obviously, the first step is to differentiate between alerts of different severity levels, and send each different one to someone who is qualified to handle the problem. A useful next step is to be able to differentiate messages based on the particular sub-system being monitored, so as to be able to tell a database administrator when the database has a problem, versus telling a webmaster when the web server is down, or the networking people when the machine is completely unreachable.
In the perfect world, the system monitor could actually fix the problem itself, and send you a happy 'dealt with it' message. Since this is unlikely to occur in the immediate future, we have to assume that various responsible humans will have to receive these messages. So an important feature of any web monitoring alert system is to do as much as possible to only intrude upon the attention of the user when something truly urgent is happening, to avoid 'message fatigue', in which the user gets so many messages from the monitoring system that they just automatically delete them, or assume that they are more of the same ... what a waste to have a truly important event missed because it is buried in a list of hundreds of 'system is ok' messages!
A simple way to keep from fatiguing the attention span (and flooding the email box) of the user is simply to remember some history about the set of alerts, and group them as a single event, like 'the web page changed significantly each of the last 20 times I looked at it', instead of sending 20 messages that say 'the web page changed'. Fewer messages mean that the ones the user does see are apt to receive more attention.
There is potentially even more value in having the alerted parties give feedback to the web monitoring system, allowing it to 'learn' when and how to alert the user, and in what circumstances. A mechanism for accepting feedback and learning from it can be as simple as offering the notified party a way to respond to the message, with the option of saying 'yes this is important', 'please do not bug me with this, it is normal', or 'do not bug me about this unless it occurs a lot'. As in any system that relies on explicit user feedback to configure itself, however, it requires a great deal of buy-in from all of the humans involved. After all, if nobody ever responds, the thing can't learn, and the whole exercise is an expensive and useless failure. And it requires patience on their part too, as it takes time and a certain amount of work by all parties involved before the system is fully responding to their preferences.
December 22, 2004 in Commentary | Permalink | Comments (0)