These downloads are provided as is with no responsibility for support or damages etc. Where the downloads are copyright of other individuals, I have tried to ensure that this is fully covered. You are however responsible for satisfying yourselves of this before use.

 

System management and reporting on a shoe string !

I am very interested in the management of technical infrastructures, and to this end have put in place a set of tools to provide my organisation with this, but on a very limited budget. I have found that successful management requires:

Activity Tools used
1) Wide ranging event capturing
  • Inbuilt syslog/snmp event capabilities of equipment.
  • Sabrenet NTSyslog open source Windows event logging software, but modified to work better with Kiwi Syslog (with a couple of further enhancements - see below)
  • Novell Groupwise logfile management - see below
2) Pro-active probing

Servers Alive

3) Centralised channelling

Kiwi Syslog and Mdaemon

4) Strong filtering and summarisation
  • Kiwi Syslog rules engine.
  • Mdaemon rules engine.
  • Specialised vb script that hooks into Kiwi that actively summarises messages - see below
5) Variety of alert mechanisms
  • Servers Alive - email/ Net Send/ SMS/ SMPP/ SNPP/ ICQ/ MSN/ SNMP traps/ Notepager/ external programs
  • Kiwi Syslog - email/ external programs/ ICQ/ Notepager
  • Mdaemon - email / external programs


All for much less than many products will give you for just one !

Sabrenet NTSyslog:

I was looking for a tool to pick up Windows eventlogs. I found two free tools; Snare andSabrenet NTsyslog, both now released under a GNU General Public License - I found NTSyslog less complex, and in an environment where it would be installed on hundreds of computers, sometimes simplicity is the best policy. Snare has moved on since my original evaluation and I know of local organisations using Snare.

During use of NTSyslog, I did find a number of issues, and made a number of modifications to the logging engine to:

  • Run less frequently (configurable)and at a lower priority, but be more responsive to Service stop requests etc
  • Produce Syslogs compatible with Kiwi (RFC3164)
  • Fixed a potential hole where it would be possible to miss an event dependent on timing
  • Stopped comms errors (for example a laptop not on the network) from being logged by NTSyslog and potentially filling up the log
  • Provided the ability to send errors directly to the Syslog server without being logged first (needed in the instance where the clock was incorrect on the client PC, which could mean the resending of the complete event log !)
  • Version bale to use the newer Service APIs (but makes it incompatible with NT if used)

I have contacted Jason at Sabernet about the changes. He would like to bring these changes into the Sourceforge version at some point, but this hasn't happened to date, therefore, I have theunofficial branch here. Note I have not updated the GUI, and therefore, to adjust the sleep period between Event Log checks, place this in registry key HKLM\Software\SabreNet\Syslog\SleepPeriod (milliseconds, defaults to 2 minutes).

NTSyslog 1.21 Installation files only

Unofficial NTSyslog 1.21 source

 

Kiwi Syslogger - email management vbscript:

The Kiwi Syslogger is excellent at gathering messages from a range of devices/processes. Syslog messages are the Unix/Linux preferred logging mechanism, however Kiwi also accept SNMP messages to cover equipment that is not Syslog aware. Even though Kiwi has an extensive filter/action rules engine, there are times that an external script will be needed. Kiwi have included the ability to call a range of external scripts in vbScript/JScript, which I used to write a filter that is used continously to manage event storms from one or many devices:

  • Collating multiple alert messages into one e-mail;
  • 'Fuzzy matching' of similar messages so they can be reported together;
  • Storm protection both from a single source and multiple sources, but with no loss of the messages;
  • Basic summary reporting.

The vbscript is reasonably commented, with constants that can be changed to make the code site specific and also more recently allows multiple reporting groups to have their own tailored and managed filtering. I have discussed with Andrew at Kiwi about them adding the ability for Kiwi to also accept email as this would complete the set for me - still hoping this will be done.

e-mail management vbscript and other script examples

Novell Syslog converter:

Having had our Groupwise 5.5 to Groupwise 6.5 upgrade go worryingly wrong, with the loss of one of the msgxx.db files, I needed to keep a better handle on what was actually happening under the cover as the supplied tools e.g. GWMon were just about useless. This code follows the comings and goings of two log files:

  • Sys$log - the main Novell log file - just about follows a standard, but certain messages just ignore this. The code attempts to map sensible Syslog values to any error lines using a combination of known messages and interpretation of the Severity, Locum and Class (not easy as sometimes not provided, and I can't believe some of the Novell programmers read the guidelines for the values !)
  • mmddPOA.nnn - the series of Groupwise Post Office Agent log files that log all activity, and errors seen. This program sifts through these, keeping track of which file it needs to open etc at any particular time - sometimes Groupwise just skips a file ?? As these logs aren't really designed to be skimmed in this way, the code makes certain assumptions to concatenate a run of error messages, avoiding many small syslog messages.

This code is not my best , but seems to serve its purpose !

The code also has the ability to translate the error codes, giving a readable summary and suggested actions from the admin guide - rather than the raw error code. An actual example is listed below. This kept us on top of the issues, which in the end were the result of an older version program that kept corrupting the Novell cache.

(GWMon) <19>Dec 30 00:00:00 NOVELLSYSLOG GWPOA:14:55:10 175 The database facility reported error [820E Cannot lock file] on msg20.db
14:55:10 181 The database facility reported error [820E Cannot lock file] on msg16.db
14:55:10 175 Queueing message for retry because of: [820E Cannot lock file]
14:55:10 181 Queueing message for retry because of: [820E Cannot lock file]
14:55:11 17B The database facility reported error [820E Cannot lock file] on msg20.db
14:55:11 17B Queueing message for retry because of: [820E Cannot lock file]
14:55:11 177 The database facility reported error [820E Cannot lock file] on msg20.db
14:55:11 17F The database facility reported error [820E Cannot lock file] on msg20.db
14:55:11 17F Queueing message for retry because of: [820E Cannot lock file]

Further details of error codes below:

820E Cannot lock file
Source: GroupWise engine; file input/output.
Explanation: File lock error.
Possible Cause: An attempt to lock a file failed.
Action: Check to verify the file is not currently locked by another process that has terminated.
Possible Cause: You cannot open the specified file because another user might have the file open.
Action: Ask the other user to close the file.
Action: Retry the action later.
Action: If no valid user or process has the file open, delete the connection to unlock the file.
Possible Cause: You cannot open the specified file because you might not have rights to open the file.
Action: Obtain rights to the file.
Possible Cause: If this error occurs from the POA when rebuilding a
database, a user might still have the database open. The POA requires exclusive access to databases when rebuilding them.
Action: Make sure no users associated with the database to be rebuilt
are currently running the GroupWise client. If none are and the database is still locked, break the
connection to unlock the file.
Action: Exit, then restart the POA.
Action: Record the conditions under which you encountered the error.
For the latest error code solutions, look up the error code in the Novell Support Knowledgebase (
http://support.novell.com/search/ kb_index.jsp).

Logged to syslog > 2003-12-30 00:00:00

Novell Logs to Syslog ***** need latest version ****requires MSWinsck.ocx. Needs to be run on a Netware client workstation that has access to the Groupwise server's log files.

 

Port manager:

Used to manage access to the network of external suppliers through the routers/VPNs as I couldn't find anything that would remember to switch off access after the agreed time. The port manager code does the following:
  • Enables an ethernet port (on a switch/router), or another action that can be switched off/on by changing an SNMP value (firewall rule) for a fixed duration and then automatically disables it after a requested duration. This overcomes the all too common problem of forgetting to close a port after it shouldn't be required any longer.
  • Shows how to implement a non-polling client server application through IP, with multiple clients allowed (all seeing a instant push update if another client changes a timing setting)
  • Shows how to use VB to both Get and Set SNMP variables ... I don't use SNMP controls - it's pure VB code
  • Another example of Tray management for a VB application with correct stealing of the focus !

The server runs on one machine with full access to the firewalls/switches/routers etc and many clients can then connect to the server.

Having used the Microsoft Winsock .OCX and found it to be very unreliable, I now use the free Catalyst Socketwrench Freeware .ocx control that is very stable and doesn't need to be pre-installed.

I know my team would like me to extend the functionality, especially a calendar function, in the main to manage weekend and early morning access requests, and the ability to call out vbscripts/external programs that can telnet etc to devices that don't accept SNMP

Port Manager Server& Client exe's


Shellit
:

A classic three line program that can be used to shell/spawn off another program in parallel to the current program. I 'wrote' it as part of my attempt to speed up the the company NT login script, but suddenly found in useful in many other areas including:
  • Continuation of a slow non time critical activity at the end of the login script where a software installation or logging activity continued 'after' the login script had 'returned' to the GUI
  • In batch Net Send scripts, which slow down when the particular taget is not online, so this was used to send everything in parallel
  • Removing the any linkage between other programs that shell another program

The Dos START command can do much of this, so check this out first.

My version was written in VB, because it is particularly easy to shell out a new program with the last program's arguments, but a version in C or anything else would be easy as well. For example: Shellit.exe NetSendIt.bat message persoan1 person2 person3

Micro VB Shellit !

Command line login batch scripts:

The login.bat in my organisation consists of many lines of batch script which rebuild the desktop and start menu based on the group membership of the user and the software locally installed on that particular machine. This was taking 28+ seconds, and for remote users up to 5 minutes at times. Having investigated the issue, I implemented a series of changes that:
  1. Invoked an IfMember request only once at the top of the script - massive saving of time for remote sites
  2. Spawned out 3+ parallel batch files, each with a specific set of instructions to carry out
  3. Wrote a simple program that waited until these parallel batch files had all finished
  4. Allowed our auditing and tidyup tasks (temp directory etc) to be carried out after returning control to the 'GUI'

For those in a similar situation, the following code may be of help:

Shellit - see above

ConvertIfMember

  • Takes the output from the standard IfMember and compresses the list for fast later retrieval
  • Not proud of this one as is is a bad use of VB as it should be blocking code to do this, but it has been working at the site for over three years now, and seems to be flawless - hence why I haven't done a proper job yet !

IsMember

  • Takes the compressed file from ConvertIfMember, and can be frequently used throughoout the batch file to make decisions based on the membership, but because there is no network activity during each check, this is significantly faster !

WaitForFlags

  • When seperate batch files are shelled, you must ensure all the critical processing has completed before finishing the initial batch file that Windows waits on before providing the desktop to the user. This low tech program waits for a specified list of files to be created (I used specific flag files in our scripts - one for each shelled batch file), and when they have appeared, exists back to the batch file allowing it to progress. One of our tricks is to continue tidying up and audit activities even after the desktop has been given control back.

Backup Exec reporter :

My team sometimes had difficulty digging through the various Backup Exec jobs each day on the various servers, and especially the tape libraries. This VB.Net program scans the data directories on each server and finds the XML logs, summarising the actual tapes used and any errors.

Backup Exec Checker - source , Backup Exec Checker - Installer