Every now and again I stumble across a tool that does exactly what I am looking for no if's or but's just exactly what I need. It is a rare moment and something to savour.

Today it happened and the tool is Microsoft's Log Parser. This is a tool that knows how to read IIS and Windows event logs out of the box, it allows you to specify filters and columns using a SQL-like syntax and dump the result straight into a SQL server database. How freakin amazing is that! To build this yourself would involve writing scripts that use WMI or .NET to interrogate the logs then produce text files in a BCP friendly format then shelling out to the BCP utility to import to a DB, in other words lots of work. This can now all be achieved in one line of code:

LogParser -i:EVT -o:NAT "SELECT * INTO c:\Output.txt FROM System"

The example above takes all the fields from the Windows System Event Log and places them into a text file called Output.txt. Once you have installed Log Parser you can copy that line of code into a command prompt on any windows machine and if you have the necessary permissions you will dump the entire system event log to a text file. It's quick too I processed ~2600 log entries to a text file in 0.7 secs on my laptop.

I am still investigating all the features, if you specify the SQL output format -o:SQL you have either to setup a DSN to your database or supply the Sql server & database (if using Sql Auth you also need username & password). After the INTO you specifiy the table name, when the command executes if the table does not exist it will be created for you using the columns in the input format. If want to import into an existing table the column names, data types and order of columns must match. Read the excellent help file that comes with the install. See the example below.

Finally there is one more cool feature I would like to mention, if you wish continually upload log entries from a log file as it grows but you wish ignore records you have already processed you are in luck! Use the iCheckPoint parameter and Log Parser will create a file where it stores the position it last read up to in the log file you are processing. This removes the need to clear out the database and reload everything which means a lot less horsepower will be required for processing your logs. You can reduce the intervals between log uploads to make your database more current.

It also has a Com+ api so you can call it directly from C# or VB. I can't believe it took me this long to discover such a useful tool.

SQL Server Example:
LogParser -i:EVT -o:SQL "SELECT * INTO EventLog FROM System" -server:Laptop\SqlExpress -database:Keeper -driver:"SQL Server" -createTable:ON

UPDATE:
If you have IIS set to create Daily log files (the default) then a bit of code is needed in your batch file to determine what the current filename will be. The following example will work on an XP/2003 server that has is local date format set to the UK. If you are deploying to a server with US date format you will need change code that sets the CurrentLogFile variable.

REM Use on servers set to UK date
SET CurrentLogFile=ex%date:~8,2%%date:~3,2%%date:~0,2%.log

REM SET the path to the IIS Log Files Folder. NOTE: If your IIS server has multiple Websites you may have to change the path.
SET LogFilesFolder=%SYSTEMROOT%\system32\Logfiles\W3SVC1\

ECHO Attempting to process IIS Log File: %LogFilesFolder%%CurrentLogFile%

PAUSE
LogParser -i:IISW3C -o:SQL "SELECT * INTO IISLog FROM %LogFilesFolder%%CurrentLogFile%" -server:Laptop\SqlExpress -database:Keeper -driver:"SQL Server" -createTable:ON
PAUSE

Update: Found a Gotcha
I discovered a gotcha with using the flag -createTable:ON when importing IISLogs into SQL server, this will set the column length to 255 max. Often columns like csReferrer, csUriQuery, csUriStem and csUserAgent will exceed 255 chars. To fix this problem simply increase the length of the columns.
I used csCookie: varchar(5000), csUriQueryString: varchar(5000), csUriStem: varchar(1000), csUserAgent: varchar(1000)

I am a big fan of Mark Steel's, I loved the "Mark Steel Lectures" series that he did for the Open University. He is very good at explaining ideas in plain english, de-jargonising them and adding touch of humor to the learning experience. The majority of the series seems to be on you tube, just search for his name and you will find them.

In his latest book he talks mostly about the last ten years of his life, the birth of his two kids, his split with his partner and the end of his membership of the SWP. If your thinking what's so interesting about that well i'm 35 year old and he is 46 so I am just about to go through what he is talking about in his book (hopefully not splitting with my wife and I'm not an SWP member). My wife is eight months pregnant with our first child. The book is honest and amusing and very reasuring that my complete lack of preparedness for becoming a father is the norm.

The book is an entertaining read and well written, I took it on holiday and it only lasted three days a good sign for me. I don't share Mark's views on Capitalism but enjoy reading his arguments, which do have merit because they are well thought out.

4/5 for me.

These are the schema design commandments I live by, they are kept here purely for my reference.
Rules marked with * are absolute and if not implemented will result in karma so bad you will be reincarnated as a bottom feeder.

1.) Pluralise table names.
2.) *Every table in the db will have a primary key, without exception.
3.) Be consistent with Case and separators when naming tables and columns
4.) Enforce referential integrity with foreign keys.
5.) Avoid acronyms wherever possible when naming tables and columns.
6.) Avoid using the float datatype for columns that will store monetary values.
7.) Use varchar instead of char(n) unless specifically required.

If you have any rules you swear by then please comment with your reasoning.
This is a work in progress.

I have a friend I'll call him Aaron because that's his name.

He has joined the company behind a new social networking site focused in the gaming community.

The have just revamped their site, it is still in beta though you wouldn't know it.

The have areas for all the major consoles and of course pc games.

Check Playfire out here.

The interesting thing about playfire from a developers point of view is that the platform is developed on Django a web development framework developed in Python. It is based loosely on the RoR MVC framework and seems to be gaining some traction. I intend to check it out soon.

There is a new search engine on the block, you've got to admire someone who's got the balls to take google on!

Check CUIL out.

My first impressions are that it's fast, it doesn't have a geographical filter like google which might make looking for online stores in your country a problem, unless it handles this behind the scenes.

As you would expect it's index is not as complete as google, I have a three websites and it only found one of them but it may just take some time to get round to the others.

I'm hoping it will be Bot unfriendly, last time I trawled my weblog's live search was referring very open ended searchs to my site, i'm guessing this is bots submitting search words from a dictionary and then working through the list of results, but i'm not sure. One thing I know is that google does not show up in the logs in this way.

One thing I know is competition is healthy if this new site can steal some market share from google it will help to keep google honest.

Robert Scoble has posted on his blog about it.

Entity Definitions
==================
Center - An organisation that runs courses
Center Admin - A course administrator
Center Instructor - A course instructor
Organisation - An organisation that sends people on courses
Organisation Admin - An administrator at an organisation that sends people on courses
Attendee - A person that attends a course
 

Complete First Aid Course Bookings
==================================
- These will be administered by ProHealth to begin with
- Add certification duration to Course Types
- Add course attendee certificate issue date / expiry fields - This should be the same date that is on the certificate and will be used to trigger reminders to be sent out.

Nice to have's
==============
- Course Search Form - Search by category / date / type / status
- Upcoming Course list - set max in list / period i.e 3 months / both
- RSS / Email Notifications for upcoming courses
Enquiry form
- filled out by PH admin when enquiry is received by phone / email / fax
- Should allow searching for repeat bookings by existing customers by post code / company name
- Will track enquiries as they become bookings
- Sends an email to prospective customer containg a link to book on line / pdf to faxback
- PDF contains enquiry ref will be keyed by ph admin when fax is recieved will save time.
  
Expiry Tracking System
======================
Because a company would be nuts if it didn't chase repeat business
- Inteligent tracking of certificate expiry dates
- Issue reminder letters, calls, or emails to org's, companies and individuals.
-- Currently this is done once a year
-- Organised companies book an entire year in advance
-- Disorganised companies book at the last minute.
-- The system should try and make life easier for both types of company
- Optout/in facility will allow reminders to be turned off

Company Portal
===============
Will follow when all in house admin functionality is complete
- Customer will be issued a login and password

I wanted a way to get into the guts of monorail so I offered to create a website for the occupational health company that my mum works for. It is very much a work in progress, you can check it out here ProHealthUk if you want to see the CRUD click the admin link. When the site goes fully live you will have to login but for the time being you can edit data.

So tonight at about 8pm I sit down on the couch with a beer when I hear a knock at the door. When I answer it there two ladies and a gent all smartly dressed in suits. The woman asks me to confirm my name and address as shown on her clipboard she then starts a vauge diatribe about how I could save money on my energy bills. At no point did she identify that she was from nPower. I learnt this from looking at the logo on her Id badge.

The lady's sales pitch continued for about a minute until I twigged what was going on. I asked her straight out "Are you offering to change my provider?" she seemed disinclined to answer the question so I asked it again. She said yes to which I replied I am not interested in changing my provider. She asked if it was due to the hassle, I said no I just do not want to change. That was the end of it.

What annoyed me was the underhand way in which the salespeople misleadingly explained their presence on my doorstep. If they had identified themselves as N-Power and offered to quote me a price for supplying fuel I would have given them a chance. I like the next person would always rather pay less for my fuel but I felt like they were trying to con me into changing without even realising I was doing so. I am now very unlikely ever to contemplate using n-power in future. They should stop this practice as they will get some very negative pr from people like me. IMHO door to door sales is about as low as it gets anyway.

Rant over. 

I am building a course booking system in monorail and in the process of uploading the source to google code. This is basically a way for me to understand the MVC approach to building websites using the castle monorail project. I figured the code may be of use to others tackling similar tasks or who just want to hack around with existing code to see how it works. I am not by any means a monorail expert so some of the approaches I have taken may be wrong or ill advised but that is kind of the point learning by experience is my favourite approach. Let me know if you spot any. All help gratefully received.

This will be a system designed to help administrators organise training courses. The initial need is for it to be able to handle first aid courses run by a health clinic.

The system will be used by non-technical admin staff who are familiar with email and basic word processing on windows pc's.

The development of the project uses the .net framework 2.0 and an MVC pattern with the Monorail Castle framework. It makes use of the Active Record ORM to handle data access. SQL Server is used for the database. I have elected to use nVelocity for the view templates. I have also elected not to write tests and use tdd. I know I should it's just that my brain cannot absorb that many new things at once.

The system allows new courses to be setup, new course types to be configured. Courses can be managed with new attendees booked on to courses.

If the courses are for certification purposes if an attendee passes the course the certificate issued can be tracked so renewal notification can be sent out.

The system allows attendees to be moved from one course to another and for their booking to be cancelled.

The system will produce reports for invoicing, courses and reminders for expired certificates.

You can find the project here:

http://code.google.com/p/centro/