Conversion and simplification of Combined NCSA/CERN format website logs
Copyright Albedo Systems Ltd 1998
http://www.albedo.co.uk/
24 March 1998
CONTENTS SUMMARY
CFX_COMBCONV is an extension tag to Allaire's Cold Fusion active web page language, written in C++, and running under Windows NT/95. It allows the conversion of conventional web site logs created in the Combined NCSA/CERN format into page-only logs in a simpler format, retaining all useful information, but rejecting all non-HTML elements and failed pages (for example, 404 Not Founds). It also allows you to exclude sub-domains of your site - which may actually be separate websites in their own right.
This program was written to support our commercial website log analysis program, LogSite, which we (naturally) commend strongly - it offers many unique features. See our website for details. However, you may well find CFX_COMBCONV useful in some other way - in which case, bon appetit.
Note: a stand-alone DOS version of this program (.exe) is available at http://www.albedo.co.uk/goodies/logsite/doslsite.cfm
CONDITIONS
CFX_COMBCONV is freeware. As such, it may be freely distributed in the form of this zip file, provided this readme file is retained. All technical queries, bug reports etc. with regard to its use should be addressed to admin@albedo.co.uk.
We'd, of course, appreciate it if you credited us/linked to us, but you know you don't have to. We'll be doing more stuff soon, so please do drop by our website at http://www.albedo.co.uk/
For more information on Cold Fusion itself and other custom tags, check out Allaire's site at http://www.allaire.com/.
WHAT IT DOES
The input file for this tag must be in the Combined NCSA/CERN format. A typical line of such a file would look like this:
127.0.0.1 255.255.255.255 - [24/Mar/1998:18:44:37 +0000] "GET /stuff/index.cfm HTTP/1.0" 200 8300 "http://someone_else/links.htm" "Mozilla/4.04"
Not all of this information is frightfully relevant to a site user/owner, though a webmaster may find it useful. The 200, for example, simply means that this was a successful request. CFX_COMBCONV will convert this information into a simpler format:
"24-Mar-98", "18:44 ", "255.255.255.255", "Mozilla/4.04", "d:\mysite\htdocs\stuff\index.cfm", "", "http://someone_else/links.htm"
...which breaks down into:
Date, Time, IP Address of visitor, Browser (or other agent) used, Page Accessed, Page Description (not available, but included for compatibility), Referring page
Additionally, non-page lines (for example gifs) are ignored - only file extensions .htm* (includes .htm, .html .html-ssi etc) and .cfm are allowed through. All failed page requests are ignored, too. Optional parameters can be set, so that all files in a particular sub-domain (i.e. a directory) are ignored too.
The next version, when we get around to writing it, will allow the user to select permissible extensions, and add page description fields.
INSTALLATION
The zip file that contains this file should contain:
- combdoc.cfm: this file.
- cfx_combconv.dll: the Cold Fusion extension dll
- Albedo6.gif, back.gif, rule_pnk.gif, dot_clr.gif: Formatting graphics for this readme file.
Click on the 'cfx tags' tab or button, then click on Add. Enter the required info, and make sure the library is not persistent in memory.
USAGE
Here's an example, with all the attributes on separate lines to make it clear... (Bear in mind that not all of these parameters are mandatory).
<CFX_COMBCONV
LOG_IN="c:\webs\mysite\logs\mysite.log"
LOG_OUT="c:\webs\mysite\logs\new.log"
SUB_DOMAIN1="sites/"
SUB_DOMAIN2="homepages/"
PATH="c:\webs\mysite\htdocs\"
>
Most of these are easy to understand, (PATH is there to preserve compatibility with CFX_LOGSITE). They are:
LOG_IN (mandatory) path specification for the log file that you wish to convert.
LOG_IN (mandatory) path specification for the output log file.
SUB_DOMAIN1 and SUB_DOMAIN2 (optional, default to NULL) search strings that allow you to exclude parts of your site from the log. Use with caution - for example, "sites/" would not merely exclude everything in http://www.mysite.co.uk/sites/ but also: http://www.mysite.co.uk/other_stuff/sites/. Just make sure they're unique.
PATH (optional, defaults to NULL). The function of this parameter is simply to provide output compatible with our CFX_LOGSITE tag, which itself was designed to work with Cold Fusion user logs. Basically, it helps to convert the server page template name to a local path, hence "stuff/index.cfm" could become "d:\mysite\htdocs\stuff\index.cfm".
OTHER ALBEDO SOFTWARE
If you liked this, check out our other software:
CFX_LOGSITE, which this tag was designed to work with, is a comprehensive site logging program. It is commercial, but inexpensive, with some quite unique features. Take a look at its output, or download a demo from http://www.albedo.co.uk/goodies/logsite/cflsite.cfm
You may also like our CFX_FONT tag, downloadable from http://www.albedo.co.uk/goodies/cfgfx.cfm. It allows you to create masses of anti-aliased text graphics and has oodles of options.
Oh, and of course, don't forget to pick up our other log conversion programs. CFX_COMMCONV converts the NCSA/CERN common format, and there are stand-alone DOS versions (.exe) of both programs at http://www.albedo.co.uk/goodies/logsite/doslsite.cfm
Fin Fahey
Fiona Daly
LEGAL DISCLAIMER
Neither Albedo Systems Ltd. nor anyone else who has been involved in the creation, production or delivery of this product shall be liable for any direct, indirect, consequential or incidental damages (including damages for loss of business profits, business interruption, loss of business information, and the like) arising out of the use or inability to use this product even if Albedo Systems Ltd. has been advised of the possibility of such damages.