Here is why flexibility is an imperative - the biggest factors in log management and compliance are:

Event Log Collection and Retention

One of the keys to effective event data collection is to first have the data reported in the event logs. Sometimes, this requires a bit of tweaking by network administrators to gain access to desired log data. As an example, for the security log in Windows environments to obtain all available activity data, success and failure auditing in certain categories must first be enabled in domain group policy, and also enabled on file, folder, and registry entries that will be monitored for access attempts.

The log retention period obviously depends on individual requirements. If you are building out the infrastructure for troubleshooting and short-term reporting, you may need to keep only one or two months of logs. But if your goal is compliance with the Sarbanes-Oxley Act you will probably need to keep years' worth for the auditors, depending on your corporate retention policy. More and more, seven years is becoming a generally accepted timeframe for log data retention.

Log volume is probably one of the most critical factors in building your infrastructure. It has a direct impact on your retention policy as well as performance in log reporting, search, aggregation, and correlation. A network or network segment’s log volume

Most security and compliance professionals agree that event logs should be stored in EVT format - preserved in long-term archives as forensic data. However, the log data does not contain the name of specific users or systems but instead stores the security identifier (SID) in Microsoft logs. This requires reports created from EVT files to obtain the SID user and system data from the domain controller to preserve the data at it existed that moment in time. If the user or system account has been removed from the domain, this data no longer exists for reports created from EVT files.

Database storage allows for the SID information to be interpreted and stored along with the rest of the event data eliminating the loss of this data if the user or system account is removed from the domain.

Relational databases like Microsoft Access, SQL, and Oracle offer additional features like complex queries to extract exact data based on matches across a number of data fields.

EVT Storage

Event log data saved in EVT files provides an archive of forensic data as it existed on the system and should be stored long-term. The option of storing the EVT file to a UNC share or to FTP the file to a remote FTP site provides flexibility for LAN/WAN networks. Finally, compression of files reduces the long-term storage volume needed to handle them.

Database Storage

To reduce overall database size, you should have the ability to select the event data you want to keep in the active database for routine compliance reporting. One way to reduce the amount of event data is to filter the event collection to the database by importing or excluding specific events.

Along with filtering, you can keep only a limited amount of data online for analysis and retain the rest. Relational databases allow users to routinely export older data to alternative or offline storage to keep the active database manageable.

The ability to either clear active log data or not during archiving - without creating duplicate entries in the database when new data is processed - adds additional flexibility to meet varying needs of users.

As event log data can be voluminous, look for a solution that can import log data into your database on a scheduled basis, preferably during off-peak times. This will minimize the impact log collection can have on network load.

Reporting

Take one look at the Sarbanes-Oxley Act - its specific requirements are extremely vague. Upon basic study of the act though, it is safely assumed by IT security professionals that general security management process must exist in order to track and thereby protect against attempted or successful unauthorized access, use, disclosure, modification, or interference with systems and applications providing key business processes.

Here are some typical reports IT professionals are looking for to help comply with SOX regulations:

· User Logons/Logoffs

· Failed Login Attempts (system and application)

· Access (authorized or unauthorized) to Audit Logs

· Account Management Changes

· Audit Policy Changes

· Administrator (e.g. superuser) activity

A pre-configured product or “SOX in a Box” offering might provide basic and very commonly used reports. However, the reporting of such data isn’t a “one size fits all” procedure. What about reporting of any other trends in event log data? What if regulations change or your auditors become more particular about the reports that should be run?

The ability to use relational database filtering to select data for less common and custom reporting provides total flexibility in reporting. The result is stronger, more reliable security and compliance strategies.

Beyond Sarbanes-Oxley

Quite often, companies must comply with multiple regulatory acts including Sarbanes-Oxley, HIPAA, or Gramm-Leach-Bliley (GLBA). Many companies find that once they implement a flexible and scalable event log management solution under one such act, the same procedure is applicable under another. Monitoring and reporting of application log data is very popular for primary applications like mail, database, custom resource software, and health care programs.

Chances are, if you are a large publicly traded health care company, health insurance company, or hospital system for example, you face multiple compliance efforts. Whether or not you personally will help manage all compliance efforts, you will want to team up with others in your organization working with compliance to look for overlap in log management practices.