Category: Event Log

Auditing Changes to Microsoft SQL Server Database Tables

tames.mctigue 0 Comment Event Log, EventSentry, Tips & Tricks

Database servers store massive amounts of data, often including sensitive information. It is not uncommon for there to be databases holding millions of rows of data, where a small subset of rows are considered critical or sensitive. This could be anything from a Social Security number to an EventSentry entry of a security event. Being notified when existing data in your database changes is crucial for log data, and can be accomplished by using triggers with Microsoft SQL Server.

For those of you not familiar with triggers, a database trigger executes code in response to events on a table or database. Triggers are essentially hooks into a table, and they usually execute SQL statements as a response to another SQL statement.

Since we love the windows event log, we’ll take advantage of SQL Server’s ability for triggers to log an event to the event log when a row in a table is modified. This allows us to not only log that activity, but also get notified immediately when suspicious or important activity occurs in the EventSentry database.

In EventSentry, we have a table named ESEventlogMain that stores Windows event information. This table constantly gets new data inserted into it, and it often gets purged as well to manage the size of the database. However, there is no reason this data should ever be modified. If it is, then we know that something is amiss and we want to trigger an event in the event log. It is also useful to know what account made that change.

The first step is to create the message in SQL. You can use this SQL statement to create it:

sp_addmessage 80000, 10, ‘Data Integrity Alert: %s’, @with_log = TRUE

The first argument is a unique SQL server message ID that should be 50001 or higher, you can delete it again using sp_dropmessage. The number 10 is the severity level, but you can read more about the different options for sp_addmessage here.

Now we create the trigger that will use this message:


CREATE TRIGGER Trigger_ESEventlogMain_Modified ON
ESEventlogMain
FOR UPDATE
AS

IF UPDATE(eventmessage) OR UPDATE(eventid) OR UPDATE(eventtime) OR UPDATE(eventcomputer)
BEGIN

     DECLARE @Msg VARCHAR(8000)
     DECLARE @EventNumber INT
     DECLARE @EventID INT
     DECLARE @Computer VARCHAR(255)
     DECLARE @EventMessageOld VARCHAR(8000)
     DECLARE @EventMessageNew VARCHAR(8000)

     SET @EventNumber = (SELECT eventnumber from deleted)
     SET @EventID = (SELECT eventid from deleted)
     SET @Computer = (SELECT A.eventcomputer from ESEventlogComputer as A, deleted as B WHERE A.id = B.eventcomputer)
     SET @EventMessageOld = (SELECT eventmessage from deleted)
     SET @EventMessageNew = (SELECT eventmessage from inserted)

     SET @Msg = ‘ESEventlogMain modified by ‘ + CONVERT(VARCHAR(20), USER_NAME(USER_ID())) + ‘ at ‘ + CONVERT(VARCHAR(20), GETDATE()) + ‘. Computer: ‘ + @Computer + ‘, Event ID: ‘ + CONVERT(VARCHAR(8), @EventID) + ‘, Event Number: ‘ + CONVERT(VARCHAR(16), @EventNumber) + ‘, EventMessage (old) =’ + @EventMessageOld + ‘, EventMessage (new) = ‘ + @EventMessageNew

     RAISERROR( 80000, 10, 1, @Msg)
END

This creates a trigger which will generate an event when the eventmessage column in the ESEventlogMain table is modified. You can remove the “IF UPDATE(eventmessage) …” clause (as well as the BEGIN & END statements) if you want to be notified of any changes to that table, this might however create some noise since acknowledging events will also perform an UPDATE on this table.

FYI: “deleted” and “inserted” are keywords that refer to either the old
record that was updated (=deleted) or the new data (=inserted).

dbtriggers_event.jpgAs you can see from the screen shot above, the message text from a logoff event was renamed to “Trigger Test”. So now that the event is in the event log, we can set up a filter in EventSentry to alert us:

trigger_filter.pngEvents generated from triggers always have the event id 17061, so it’s a good idea to restrict the filter further using the “Content Filter” field. From now on, when the ESEventlogMain table is modified, we will get an entry in the event log as well as an email.
Just remember that any database administrator can delete or modify triggers, so it’s crucial that you keep dba access to your database as restricted as possible.

Please see the Table Relationships topic in the EventSentry help file for more information on the database tables used by EventSentry.

Best,
Tames, Ingmar + Ryan.

Read more

EventSentry v2.90: Compliance Tracking for SOX, PCI, GLBA, HIPAA, FISMA, COBIT, …

ingmar.koecher 0 Comment Announcements, Event Log, EventSentry

This is round two in the new features available in EventSentry v2.90, and this time I’ll be covering the new compliance features.

Even though EventSentry was not originally designed to help with compliance, its event log consolidation capabilities made it an effective and economical solution to help our customers with their various compliance efforts throughout the years.

But while being able to filter and search through security events is helpful, it is not enough to quickly create reports that group information based on key elements, such as user creations, group modifications, policy changes and more.

In version 2.90 we addressed this by creating the new Compliance Tracking features which are based on the previous Tracking features.

This means that in addition to the “standard” event log consolidation that simply collects events and records them as is, compliance tracking intercepts specific events (e.g. account creation, logon/logoff, process creation), parses them, extracts the required information and records the relevant information in the EventSentry database.

Compliance Tracking covers the following auditing areas in Windows:

  1. Process Activity
  2. Console & Network Logons
  3. File Access Activity
  4. Account Management (User, Group & Computer accounts)
  5. Policy Changes
  6. Print Jobs

For example, finding out which group memberships changed over the last week is matter of two clicks in the web reports – and restricting a report to only reflect a particular group and/or action is just as easy.

But let me briefly outline the benefits of the individual tracking features:

Process Tracking
This feature records all process activity and lets you know which processes where started when, by whom, for how long and from which computer. This feature is not only useful for security purposes, but also helpful when troubleshooting or requiring statistical information (e.g. how often is PowerPoint being run).

Logon Tracking
This component tracks everything logon-related on your network, including console, successful as well as failed network logons. Using the console logon tracking for example, you can generate reports that show what time users logon and logoff, including from which computer, whether they are local admin and more details. Using the new network logon tracking, you can track successful as well as failed network logons. The included reports can reveal information such as which users logged on with a failed password, logon protocol distribution, most common reason for failed logons and more.

File Access Tracking
This feature is new in v2.90 and tracks all successful file access activity that has been enabled on files or directories. EventSentry does this by intercepting audit events that are generated when files or folders which are being audited. Since Windows Server 2003 and earlier don’t actually audit when objects are changed, but instead only audit the requested file access (click here for a related post), EventSentry can perform additional checks and verifications to complement the native auditing capabilities of the OS – such as checksum creation. Of course EventSentry also gathers additional information – such as the source computer from where a change was made.

Account Management Tracking
Also new in v2.90 is account management tracking, which encompasses user, group and computer account management tracking. This feature really makes life easier when you deal with large quantities of user, group and / or computer account changes.

For example, tracking a users group membership changes – even across computers and domains – is only a few mouse clicks away. Do you need to know which computer accounts were created in the last week in your domain? This only takes three clicks in the web reports.

Policy Change Tracking
Another feature added in v2.90, policy change tracking records the following “policy” events:

  • Domain Policy Changes
  • Audit Policy Changes
  • Kerberos Policy Changes
  • User Right Changes
  • Logon Right Changes
  • Trust Relationship Changes

Again, getting information about any of the above scenarios is extremely easy – such as seeing which user/logon rights were assigned in the last week or on which server the password policy was changed in the last 2 weeks.

Since none of tracking features are limited to hard-coded reports but instead are easily adaptable, they not only make your auditors happy – they provide you with valuable information. This allows you to utilize EventSentry not only for compliance but many other tasks, whether is security-related, for troubleshooting or something else.

As always, please see the documentation for more information. You can take a look at version history as well for a complete list of changes and new features in the 2.90 release of EventSentry.

Enjoy,
Ingmar.

EventSentry v2.90: Event Log Monitoring Changes

ingmar.koecher 4 Comments Event Log, EventSentry

Since we have just released EventSentry v2.90, we’ll be blogging about the improvements and new features in the coming weeks. Since event log monitoring is how it all started, my first post in this series will be about the improvements and new features in our event log monitoring engine.

Vista/Windows 2008
The biggest change in v2.90, in regards to event log monitoring, is of course the native support of the Windows Vista and Server 2008 event log API. As many of you know, Microsoft introduced a new API for event log monitoring while still keeping the legacy API in place for applications that don’t support the new API yet.

EventSentry v2.81 uses this legacy API with some work-arounds to monitor the new event logs, but I highly recommend upgrading to v2.90 if you’re monitoring Server 2008 and/or Vista event logs. Upgrading will result in less overhead and better formatting and presentation of events since the agents now access the event log with the native API. Naturally, the event log backup feature will backup event logs in the new evtx format on Vista/Server 2008 computers.

The new version also supports the new Operational event logs which are displayed under Application and Services Logs/Microsoft, for example the Microsoft-Windows-Backup/Operational log.

eventlogblog_290_eventviewer_1.pngThese operational logs need to be configured as custom event logs in EventSentry, by specifying the full path (e.g. Microsoft-Windows-Backup/Operational) as the name of the custom event log.

Please see one of my previous posts about the event log changes in Vista (which also applies to Server 2008) for more information.

Note that support for the new event log API is transparent, and there is still only one executable of the EventSentry agent for all versions of Windows.

64-Bit
EventSentry v2.81 did not format some events on 64-bit editions of Windows correctly, and we have resolved this problem in 2.90 which renders all events on 64-bit machines correctly. The EventSentry agent still runs as a 32-bit application in 2.90, but we have long-term plans to supply a 64-bit agent for x64 operating systems.

Filter Timers
Filter Timer filters allow you to ignore events that would otherwise trigger an alert, if they are followed by another event within a preset time period. For example, if an event indicating that a critical service is stopped is being immediately followed by another event indicating that the service is running again, then a filter would allow you to suppress both events.

Previously however, filter timers had to be setup exactly for each event pair. This meant that if you wanted to use a filter timer for 5 services, then you would have to create 10 events. Starting with 2.90 you only have to create 2 events now, as long as the first event and the clearing event share the same order of insertion strings – which is usually the case.

Please see the documentation for more information.

Action Trigger History
Selected actions (e.g. email, pager) now include the ability to log their trigger history – that is every time they are triggered by an event – to the database. This helps you confirm that a notification was in fact performed, and also gives you the ability to gather statistics about which actions are being triggered and how often.

eventlogblog_290_actiontrigger_1.pngThe action trigger history includes the following information:

•    Date/Time
•    Computer
•    Action Name, Action Recipients
•    Event Log Package, Filter Name
•    Event Log, Event Source, Event ID, Event Number

Please see the documentation for more information.

Web Reports: Error Explanation
Many events from the security event log, for example audit failure event 675, contain error numbers and failure codes inside the event that require you to research them in order to find out what they mean. Here is an example:

eventlogblog_290_event_1.pngYou can see that the failure code of 0x25 in itself doesn’t reveal too much, but if you view the same exact event through our web reporting, then the failure code is automatically explained for you:

eventlogblog_290_event_2.pngAs you can see in the screenshot above, the Kerberos failure code of 0x25 is automatically explained as “Clock skew too great”.

Copying / Pasting event details from Emails
If you have been using EventSentry for a while, then you’ve probably setup event exclusions more than once, most likely after receiving an email from one of the agents. Starting with 2.90, you can now copy the event in your email client and paste it into a new filter. The management console will parse the event properties and automatically fill in the following fields for you:

•    Event Log
•    Event Severity
•    Event Source
•    Event Category
•    Event ID

Please see the documentation for more information.

You can take a look at version history as well for a complete list of changes and new features in the 2.90 release of EventSentry.

Enjoy,
Ingmar.

NTFS Alternate Data Streams: Hiding data in plain sight since 1993

ingmar.koecher 0 Comment Event Log, EventSentry, Tips & Tricks, Tools & Utilities

I recently read an article in the Hackin9 magazine (worth taking a look if you haven’t heard about it) about alternate data streams (ADS) in NTFS. I had heard about this hidden feature in NTFS a long time ago actually, but over the years forgot about its existence again.

Background
In a nutshell, the NTFS file system, which was introduced with Windows NT 3.1, supports ADS – sometimes also referred to as “hidden streams”. This means that you can attach or associate any number of files to an existing file, yet those files will not be visible to the vast majority of file management applications – including explorer and the “dir” command (Vista can show ADS with a parameter). One thing I find interesting about streams is that a lot of people in IT do not seem to
know about them, even people otherwise very familiar with the Windows
Operating System.

Now, streams are created and accessed by appending the “host” file name with a colon, followed by the name of the stream. Let’s say you want to create a text file called financials.txt and hide it with winhelp.exe, you would run notepad C:\Windows\winhelp.exe:financials.txt. This will bring up notepad which will prompt you to create the file since it doesn’t exist (since the alternate stream is basically a file). You can then save any text in the hidden file and save it. You will notice that the file you just created will not show up when you do a directory list (dir C:\Windows) and will also not show up in the Windows Explorer. Note that the timestamp of the host file will change however.

Now there are of course a variety of utilities that have been developed in the last 15 (!) years that will allow you to find hidden streams, but more on that later. Hidden streams still exist on Vista and later, though the feature seems to have become more restrictive.

There are apparently no limits as to how many streams one can associate with a file, or the type of file that can be associated. This means that you can associate an executable as much as you can an ASCII file. There are however some limitations as to how user mode applications (e.g. notepad) can access hidden streams. Let’s go back to the previous example where we created the file financials.txt in winhelp.exe. If you open a command prompt and execute type C:\Windows\winhelp.exe:financials.txt, then you will not be able to see the contents of the hidden file. If you use notepad instead however, you will be able to see the file (notepad C:\Windows\winhelp.exe:financials.txt). This is probably because cmd.exe and its built-in commands up until Windows XP are not aware of alternate streams. ON a Windows XP machine I also could not open that same file if I tried to open it from inside notepad with the File -> Open command.

Creating Streams
Things get more interesting when you attach executables to files – and execute them! Let’s say I wanted to hide popular windows game solitaire inside the file C:\Windows\wganotify.log and call the stream “calc.exe”. Here is what you do:

type system32\sol.exe > C:\Windows\WgaNotify.log:calc.exe
start C:\windows\WgaNotify.log:calc.exe

Auditing Alternate Data Streams
Those of you interested in auditing will probably wonder how Windows tracks access to hidden streams in the event log. Well, there is good and bad news. The bad news is that object tracking (the famous event 560) does not show hidden streams, and instead only shows the “host” file name being accessed. Process Tracking on the other hand shows hidden streams in the expected manner. For the above example, a 592 event will show that file C:\windows\WgaNotify.log:calc.exe was executed.

Exploiting Streams
Scary, huh? This opens up a can of worms when you think about malware hiding inside otherwise innocent files – such as a log file. At appears as if most AntiVirus products do not detect hidden streams, at the same time there doesn’t seems to be a significant number of mainstream malware applications out there are that rely on hidden streams. I’m not sure why that is, since this feature seems almost too good to be true for the writer of any malicious applications. One reason might be that malware writers mostly target home machines, and many of those computers are still formatted with the FAT(32) file system, which of course doesn’t support ADS. This might change over time though, as more (home) computers use NTFS as their file system.

So after reading up on ADS, playing around with it last week, scanning my computer for hidden streams, I arrived at the inevitable question: What is the higher purpose of Alternate Data Streams? I mean, many applications don’t support it, most people don’t know about it, and a scan didn’t reveal any hidden streams besides a couple inside some Microsoft installers that apparently use them as some sort of meta data.

As it turns out, ADS was created for compatibility with the Macintosh HFS file system, which uses a data fork and resource fork to store data in a file (OS X now uses the HFS+ file system). But over the years (it’s been 15 after all) some developers at Microsoft decided to utilize this feature. For example, when you specify summary information about a file (right-click -> properties -> summary), then this information will be stored in ADS.

As mentioned earlier, there have been some improvements in regards to ADS with Vista and later. Vista can now show alternate streams with the /R switch of the “dir” command. My preliminary research also shows that hidden streams can no longer be executed in Vista or later – so what we did in the above example will not work. I think that’s a good thing, since there really is no practical reason (unless you develop malware) to do this. The screen shot below shows the output of a regular dir command and the dir /R command on a Windows 2008 server (note the file setupact.log).

ADS_Win2k8.jpgIn my humble opinion, Microsoft should get rid of alternate streams in future versions of Windows, and instead come up with some sort of structured way of embedding meta data in files. Anything contained in meta data should be non-executable and limited in size, e.g. 256kb.

Discovering Streams
So what does all this mean for you, the person responsible for security in your network? How can you find hidden streams and detect if streams are being added to files?

There are many free third-party utilities out there that show and manipulate hidden streams, but the discovery of this feature led us to extend the functionality of the File Monitoring feature of EventSentry to include the automatic detection of hidden streams in real-time. This means that any stream added, modified or removed from a file in a monitored location will be detected by EventSentry.

We have also developed a new command-line tool, adslist.exe, that will list all alternate data streams on a directory and optionally its sub directories. The tool is part of the NTToolkit v1.96 and I recommend that you schedule to run this tool with the Application Scheduler feature of EventSentry on a regular basis, or schedule it with the Windows Task Scheduler and email the results (adslist.exe C:\ /s). The advantage of using EventSentry is that the results of adslist.exe can automatically be emailed to you only if alternate streams were found. You can do this because the %ERRORLEVEL% is set to 1 by adslist.exe when one or more streams are found. The screenshot below shows what this would look like in the email sent by EventSentry:

EventSentry_ApplicationScheduler_ADSList.pngManipulating Streams
While Microsoft doesn’t offer a tool to search for and discover alternate data streams, they do offer a good explorer-extension that allows you to view and delete alternate data streams. You can download it from https://docs.microsoft.com/en-us/sysinternals/downloads/streams, the zip file contains the source code as well as another utility to create hard links on NTFS volumes. After extracting the archive, navigate to the \StrmExt\ReleaseMinDependency folder and run regsvr32 StrmExt.dll. You will then have an additional tab when viewing file properties in explorer called “Streams”:

StrmExt.jpgAnother way to get rid of hidden streams is to copy a file to a FAT[32] volume and then back to the NTFS volume, or – if you don’t have a FAT[32] volume available – simply compress and uncompress the file again.

Well, I hope this gives you a better understanding of alternate data streams, even if you were already familiar with them. Like I mentioned earlier, it doesn’t appear as if ADS is used for evil in a large scale quite yet (so no reason to panic!), but I believe it is better to be safe than sorry.

Event 4964: Special Groups Feature for Vista + Windows 2008 Entrepreneurs

ingmar.koecher 0 Comment Event Log, EventSentry, Pure Knowledge, Tips & Tricks

There is certainly a lot of talk about the benefits of using Vista, but a lot of administrators and users seem to be avoiding it and instead hold on to Windows XP – which now appears to have a better reputation than ever! Well, here is a small reason to upgrade to Vista or Windows Server 2008.

Microsoft introduced a new event, 4964, called the Special Groups Feature. The purpose of this feature is to log event 4964 to the security event log when a member of a group you specify logs on to a computer.

So let’s say you want to know when a member of a local Administrator group logs on to a computer (and with EventSentry you could get an email when that happens for example), then you can accomplish that with the special groups feature.

In order to use this feature you need to do three things:

  • Determine the SID of the group(s) you want to monitor
  • Specify the SID(s) of the groups you want to monitor in a registry key
  • Ensure that you are auditing the Special Logon Feature (enabled by default)

One way to obtain the SID of a group is to use the getsid.exe tool which is part of the Windows XP SP2 Support Tools and other Microsoft Resource Kits. Note that the primary purpose of this tool is to compare the SID of two user accounts (so it requires you to specify two user/group accounts), but you can just enter the same group name twice to get around this. Here is an example output of the tool:

getsid \\mydc “Domain Admins” \\mydc “Domain Admins”

The SID for account BUILTIN\Domain Admins matches account BUILTIN\Domain Admins
The SID for account BUILTIN\Domain Admins is S-1-5-21-9817441204-4587651373-9817264971-512
The SID for account BUILTIN\Domain Admins is S-1-5-21-9817441204-4587651373-9817264971-512

As you can see you need to point to tool to computer where the group exists, in our case I used a domain controller since I want to monitor if somebody from the Domain Admins group logs on to the computer. If you monitor a built-in group (e.g. Administrators) then you will see that the SID is much shorter and the same across all your computers.

Now that we know the SID, we can specify it in the registry. Navigate to key HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\Audit and create a new String with the name SpecialGroups.

The value for this new string will be the SID of the group you want to monitor, and you can separate multiple SIDs with a semicolon. For example:

S-1-5-32-544;S-1-5-32-123-54-65

You do not have to reboot after making this change, it is effective immediately with the first subsequent login. The event that is being logged will look similar to this (screen shot from the EventSentry Web Reports):

Special Groups Logon 4964 ScreenshotThe relevant information is shown in the lower part of the event in the New Logon section. Security ID shows the user that logged on, and Special Groups Assigned shows the group the account is a member of (of course this group has to be specified in the registry).

Voila. This feature probably makes most sense on critical servers, though I would recommend enabling it on all workstations as well since you probably want to know if a member of the local Administrators group logs on. But of course this also means that you need to be running Vista on your network :-).

Since this feature needs to be activated using the registry, you can use AutoAdministrator to push this registry change to multiple computers. AutoAdministrator has actually been rewritten from scratch and we will be releasing a new version 2.0 very soon.