EventSentry v2.90: Compliance Tracking for SOX, PCI, GLBA, HIPAA, FISMA, COBIT, …

ingmar.koecher 0 Comment Announcements, Event Log, EventSentry

This is round two in the new features available in EventSentry v2.90, and this time I’ll be covering the new compliance features.

Even though EventSentry was not originally designed to help with compliance, its event log consolidation capabilities made it an effective and economical solution to help our customers with their various compliance efforts throughout the years.

But while being able to filter and search through security events is helpful, it is not enough to quickly create reports that group information based on key elements, such as user creations, group modifications, policy changes and more.

In version 2.90 we addressed this by creating the new Compliance Tracking features which are based on the previous Tracking features.

This means that in addition to the “standard” event log consolidation that simply collects events and records them as is, compliance tracking intercepts specific events (e.g. account creation, logon/logoff, process creation), parses them, extracts the required information and records the relevant information in the EventSentry database.

Compliance Tracking covers the following auditing areas in Windows:

  1. Process Activity
  2. Console & Network Logons
  3. File Access Activity
  4. Account Management (User, Group & Computer accounts)
  5. Policy Changes
  6. Print Jobs

For example, finding out which group memberships changed over the last week is matter of two clicks in the web reports – and restricting a report to only reflect a particular group and/or action is just as easy.

But let me briefly outline the benefits of the individual tracking features:

Process Tracking
This feature records all process activity and lets you know which processes where started when, by whom, for how long and from which computer. This feature is not only useful for security purposes, but also helpful when troubleshooting or requiring statistical information (e.g. how often is PowerPoint being run).

Logon Tracking
This component tracks everything logon-related on your network, including console, successful as well as failed network logons. Using the console logon tracking for example, you can generate reports that show what time users logon and logoff, including from which computer, whether they are local admin and more details. Using the new network logon tracking, you can track successful as well as failed network logons. The included reports can reveal information such as which users logged on with a failed password, logon protocol distribution, most common reason for failed logons and more.

File Access Tracking
This feature is new in v2.90 and tracks all successful file access activity that has been enabled on files or directories. EventSentry does this by intercepting audit events that are generated when files or folders which are being audited. Since Windows Server 2003 and earlier don’t actually audit when objects are changed, but instead only audit the requested file access (click here for a related post), EventSentry can perform additional checks and verifications to complement the native auditing capabilities of the OS – such as checksum creation. Of course EventSentry also gathers additional information – such as the source computer from where a change was made.

Account Management Tracking
Also new in v2.90 is account management tracking, which encompasses user, group and computer account management tracking. This feature really makes life easier when you deal with large quantities of user, group and / or computer account changes.

For example, tracking a users group membership changes – even across computers and domains – is only a few mouse clicks away. Do you need to know which computer accounts were created in the last week in your domain? This only takes three clicks in the web reports.

Policy Change Tracking
Another feature added in v2.90, policy change tracking records the following “policy” events:

  • Domain Policy Changes
  • Audit Policy Changes
  • Kerberos Policy Changes
  • User Right Changes
  • Logon Right Changes
  • Trust Relationship Changes

Again, getting information about any of the above scenarios is extremely easy – such as seeing which user/logon rights were assigned in the last week or on which server the password policy was changed in the last 2 weeks.

Since none of tracking features are limited to hard-coded reports but instead are easily adaptable, they not only make your auditors happy – they provide you with valuable information. This allows you to utilize EventSentry not only for compliance but many other tasks, whether is security-related, for troubleshooting or something else.

As always, please see the documentation for more information. You can take a look at version history as well for a complete list of changes and new features in the 2.90 release of EventSentry.

Enjoy,
Ingmar.

EventSentry v2.90: Event Log Monitoring Changes

ingmar.koecher 4 Comments Event Log, EventSentry

Since we have just released EventSentry v2.90, we’ll be blogging about the improvements and new features in the coming weeks. Since event log monitoring is how it all started, my first post in this series will be about the improvements and new features in our event log monitoring engine.

Vista/Windows 2008
The biggest change in v2.90, in regards to event log monitoring, is of course the native support of the Windows Vista and Server 2008 event log API. As many of you know, Microsoft introduced a new API for event log monitoring while still keeping the legacy API in place for applications that don’t support the new API yet.

EventSentry v2.81 uses this legacy API with some work-arounds to monitor the new event logs, but I highly recommend upgrading to v2.90 if you’re monitoring Server 2008 and/or Vista event logs. Upgrading will result in less overhead and better formatting and presentation of events since the agents now access the event log with the native API. Naturally, the event log backup feature will backup event logs in the new evtx format on Vista/Server 2008 computers.

The new version also supports the new Operational event logs which are displayed under Application and Services Logs/Microsoft, for example the Microsoft-Windows-Backup/Operational log.

eventlogblog_290_eventviewer_1.pngThese operational logs need to be configured as custom event logs in EventSentry, by specifying the full path (e.g. Microsoft-Windows-Backup/Operational) as the name of the custom event log.

Please see one of my previous posts about the event log changes in Vista (which also applies to Server 2008) for more information.

Note that support for the new event log API is transparent, and there is still only one executable of the EventSentry agent for all versions of Windows.

64-Bit
EventSentry v2.81 did not format some events on 64-bit editions of Windows correctly, and we have resolved this problem in 2.90 which renders all events on 64-bit machines correctly. The EventSentry agent still runs as a 32-bit application in 2.90, but we have long-term plans to supply a 64-bit agent for x64 operating systems.

Filter Timers
Filter Timer filters allow you to ignore events that would otherwise trigger an alert, if they are followed by another event within a preset time period. For example, if an event indicating that a critical service is stopped is being immediately followed by another event indicating that the service is running again, then a filter would allow you to suppress both events.

Previously however, filter timers had to be setup exactly for each event pair. This meant that if you wanted to use a filter timer for 5 services, then you would have to create 10 events. Starting with 2.90 you only have to create 2 events now, as long as the first event and the clearing event share the same order of insertion strings – which is usually the case.

Please see the documentation for more information.

Action Trigger History
Selected actions (e.g. email, pager) now include the ability to log their trigger history – that is every time they are triggered by an event – to the database. This helps you confirm that a notification was in fact performed, and also gives you the ability to gather statistics about which actions are being triggered and how often.

eventlogblog_290_actiontrigger_1.pngThe action trigger history includes the following information:

•    Date/Time
•    Computer
•    Action Name, Action Recipients
•    Event Log Package, Filter Name
•    Event Log, Event Source, Event ID, Event Number

Please see the documentation for more information.

Web Reports: Error Explanation
Many events from the security event log, for example audit failure event 675, contain error numbers and failure codes inside the event that require you to research them in order to find out what they mean. Here is an example:

eventlogblog_290_event_1.pngYou can see that the failure code of 0x25 in itself doesn’t reveal too much, but if you view the same exact event through our web reporting, then the failure code is automatically explained for you:

eventlogblog_290_event_2.pngAs you can see in the screenshot above, the Kerberos failure code of 0x25 is automatically explained as “Clock skew too great”.

Copying / Pasting event details from Emails
If you have been using EventSentry for a while, then you’ve probably setup event exclusions more than once, most likely after receiving an email from one of the agents. Starting with 2.90, you can now copy the event in your email client and paste it into a new filter. The management console will parse the event properties and automatically fill in the following fields for you:

•    Event Log
•    Event Severity
•    Event Source
•    Event Category
•    Event ID

Please see the documentation for more information.

You can take a look at version history as well for a complete list of changes and new features in the 2.90 release of EventSentry.

Enjoy,
Ingmar.

Your favorite tools and utilities always available everywhere (almost)

ingmar.koecher 0 Comment Tips & Tricks, Tools & Utilities

If you are managing more than a handful of servers
and workstations in an Active Directory domain then you have probably come to
rely on a small set of utilities to help you with your daily tasks of managing
your servers and workstations. You probably carry those apps with you on a USB
stick (check out PortableApps.com) or have them sitting on some network drive for access when you need them,
and most of these tools are probably from different
developers and vendors.

 

I personally like a lot of the Windows Sysinternals tools such as
psexec, process monitor, Microsoft Network Monitor and of course the NTToolkit tools that we develop – but
everybody has his or her own preference as to what they need to get the job
done. And that’s exactly my point.

 

Wouldn’t it be great that every computer you manage –
workstation or server – always had
the tools you need automatically pre-installed? That way, they could be used in scheduled tasks, batch scripts etc. without you having to worry whether the tool is installed or not.

 

Well, it is possible – and you can do it without spending a
dime – thanks to
Active Directory and the folks at Caphyon in Romania
that provide a freeware edition of their Advanced Installer.

 

You probably see where I am getting at. Combine a MSI
created by Advanced Installer with your favorite tools and Active Directory’s
“Software Distribution” feature, and you get all your tools at your fingertips
– whenever you need them. In this post I’ll show you how to create a MSI with
your favorite tools and automatically publish that to some or all computers in
your AD domain. This approach also supports updates, so that you can publish
revised MSI packages with new tools and/or updates.

 

The nice thing about this solution is that the only
prerequisite is Active Directory, and even if you don’t have AD you still end
up with a MSI that you can easily install on any computer.

 

  1. Download
    & Install Advanced Installer Freeware. You can also go with one of
    their commercial products of course, but the freeware version is enough
    for our purposes here.

 

  1. Organize
    all of your tools into one location so that they can easily be added to
    the installer. This is not absolutely necessary but will make working on the installer a bit easier.

 

  1. Open Advanced Installer. It should open a dialog to create a new project. If not, go to File -> New.
  2. Select a project type of Simple and make sure “Use wizard to create the project” is checked.

    advanced_installer_create_project.png

  3. The New Simple Project Wizard comes up. Click next to continue, and give your project a descriptive name.
  4. Browse to the folder containing all your tools.
  5. The next box allows you to create shortcuts to specific programs. If you have any GUI programs in there then you may want to check them and then hit Next.

    advanced_installer_select_source_files.png

  6. Now you can hit Finish on the final screen to build the project. It will save it as a .aip file in the location of your choice.
  7. At this point we can just call it a day and compile the MSI file. I like having my command-line tools in the PATH environment variable however, so that I can access them conveniently from any folder. Keep reading for more info on this, otherwise just skip to step 12.
  8. Click on the “Environment” link in the left panel. Now right click anywhere in the blank space on the left side and choose “New Variable”.
  9. In the screen shot below you can see what settings to use to add it to the Path system variable.

    advanced_installer_environment_variables.png

  10. Now just click the “Build” button to create your MSI file.

Now that you have the MSI file, you can roll it out using Active Directory. If you need a refresher on how to do this, then please see the earlier blog post Applying Patches and Updates with Active Directory that shows how to distribute MSI-based patches with Active Directory – essentially the same concept.

Drawbacks & Considerations

Now, I have to admit that there are some drawbacks to using Active Directory to publish your MSI. Currently, group policy can only apply software installations in the foreground, which means that you will have to reboot a computer in order to have your new MSI installed. If you know a way around that – other than using a 3rd party software deployment suite – then please let me know.

Also keep in mind that any tool you install on your servers and/or workstations will, by default, be available to any user on that machine unless you adjust the permissions of the target folder manually. As such, I would refrain from including utilities in your MSI that make gaining unauthorized access easier, and also ensure that you always have the latest version of your tools in your MSI.

I hope this tip helps managing your servers and workstations a bit easier. Until next time,

Ingmar.

Mr. Fix It: Reviving a relative’s computer

ingmar.koecher Tips & Tricks, Tools & Utilities

Let’s face it, whether you’re a DBA, IT Manager or sys admin – you are eventually asked the inevitable by your parents, mother-in-law (hint) or brother-in-law: The computer is soooo slow – is there something we can do?

What you will probably find is a computer running Windows XP, an out-of-date Anti Virus software that was pre-installed, missing patches, 17 toolbars for Internet Explorer, some AdWare and a boatload of other software that nobody needs. If it’s bad then you’ll also find some SpyWare and viruses.

It unfortunately requires multiple steps of corrective action to get garbled systems like that back up to normal, and so I created this list for myself so not forget steps along the way. You can change the order, but this order should be most effective. If most of the items listed are obvious to you, then you can still use the list as a simple check list.

1. Uninstall all unneeded software: This should speed up the computer right away and get rid of some of the resource hogs. Don’t forget to get rid of any outdated AntiVirus software as well at this point.

2. Autoruns: Use the Sysinternals Autoruns tool to remove any applications that have nested themselves into one of the many autorun locations. I found the “Logons” and the “Services” tab to be most effective, though I recommend you check all of them. I also recommend saving the current setup prior to disabling things. After a reboot the computer should already be faster.

3. Remove Spyware: It’s generally a good idea to make sure that no Spyware is present, and I recommend running Super Antispyware on the computer. They have a free home edition that works quite well – and don’t get suspicious because of their cheesy web site.

4. Apply Updates: Now it’s time to switch to Microsoft Update and install any available critical updates and hardware updates that are relevant. You can switch to Microsoft Update by navigating to Windows Update and clicking the “Microsoft Update” link on that page on the left or right hand side. I generally recommend including optional updates, such as IE 7 and Windows Media Player as well. They might not use them, but this ensures that they are only using software that is going to be patched.

5. Anti-Virus: If the computer does not have Anti-Virus software installed at this point, download either a free package (e.g. Avast, AVG, etc.) or purchase a commercial one if they don’t mind paying an annual fee. Avast has the option to do an offline virus scan of your hard drive before the system boots, which I found quite useful.

6. Drivers & BIOS: Depending on the age of the system, chances are that at least some drivers are out of date, though it’s much more likely on a new computer than it is on an old one. A first start is Windows Update, but checking the vendor’s web site (some of them come with update software as well) usually yields better results and offers more recent drivers.

7. Defrag: I have a sort of love-hate relationship with defragging tools, and think that the benefits are often overstated. I have however seen many cases over my career where defragging does indeed improve performance. I don’t think a daily defrag is necessary on a workstation, but a computer that has never been defragged in years can definitely benefit from it. I have used Raxco’s PerfectDisk successfully on servers and workstations, and they have a fully functional 30-day trial available.

8. Hardware: In some cases, especially with older systems, it might help to upgrade the hardware. Upgrading memory is usually most effective, since it’s both cheap and easy to install. I listed it as a last step, even though upgrading it right-away might make performing all the other steps more enjoyable (=faster). Don’t just throw memory at the problem though, cleaning the machine up is the most important!

Or, you can do a re-install :-). Depending on the state of the OS, the above steps can sometimes take a very long time, and a re-install might be a better option – especially when the computer is infected with viruses and Spyware. As a matter of fact, if you still have access to the recovery CD and the computer wasn’t overly customized (e.g. used only for email, web and pictures), then a re-install will almost certainly be a better option and take less of your time.

Hope this helps – and if all else fails then you can always install Linux 😉

NTFS Alternate Data Streams: Hiding data in plain sight since 1993

ingmar.koecher 0 Comment Event Log, EventSentry, Tips & Tricks, Tools & Utilities

I recently read an article in the Hackin9 magazine (worth taking a look if you haven’t heard about it) about alternate data streams (ADS) in NTFS. I had heard about this hidden feature in NTFS a long time ago actually, but over the years forgot about its existence again.

Background
In a nutshell, the NTFS file system, which was introduced with Windows NT 3.1, supports ADS – sometimes also referred to as “hidden streams”. This means that you can attach or associate any number of files to an existing file, yet those files will not be visible to the vast majority of file management applications – including explorer and the “dir” command (Vista can show ADS with a parameter). One thing I find interesting about streams is that a lot of people in IT do not seem to
know about them, even people otherwise very familiar with the Windows
Operating System.

Now, streams are created and accessed by appending the “host” file name with a colon, followed by the name of the stream. Let’s say you want to create a text file called financials.txt and hide it with winhelp.exe, you would run notepad C:\Windows\winhelp.exe:financials.txt. This will bring up notepad which will prompt you to create the file since it doesn’t exist (since the alternate stream is basically a file). You can then save any text in the hidden file and save it. You will notice that the file you just created will not show up when you do a directory list (dir C:\Windows) and will also not show up in the Windows Explorer. Note that the timestamp of the host file will change however.

Now there are of course a variety of utilities that have been developed in the last 15 (!) years that will allow you to find hidden streams, but more on that later. Hidden streams still exist on Vista and later, though the feature seems to have become more restrictive.

There are apparently no limits as to how many streams one can associate with a file, or the type of file that can be associated. This means that you can associate an executable as much as you can an ASCII file. There are however some limitations as to how user mode applications (e.g. notepad) can access hidden streams. Let’s go back to the previous example where we created the file financials.txt in winhelp.exe. If you open a command prompt and execute type C:\Windows\winhelp.exe:financials.txt, then you will not be able to see the contents of the hidden file. If you use notepad instead however, you will be able to see the file (notepad C:\Windows\winhelp.exe:financials.txt). This is probably because cmd.exe and its built-in commands up until Windows XP are not aware of alternate streams. ON a Windows XP machine I also could not open that same file if I tried to open it from inside notepad with the File -> Open command.

Creating Streams
Things get more interesting when you attach executables to files – and execute them! Let’s say I wanted to hide popular windows game solitaire inside the file C:\Windows\wganotify.log and call the stream “calc.exe”. Here is what you do:

type system32\sol.exe > C:\Windows\WgaNotify.log:calc.exe
start C:\windows\WgaNotify.log:calc.exe

Auditing Alternate Data Streams
Those of you interested in auditing will probably wonder how Windows tracks access to hidden streams in the event log. Well, there is good and bad news. The bad news is that object tracking (the famous event 560) does not show hidden streams, and instead only shows the “host” file name being accessed. Process Tracking on the other hand shows hidden streams in the expected manner. For the above example, a 592 event will show that file C:\windows\WgaNotify.log:calc.exe was executed.

Exploiting Streams
Scary, huh? This opens up a can of worms when you think about malware hiding inside otherwise innocent files – such as a log file. At appears as if most AntiVirus products do not detect hidden streams, at the same time there doesn’t seems to be a significant number of mainstream malware applications out there are that rely on hidden streams. I’m not sure why that is, since this feature seems almost too good to be true for the writer of any malicious applications. One reason might be that malware writers mostly target home machines, and many of those computers are still formatted with the FAT(32) file system, which of course doesn’t support ADS. This might change over time though, as more (home) computers use NTFS as their file system.

So after reading up on ADS, playing around with it last week, scanning my computer for hidden streams, I arrived at the inevitable question: What is the higher purpose of Alternate Data Streams? I mean, many applications don’t support it, most people don’t know about it, and a scan didn’t reveal any hidden streams besides a couple inside some Microsoft installers that apparently use them as some sort of meta data.

As it turns out, ADS was created for compatibility with the Macintosh HFS file system, which uses a data fork and resource fork to store data in a file (OS X now uses the HFS+ file system). But over the years (it’s been 15 after all) some developers at Microsoft decided to utilize this feature. For example, when you specify summary information about a file (right-click -> properties -> summary), then this information will be stored in ADS.

As mentioned earlier, there have been some improvements in regards to ADS with Vista and later. Vista can now show alternate streams with the /R switch of the “dir” command. My preliminary research also shows that hidden streams can no longer be executed in Vista or later – so what we did in the above example will not work. I think that’s a good thing, since there really is no practical reason (unless you develop malware) to do this. The screen shot below shows the output of a regular dir command and the dir /R command on a Windows 2008 server (note the file setupact.log).

ADS_Win2k8.jpgIn my humble opinion, Microsoft should get rid of alternate streams in future versions of Windows, and instead come up with some sort of structured way of embedding meta data in files. Anything contained in meta data should be non-executable and limited in size, e.g. 256kb.

Discovering Streams
So what does all this mean for you, the person responsible for security in your network? How can you find hidden streams and detect if streams are being added to files?

There are many free third-party utilities out there that show and manipulate hidden streams, but the discovery of this feature led us to extend the functionality of the File Monitoring feature of EventSentry to include the automatic detection of hidden streams in real-time. This means that any stream added, modified or removed from a file in a monitored location will be detected by EventSentry.

We have also developed a new command-line tool, adslist.exe, that will list all alternate data streams on a directory and optionally its sub directories. The tool is part of the NTToolkit v1.96 and I recommend that you schedule to run this tool with the Application Scheduler feature of EventSentry on a regular basis, or schedule it with the Windows Task Scheduler and email the results (adslist.exe C:\ /s). The advantage of using EventSentry is that the results of adslist.exe can automatically be emailed to you only if alternate streams were found. You can do this because the %ERRORLEVEL% is set to 1 by adslist.exe when one or more streams are found. The screenshot below shows what this would look like in the email sent by EventSentry:

EventSentry_ApplicationScheduler_ADSList.pngManipulating Streams
While Microsoft doesn’t offer a tool to search for and discover alternate data streams, they do offer a good explorer-extension that allows you to view and delete alternate data streams. You can download it from https://docs.microsoft.com/en-us/sysinternals/downloads/streams, the zip file contains the source code as well as another utility to create hard links on NTFS volumes. After extracting the archive, navigate to the \StrmExt\ReleaseMinDependency folder and run regsvr32 StrmExt.dll. You will then have an additional tab when viewing file properties in explorer called “Streams”:

StrmExt.jpgAnother way to get rid of hidden streams is to copy a file to a FAT[32] volume and then back to the NTFS volume, or – if you don’t have a FAT[32] volume available – simply compress and uncompress the file again.

Well, I hope this gives you a better understanding of alternate data streams, even if you were already familiar with them. Like I mentioned earlier, it doesn’t appear as if ADS is used for evil in a large scale quite yet (so no reason to panic!), but I believe it is better to be safe than sorry.