Saturday Jan 20

Posts Tagged ‘correlation’

Logging, Correlation and IT Search: An Analogy
Last Updated on Monday, 21 July 2008 11:44
Written by JJ
Friday, June 6th, 2008

We were having some in-house training the other day and trying to demonstrate and explain the value of IT logging, event correlation and IT search functions to non-technical folk. Unfortunately, I think the data being used was unfamiliar and made it difficult to get the point across of what we can do with these tools and why we like them. Everyone was caught up in the whole “what does that src mean” and “what IP address is that” etc.

Sometimes I’m the queen of analogies (likely a trait I inherited from my Dad). Quite often my analogies are pretty silly, but they almost always get the point across.

So I was trying to work out an analogy to explain how we can use logs, events and searching and why these are advantageous. I was in the shower and it hit me! And… here it is.  FYI– If you’re a techie, just stop reading now… (I warned you).

The analogy. Imagine a house… actually, imagine your house. Let’s say that your house is like a network. The house and all the major appliance and structures of the house are like infrastructure devices- switches and servers, for example. Of course, the people living in your house are users. In addition you have ‘gateways’ from your house to the outside world, in the form of doors, windows, vents, etc. These house gateways are like our WAN devices- firewalls, IDS/IPS and other gateway appliances.

Let’s say you live in the house with your spouse and family. You’re going to be the wife for now, so imagine you, your husband, three kids and a dog (only because that amuses me). Each of your house users have a key to get in.

graphic_toastersyslog_lg.gifYour major appliances- the TVs, refrigerator, oven, the family computers and alarm system are all creating logs when anything happens and they’re all giving their logs to the toaster. (The toaster is greatly under appreciated so I’m giving him a big role here- yes- your toaster is the Syslog server). The doors, windows and other ‘portals’ to the outside are also creating events and logging each time they’re opened, closed, locked or broken and, they too, are sending their info to the toaster.

Here’s where life in your house gets interesting. Let’s figure out what’s normal… it’s probably normal for your husband to come home, do some work on the computer while you cook, and then everyone watch TV. The kids are doing their homework, playing on the computer and probably rummaging around the fridge for an after-school snack. You see your syslogging toaster shows you…  

  • the src= Refrigerator was opened multiple times in a short period of time between 3:43pm and 4:16pm by multiple users
  • the src= Kids Computer was logged off the Internet at 4:30 by user: Kid2
  • the src= Front Door was opened at 5:20pm by user: Husband
  • the src= Oven was turned on Bake at 350 at 5:32pm by user: You
  • the src= LivingRoom TV was turned on at 5:56pm by user: Husband
  • the src= LivingRoom TV channel was modified multiple times in a short period of time between 5:56pm and 6:02pm (your husband was probably looking for the ball game)

These are all things you expect to see. So, what’s not normal? Some things your toaster may tell you that would be out of the ordinary…

  • the src= Refrigerator was opened at 02:40am by user: Kid1
    What does this mean? Someone’s late-night snacking, no big deal.
  • the src= Kids Computer was logged onto the Internet at 02:45am by user: Kid1
    Uh-oh, Kid1 is gallivanting on the Internet in the middle of the night un-chaperoned. Might need to check that out.
  • the src= Front Door was attempted to be opened unsuccessfully 14 times in a short period of time beginning at 10:15am by user: UNKNOWN. The toaster logged the key code attempts tried by user UNKNOWN.
    Kids were at school, you were at work- someone’s trying to break in.
  • the src= Front Door was opened the next day at 1:20pm by user: ROOT
    You were still not home- someone just broke into your house.

Maybe we want to be alerted when these things are happening, or have happened. With some log search and correlation tools, in conjunction with your toaster syslog, we can get immediate alerts when something unexpected is happening. We could tell the log search to keep talking to the toaster and immediately send us a text message if the toaster sees the front door or any windows being accessed between 09:00am and 3:00pm on any weekday, by any user. If the toaster saw something happening, we would know immediately and could take appropriate actions- maybe call the police to notify them of a break-in.

Now, back to the network. Now that you have an idea of how we can use logs and events in the house to identify what’s going on and spot abnormal activity, we can port that over to our network. Go back and again think of the house and its appliances as resources on the network. We can see when someone- inside or outside- is trying to or has successfully accessed something and we can alert, take action, or keep logs and reports for future use and accounting.

Replaying events. If you’re using a super-nifty tool, you may be able to replay specific events back in a visual format- almost like a video into the network. Let’s take our Kid1’s midnight snacking. If we replayed all the events that contained user= Kid1 from time 10:00pm (bedtime) to 07:00am (gettin’ up time) we could see Kid1 go from the bedroom down to the kitchen, opening the fridge, watching TV for a bit before going back to the room and surfing the Internet for an hour. We could actually ‘watch’ these events happening with a re-constructed timeline. A great example (and my favourite toy) to do this is Splunk’s Replay application.

That’s the basic gist of it all. There are some other detailed ‘things’ we can do with these technologies, and I may elaborate on those another time. We all have A.D.D. and this one is long enough already!

# # #

Grasping Security thru Visualization
Last Updated on Monday, 21 July 2008 11:50
Written by JJ
Sunday, May 4th, 2008

Visualization is not a new concept to me- I’ve been turning data into various types of trends, charts, graphs, maps and 3D images for years. But, the concept of viewing and interpreting security and network data through visualization is relatively new- and I think you’re going to be seeing a lot more of this in the coming months and years.

One of the things I have the… pleasure… of doing, is consulting with various manufacturers to see how they can make their products and interfaces more usable. Specifically, I try to help them understand what to add or change in order to allow customers to interpret and use the data that’s being delivered to them. How can they take all this stuff, make sense of it, and correlate it to events on the network.

A lot of times that means finding ways to map data sources to known devices on the network, and parsing out what’s expected vs unexpected, or anomalous. We do this for WAN and LAN-based data, and for sources within the network, the DMZ and externally. It’s a lot of work and still not as wizard-like as we might hope.

But, I think I’ve just found my new favourite toy- and it came via Splunk. When I saw it, I just had to have it. :)

I didn’t get far with the Splunk demo at RSA, but totally made up for it at Interop, by way of an extremely knowledgeable woman – Christina Noren, the VP of Product Management there at Splunk. Talk about someone who knows her stuff. I was really amazed with what this little log search engine can do. And, add to that the overview of visualization I got from Raffy Marty, Chief Security Strategist, and I was totally blown away. With Splunk, you can quickly gain insight into the events happening on your network, and the visualization tools give you a unique and easy-to-interpret representation of the data.

The two together build a foundation for some great security tools, and ways to visualize data and trends for everything from PCI compliance to Change Management to Phishing attacks… and more.

Why is this important? I’m always looking for new ways to present data to customers. We can throw all the gadgets we want to on the network, but ultimately someone (not someTHING) needs to know what’s going on– especially in a world now where people are being held personally responsible for security- or lack there of. There’s a lot of data and events, and we need a way to turn that information into something useable. 

Go forth and play… You can download Splunk (yes, for free) at Check out the blogs and SplunkBase to get more cool tools and plug-ins. In a couple of months, Raffy’s new book Applied Security Visualization will be released and includes more in-depth information on using visualization in your environment. I strongly suggest you read it. Need more reasons to check it out? They have the BEST t-shirts ever…

Expect to see more from me on this topic, and some tips and tricks for Splunk…

# # #

More Content

Find more of my content at
- Low Tech Hacking book
- Dark Reading
- Network Computing
- SearchSecurity
- TechTarget

Get Social



Enter your email address:

Delivered by FeedBurner