Logging, Correlation and IT Search: An Analogy

We were having some in-house training the other day and trying to demonstrate and explain the value of IT logging, event correlation and IT search functions to non-technical folk. Unfortunately, I think the data being used was unfamiliar and made it difficult to get the point across of what we can do with these tools and why we like them. Everyone was caught up in the whole “what does that src mean” and “what IP address is that” etc.

Sometimes I’m the queen of analogies (likely a trait I inherited from my Dad). Quite often my analogies are pretty silly, but they almost always get the point across.

So I was trying to work out an analogy to explain how we can use logs, events and searching and why these are advantageous. I was in the shower and it hit me! And… here it is.  FYI– If you’re a techie, just stop reading now… (I warned you).

The analogy. Imagine a house… actually, imagine your house. Let’s say that your house is like a network. The house and all the major appliance and structures of the house are like infrastructure devices- switches and servers, for example. Of course, the people living in your house are users. In addition you have ‘gateways’ from your house to the outside world, in the form of doors, windows, vents, etc. These house gateways are like our WAN devices- firewalls, IDS/IPS and other gateway appliances.

Let’s say you live in the house with your spouse and family. You’re going to be the wife for now, so imagine you, your husband, three kids and a dog (only because that amuses me). Each of your house users have a key to get in.

Your major appliances- the TVs, refrigerator, oven, the family computers and alarm system are all creating logs when anything happens and they’re all giving their logs to the toaster. (The toaster is greatly under appreciated so I’m giving him a big role here- yes- your toaster is the Syslog server). The doors, windows and other ‘portals’ to the outside are also creating events and logging each time they’re opened, closed, locked or broken and, they too, are sending their info to the toaster.

Here’s where life in your house gets interesting. Let’s figure out what’s normal… it’s probably normal for your husband to come home, do some work on the computer while you cook, and then everyone watch TV. The kids are doing their homework, playing on the computer and probably rummaging around the fridge for an after-school snack. You see your syslogging toaster shows you…  

• the src= Refrigerator was opened multiple times in a short period of time between 3:43pm and 4:16pm by multiple users
• the src= Kids Computer was logged off the Internet at 4:30 by user: Kid2
• the src= Front Door was opened at 5:20pm by user: Husband
• the src= Oven was turned on Bake at 350 at 5:32pm by user: You
• the src= LivingRoom TV was turned on at 5:56pm by user: Husband
• the src= LivingRoom TV channel was modified multiple times in a short period of time between 5:56pm and 6:02pm (your husband was probably looking for the ball game)

These are all things you expect to see. So, what’s not normal? Some things your toaster may tell you that would be out of the ordinary…

• the src= Refrigerator was opened at 02:40am by user: Kid1
  What does this mean? Someone’s late-night snacking, no big deal.
• the src= Kids Computer was logged onto the Internet at 02:45am by user: Kid1
  Uh-oh, Kid1 is gallivanting on the Internet in the middle of the night un-chaperoned. Might need to check that out.
• the src= Front Door was attempted to be opened unsuccessfully 14 times in a short period of time beginning at 10:15am by user: UNKNOWN. The toaster logged the key code attempts tried by user UNKNOWN.
  Kids were at school, you were at work- someone’s trying to break in.
• the src= Front Door was opened the next day at 1:20pm by user: ROOT
  You were still not home- someone just broke into your house.

Maybe we want to be alerted when these things are happening, or have happened. With some log search and correlation tools, in conjunction with your toaster syslog, we can get immediate alerts when something unexpected is happening. We could tell the log search to keep talking to the toaster and immediately send us a text message if the toaster sees the front door or any windows being accessed between 09:00am and 3:00pm on any weekday, by any user. If the toaster saw something happening, we would know immediately and could take appropriate actions- maybe call the police to notify them of a break-in.

Now, back to the network. Now that you have an idea of how we can use logs and events in the house to identify what’s going on and spot abnormal activity, we can port that over to our network. Go back and again think of the house and its appliances as resources on the network. We can see when someone- inside or outside- is trying to or has successfully accessed something and we can alert, take action, or keep logs and reports for future use and accounting.

Replaying events. If you’re using a super-nifty tool, you may be able to replay specific events back in a visual format- almost like a video into the network. Let’s take our Kid1’s midnight snacking. If we replayed all the events that contained user= Kid1 from time 10:00pm (bedtime) to 07:00am (gettin’ up time) we could see Kid1 go from the bedroom down to the kitchen, opening the fridge, watching TV for a bit before going back to the room and surfing the Internet for an hour. We could actually ‘watch’ these events happening with a re-constructed timeline. A great example (and my favourite toy) to do this is Splunk’s Replay application.

That’s the basic gist of it all. There are some other detailed ‘things’ we can do with these technologies, and I may elaborate on those another time. We all have A.D.D. and this one is long enough already!

# # #