Enjoy your IDS (Part 1)

Although one would guess that you install an IDS (or more) within your infrastructure to get alerts and react against them, that is usually far from reality. Many of those systems, even quite a few in big organizations, produce logs that are only reviewed when a serious incident happens. I know that is hard to imagine, but let me guess that it can happen to you too.

Unfortunately the amount of alerts generated by an IDS under normal conditions is normally huge, especially when it hasn't been tuned up for the environment that it lives in. In those cases, reading the alerts one after the other is not an option for the average security professional that must dedicate his time to many other things. As computer geeks we believe that computers can do this kind of tasks better than us. So scripting should be your friend for these cases.

In today's post I will not only provide you with my opinion on how to do things, but I will also do some extra work ( :-) ) and contribute with one simple script to extract some info from your alerts file. So let me first define what the script can do for you.

If you don't read your alerts very often, chances are that you have to check a lot of false positives or, at least, not very relevant alerts. So it would be nice if you could separate your data into groups of different alerts and decide if they are relevant to you, maybe for later investigation, or can be safely ignored.

Let's assume that:

• your IDS is snort,


• the alerts are generated in the default format. One line per alert with Timestamp interface, alert ID, alert message, priority, protocol, source IP address, source port, destination IP address and destination port)

Your first move could then be starting by grouping the alerts by the alert message (i.e. the supposed attack that has been detected). In each of the groups resultant of that classification you can have alerts that can be safely ignored together with others that must be treated as an incident and the difference could be either the source IP address or the destination IP address (maybe, even the time stamp). So what a useful tool could do is classify the alerts into groups using one of the fields, showing the number of alerts per group. To be able to go deeper in your investigation you also want the tool to be able to fix that field to the value of the group that you want to investigate further (i.e. alert equals portscan) and show the data grouped by another field (for example, source IP address). If you use the criteria that I have just mentioned and you find out that you have several portscanning attempts from one internal address, you can investigate further if:

• This system is being used by some evil hacker.


• The system is compromised and some kind of malware is running in it.

This is what this program does. To be able to get the groups of alerts you would use:
alertgrp.py /var/log/snort/alert alert
Then, after the portscan alerts catching your eye, you can try to identify the source IP address by using:
alertgrp.py /var/log/snort/alert srcip alert='.*portscan.*'
You can verify which IP addresses were scanned by the allegedly guilty system using:
alertgrp.py /var/log/snort/alert dstip alert='.*portscan.*',srcip=10.0.0.1

I hope that these examples give you some glimpse of the way the tool can be used. Anyhow, the source is there and you can read it and modify it at your will.

I hope you agree with me that the language that you use to write this script is not very relevant as long as it is able to work with regular expressions and you are familiar enough with it to make it do what you want (Please make my day with a programming language religious war ;-P). In this case I chose Python (instead of Perl or shellscript + awk or grep) because I want to use this script for my next post an integrate this results with a visualization tool to enhance the experience and get more information from a glance to the data. More on that in the best blog near you really soon now.

Please, keep in mind that I am not claiming to by a Python guru (which in fact I am not), so the script might (and it probably will) contain errors. If you find any, let me know and I'll do my best to fix it and update the version. However, using memory to store all the field values instead of just counting them when reading is intentional and you will see why in my next post.

Hope you like to put some regular expressions on your life!

Are we improving security for VoIP while reducing security for the rest of the network?

On episode 66 of the BlueBox Podcast, a listener, Rhodri Davies, provided feedback about the "PSTN being more secure" (minute 44:43) where he played devil's advocate and pointed out the following facts about VoIP security (please, listen to the audio for the original comments):

• Does the improved security just apply to telephony itself rather than to the IP network as a whole?
• It seems VoIP protocols were designed to be hardly secured ;): Complex signaling, dynamically negotiated ports, point to point communications, etc. It is hard for conventional firewalls to deal with these protocols.
• Protecting the whole IP network is made more complicated by the presence of VoIP.
• Are we trying to improve security for the telephony while reducing security for everything else in the network?

He wanted to now Dan and Jonathan opinions, and Dan asked for other listeners comments, so I decided to jump in ;)
I agree with the comment that the design of VoIP protocols like SIP and RTP was not gracefully aligned with security. Fortunately, we've found security solutions to this (see below), and this is changing with the introduction of new VoIP protocols like IAX2. The fact the signaling and media traffic travels through a single port makes IAX2 more firewall friendly. At least it seems we are learning from the past, something that it is not always true in infosec.

It is true also that complexity and security are not very good friends, and that complex VoIP protocols are difficult to secure. However, we already have (and are in the process of getting standards for) the technologies required to do so, such as stronger signaling authentication and identity, signaling encryption through SIPS (TLS-based SIP) or SIP over DTLS, or media encryption using SRTP (Secure RTP) and key exchange mechanisms, like ZRTP or DTLS-SRTP.

In relation to the firewalls analysis, I agree that conventional firewalls were not designed to, and cannot, deal with VoIP protocols easily. Similarly to Web applications protection today, where we need specific Web-based firewalls to inspect all HTTP, Web 2.0 and Web services traffic, for VoIP we require special security devices like Session Border Controllers (SBC's) or Application-layer Gateways (ALG's) to inspect and filter VoIP traffic. These can be independent devices or get integrated as a module into traditional firewalls. Similarly to Web encrypted traffic (HTTPS), we also need solutions to deal with the inspection of VoIP encrypted communications. Apart from the obvious recommendation of segregating data and voice traffic in different networks or VLANs, I suggest to use different and separate devices/firewalls for the standard IP data traffic and for VoIP.

In the same way we do not consider Web and secure Web applications, and its inherent complexity, a motivating factor involved in reducing the overall network security (or we do? ;) ), I don't think VoIP does. Of course VoIP introduces new attack vectors, threats and risks, and we have the opportunity and the tools to enjoy dealing with them ;)

Honestly, I think VoIP is helping a lot to increase the overall security of network architectures. The introduction of VoIP brings network segregation awareness, and new and more secure network designs. A good design and architecture, based on defense in depth and layered security principles, directly helps to increase the overall network security.

As a security consultant, about seven years ago I started promoting the usage of network authentication and access through 802.1x (it even appeared on my first GIAC paper). I could never understand why we all agreed to the need of strong authentication and authorization mechanisms for systems and applications, but this was "never" the feeling for the network itself. With the help of penetration testing exercises, I've been educating organizations through the years about the internal threat risk and the need of layer-2 controls in data networks. The introduction of wireless technologies, with no physical barriers, incredibly helped to promote the usage of 802.1x/EAP. In some environments, 802.1x has been extended to the wired network too, and I think VoIP is going to tremendously help even more to extend 802.1x/EAP to any network based device (data or voice, wired or wireless). This will lead to more advanced network access/admission control (NAC) or protection (NAP) environments, even to specific ones focused on VoIP endpoints.

As the main author of the new hands-on SANS VoIP Security course (Security 540), I reflect in the author statement the need to overcome the "VoIP cannot be secure" misconception, and briefly answer the common question: "What is (or could be) more secure, VoIP or the existing PSTN?"

To sum up, to secure a VoIP network, and the data network as a consequence, besides architecting the network from a security perspective, you must implement multiple security controls:

• At layer-4 and above, over TCP or UDP, you need to use secure VoIP protocols (there are other options, but these are the most promising ones today):
  
  • Signaling: SIPS (TCP) or SIP over DTLS (UDP)
  • Media: SRTP
    
  • Key-exchange: ZRTP or DTLS-SRTP
• At layer-3 there are firewall-related VoIP solutions like ALG's and SBC's.
• At layer-2 there are multiple network related solutions, such as 802.1x/EAP, VLAN's, Private VLAN's (PVLAN's), switch port security, Port-based ACL's (PACL's), VLAN ACL's (VACL's), MAC and ARP monitoring, etc. We just must get used to implement all them in any IP-based network, data or voice-based!!

BTW, if your current or planned VoIP deployment includes DTLS, you must be aware of a recent critical flaw in OpenSSL's DTLS implementation (CVE-2007-4995) that could permit remote code execution. Please, check with your vendor if their VoIP products are based on the OpenSSL implementation and ask for fixes.

Investigating File Deletion from Windows File Servers - Part II

This article is the continuation of Part I, where I provided a network capture file in pcap format (file_deletion_full_trace.cap) and asked a few questions about the deletion of some files from a Windows file server:

Q1 - How many files were deleted?
Q2 - When?
Q3 - How?
Q4 - Who did it?
Q5 - From where?

First of all, let me thank those of you who submitted responses even though there was no prize. Thank you so much for sharing! Most of you, though, only included straight answers and not so much information about how you obtained them. That's OK, but if you did things differently than I'll be showing, please leave a comment with your method so we all can learn from it! There's usually more than one way to do... about everything!

OK, let's go for it. In this article I'll be addressing questions Q1 and Q2 and I'll leave the rest for a future article.

Q1 - How many files were deleted?

A quick method to find out which files were deleted through SMB is using wireshark's display filter "smb.disposition.delete_on_close==1" either in the GUI or with "tshark", wireshark's command line version. Note that I'll be showing tshark (text) output to make the post as light as possible, but you are welcome to use the same filters in the GUI if you want to follow along.

Again, using that filter:

C:\> tshark -nn -r file_deletion_full_trace.cap \
-R smb.disposition.delete_on_close==1 -t ad

1707 2007-03-23 09:45:55.436826  10.10.10.11 -> 10.10.10.3
 SMB Trans2 Request, SET_FILE_INFO, FID: 0x8004
2694 2007-03-23 09:51:29.713222  10.10.10.11 -> 10.10.10.3
 SMB Trans2 Request, SET_FILE_INFO, FID: 0x8001

We see that two files were marked for deletion. The way file deletion works in SMB is a two step process: first, the file is marked for deletion when closed, then, when the client closes the file the server proceeds to delete it. If we want to see the actual moment of the deletion we can show the "close" commands too:

C:\> tshark -nn -r file_deletion_full_trace.cap \
-t ad -R "smb.disposition.delete_on_close==1 or \
(smb.cmd==0x04 and (smb.fid==0x8004 or smb.fid==0x8001))"

1707 2007-03-23 09:45:55.436826 10.10.10.11 -> 10.10.10.3
 SMB Trans2 Request, SET_FILE_INFO, FID: 0x8004
1709 2007-03-23 09:45:55.444123 10.10.10.11 -> 10.10.10.3
 SMB Close Request, FID: 0x8004
1722 2007-03-23 09:45:58.063573 10.10.10.11 -> 10.10.10.3
 SMB Close Request, FID: 0x8001
2694 2007-03-23 09:51:29.713222 10.10.10.11 -> 10.10.10.3
 SMB Trans2 Request, SET_FILE_INFO, FID: 0x8001
2696 2007-03-23 09:51:29.717148 10.10.10.11 -> 10.10.10.3
 SMB Close Request, FID: 0x8001

Frame 1707 marks file 0x8004 for deletion and frame 1709 closes that file, finally provoking its deletion. The same goes for file 0x8001 on frames 2694 and 2696. Frame 1722 shows another close command for a file also identified as 0x8001, but that corresponds to a different file since file identifiers get reused after the previous file has been closed.

Now we need to map those file identifiers to file names. That's easy since Wireshark can tell us if only we ask for payload decoding (option -V in tshark; output trimmed for clarity):

Frame 1707
[...]
      Tree ID: 2051
          [Path: \\SERVER1\PROJECT1]
          [Mapped in: 1659]
      Process ID: 1440
      User ID: 2049
      Multiplex ID: 5504
  Trans2 Request (0x32)
[...]
      SET_FILE_INFO Parameters
          FID: 0x8004
              [File Name: \file4.txt]
              [Opened in: 1704]
          Level of Interest: Set Disposition Information (1013)
[...]
          .... ...1 = Delete on close: DELETE this file when closed
[...]
Frame 2694
[...]
      Tree ID: 2052
          [Path: \\SERVER1\C$]
          [Mapped in: 1767]
      Process ID: 348
      User ID: 2051
      Multiplex ID: 28674
  Trans2 Request (0x32)
[...]
      SET_FILE_INFO Parameters
          FID: 0x8001
              [File Name: \Shared Folders\Project1\file8.txt]
              [Opened in: 2691]
          Level of Interest: Set Disposition Information (1013)
[...]
          .... ...1 = Delete on close: DELETE this file when closed

Combining the tree paths and the file names we obtain the full network paths of the files being deleted:

FID 0x8004:  \\SERVER1\PROJECT1\file4.txt
FID 0x8001:  \\SERVER1\C$\Shared Folders\Project1\file8.txt

Note that although both files may have lived in the same directory in the server (that was the case in this particular case) they were accessed and deleted through different network shares (\\SERVER1\PROJECT and \\SERVER1\C$).

A question remains, though. How does Wireshark know what file name corresponds to which file identifier (FID) and network share? Hint: it's using the ID fields (Tree ID, Process ID, User ID, Multiplex ID). Yes, but how? Would you be able to verify whether Wireshark is correct by checking those fields yourself using appropriate filters without looking at Wiresharks' automatic decoding shown between square brackets? My recomendation: try it! It's fun and you'll learn a lot about SMB!

OK, Q1 solved. On to Q2.

Q2 - When?

This would seem trivial after having found all the information above. However, I don't think it is so much so. Let me explain why.

It might seem obvious that the files were deleted at the following dates and times:

\\SERVER1\PROJECT1\file4.txt:
2007-03-23 09:45:55.436826  (frame 1707)

\\SERVER1\C$\Shared Folders\Project1\file8.txt:
2007-03-23 09:51:29.713222  (frame 2694)

However, I have two problems with that statement.

For one thing, it can be argued that since these frames only contained the request for the files to be marked for deletion the files were actually deleted a little later, when the files were closed. Actually, it would be a little after that, when the server received the close command and then proceeded to delete them, but how long would that be if we want to be exact?

Let me add an additional piece of information to consider. Right after each deletion, we can see in the trace a message sent by the server to the client notifying that the deletion has been completed (filter "smb.nt.notify.action==2", frames 1711 and 2699). Combining this filter with the previous to show the mark for deletion, the close and the notify commands for each file we obtain this:

C:\>tshark -nn -r file_deletion_full_trace.cap -t ad \
-R "smb.disposition.delete_on_close==1 or (smb.cmd==0x04 and \
(smb.fid==0x8004 or smb.fid==0x8001)) or smb.nt.notify.action==2 \
and not frame.number==1722"

1707 2007-03-23 09:45:55.436826  10.10.10.11 -> 10.10.10.3
 SMB Trans2 Request, SET_FILE_INFO, FID: 0x8004
1709 2007-03-23 09:45:55.444123  10.10.10.11 -> 10.10.10.3
 SMB Close Request, FID: 0x8004
1711 2007-03-23 09:45:55.457336   10.10.10.3 -> 10.10.10.11
SMB NT Trans Response, NT NOTIFY
2694 2007-03-23 09:51:29.713222  10.10.10.11 -> 10.10.10.3
 SMB Trans2 Request, SET_FILE_INFO, FID: 0x8001
2696 2007-03-23 09:51:29.717148  10.10.10.11 -> 10.10.10.3
 SMB Close Request, FID: 0x8001
2699 2007-03-23 09:51:29.718439   10.10.10.3 -> 10.10.10.11
SMB NT Trans Response, NT NOTIFY

Now we can be more precise and say that the files were deleted at some point in the following time intervals:

\\SERVER1\PROJECT1\file4.txt, between
 2007-03-23 09:45:55.444123  (frame 1709), and
 2007-03-23 09:45:55.457336  (frame 1711)

\\SERVER1\C$\Shared Folders\Project1\file8.txt, between:
 2007-03-23 09:51:29.717148  (frame 2696), and
 2007-03-23 09:51:29.718439  (frame 2699)

However, I still have a problem with those timestamps. Let us concentrate on the first frame (1709), for example. It was seen on the network at 09:45:55.444123 on 2007-03-23, right? OK, but 09:45 in Madrid, in New York, or where? Actually, those of you living in a different time zone than me and trying the above commands will already be seeing different timestamps for the same frame numbers! Stating a date and time without stating the time zone is ambiguous.

To make a long story short, pcap files contain timestamps in UTC and wireshark translates them to local time at display time. In wireshark's website you can find more information.

You can see the difference in the time displayed by changing the time zone in your computer. Try this:

C:\>systeminfo | find "Time Zone"
Time Zone: (GMT+01:00) Brussels, Copenhagen, Madrid, Paris

C:\>tshark -n -r file_deletion_full_trace.cap -t ad \
-R frame.number==1709

1709 2007-03-23 09:45:55.444123  10.10.10.11 -> 10.10.10.3
 SMB Close Request, FID: 0x8004

Then, change your time zone to something different (e.g. GMT) and try again:
(tip: you can invoke the Date and Time control panel applet directly with "control timedate.cpl")

C:\>systeminfo | find "Time Zone"
Time Zone: (GMT) Greenwich Mean Time : Dublin, Edinburgh, Lisbon, London

C:\>tshark -n -r file_deletion_full_trace.cap -t ad \
-R frame.number==1709

1709 2007-03-23 08:45:55.444123  10.10.10.11 -> 10.10.10.3
 SMB Close Request, FID: 0x8004

Obviously, the above output is from a Windows system. Unix folks will need to play with the environment variable "TZ".

Finally, bear in mind that the option to "Automatically adjust clock for daylight saving changes" also affects the output, although in this particular case it doesn't because the 23mar07 the time hadn't been yet changed for the summer.

With all this, we can conclude that the files were deleted in the time frames stated above (*around* 09:45:55 and 09:51:29 on 2007-03-23), those times being expressed in the timezone "(GMT+01:00) Brussels, Copenhagen, Madrid, Paris" with daylight saving changes enabled.

Answers to Q3 through Q5 to come in a future article...

The Evidence is in your Monthly Cell Phone Bill...

Yesterday I got my monthly cell phone bill and got surprised by the SMS (Short Message Service) charges. It was not a lot, about 5 €, although it is too much for the amount of SMS messages I'm used to send in a month (just a few). Therefore, I decided to review the invoice details. This is why we are tired of recommending you need to collect and review your logs, no matter the device ;)

Surprisingly, I saw 26 consecutive messages going to the same cell number (BTW, not familiar to me) in a period of 15 minutes.

Imagine you find yourself in that situation. Try to answer the following questions:
1) What are the incident response steps you would follow to discern what happened?
2) What do you think is the most probable security threat/vulnerability/exploit that could explain this type of incident?


I'll detail in a later post what my initial analysis and actions were, trying to understand what happened and get the money back ;)

Please, send your answers to radajo@gmail.com and I'll post a summary of your comments together with my post. Sorry, there are no prizes this time, just the glory of getting your name posted in this blog (and the fun of the exercise) ;)