Security Book Review: Chained Exploits

Chained Exploits: Advanced Hacking Attacks from Start to Finish
Author: A. Whitaker, K. Evans, J. B. Voth
Editorial: Addison-Wesley Professional
Publication date: March 9, 2009
ISBN-10: 032149881X
ISBN-13: 978-0321498816



Summary: A multi-scenario hacking adventure novel focused on combined real-world attacks.

Score: 5/5

Review:
The penetration testing (and criminal) field has focused during the last years on increasing the foothold on compromised systems, proving advanced pivoting and post-exploitation techniques that might help to expand the compromise to other systems or critical resources. This book is a novel that describes these reality by telling hacking stories where multiple techniques, tools and vulnerable input vectors are exploited in order to accomplish a variety of clearly defined attacks and goals.

Each chapter is a well structured story describing multiple attack scenarios. From credit card theft, to insider threat, going through corporate espionage focused on stealing confidential intellectual property, the launch of a DoS attack in a key point in time, the risk and exploitation of inter-corporation network connections, physical access to healthcare records, up to social networking and wireless break-ins.

The book is a modern fictional narrative with technical touches, covering attacks from start-to-finish in elaborated stories (my score evaluates the book from this perspective). However, by reading the book description, you might expect a deeply technical book that will teach you how to perform those attacks, and... it is not.

Every attack story is introduced by setting the stage and the overall attacker approach. Besides that, it is surrounded by a few final defensive tidbits and conclusions, describing countermeasures to mitigate the various attacks covered. This book may act as an excellent eye opener for managers and top level positions (see recommended audience below) in order to understand how small security investments and tweaks can definitely help to increase the overall protection of a target environment substantially.

Unfortunately, from a technical perspective, some of the technical details have not been thoroughly reviewed, such as the output of nmap (order of ports), the unexplained switching of target systems from Vista to XP, the targeting of RDP while not on the port scan (chapter 4) , or the coverage of some tools. Some attacks are a bit outdated, such as the silent winpcap installation to capture traffic from a target box. However, I must admit this book inspired some of the components of a recent "Prison Break" hacking challenge I released this summer (2009).

Specific portions of the book and, overall, the story plot, is well written from a novel perspective, and as particular attacks are progressing, it made me feel the common excitement we get when we are involved in a real penetration test and successfully progressing through the targets, getting the adrenalin going.

This book is highly recommended for people entering in the security field, and for experienced technical security pros in two ways. On the one hand, it's an enjoyable and entertaining novel for a weekend or vacation period. On the other hand, it is a very good reference to give to managers and CxO positions so that they can get a feeling of how real-world attacks look like nowadays and the kind of targeted threats they may face.

UPDATE: Amazon review.

Security Book Review: VMware vSphere and Virtual Infrastructure Security - Securing the Virtual Environment

"VMware vSphere and Virtual Infrastructure Security: Securing the Virtual Environment"
Author: Edward L. Haletky
Editorial: Prentice Hall PTR
Publication date: July 2, 2009
ISBN-10: 0137158009
ISBN-13: 978-0137158003



Summary: The reference for securing virtual environments, in particular, VMware-based.

Score: 5/5

Review:
In the first half of this year (2009), I was involved on extending my previous research on virtualization security, and specifically, I focused on securing and hardening VMware ESX environments. This stirred up my interest on this book. To sum up what this book is all about: "I would have loved to have this book handy back by that time, as it would have saved me tons of time" Instead, I had to read and compare multiple VMware security guides from VMware, CIS, NIST, etc, and perform an extensive hands-on research on my own.

The book offers a very solid and broad analysis of multiple security issues on virtual environments, covering not only the technical aspects associated to the virtualization hosts, virtual machines, and virtual data and storage networks, but also management and operational issues, availability concerns, and other common related tasks on newly deployed, or already established, virtualization setups.

The first two chapters focus on security threats and attacks, a basic foundation required for the cross-references available throughout the book, that can be skipped by the on-the-field security readers.

The next three chapters focus on offering best practices and security recommendations for different key components of any virtualization platform, such as the hypervisor, the storage network, and virtual clusters. The next couple of chapters cover most of the security aspects that must be considered on the design, deployment and operation of a virtual environment.

Although all these chapters provide a very good quality security advice, it is not complemented with hands-on examples. I think this could be improved by adding more detailed sections describing step-by-step how to complete the security recommendations exposed, not just what need to be done. However, I understand it is required to cut the size of the book at some point. A good example of how to extend this idea can be observed on chapter 6, where the integration between VMWare ESX and a directory service is covered in depth.

However, both the technical and operational aspects are integrated smoothly, offering a great in-depth overview. Apart from that, the whole recommended list of things to consider in order to get a more secure virtualization infrastructure is summarized in a useful set of boxes called "Security Notes" and spread all throughout the book. These boxes can be easily used as a checklist when deploying or assessing the security of virtual solutions.

My favourite chapters are chapter 8, and specially 9, where virtual machine and virtual networking security is analyzed, respectively. Chapter 9 offers a whole set of networking scenarios and discusses pros and cons to the number of (physical and virtual) network cards and its configuration. A very practical and thorough work!

The book ends up with three special chapters. Chapter 10 covers the new VMware virtual desktop infrastructure (VDI) and the security issues around it. Due to all the client-based attacks nowadays, most probably it is going to be a de-facto standard pretty soon, so getting involved on the virtualization of client systems is a must. Chapter 11 provides a detailed guide to harden VMware ESX and ESXi hosts, a mandatory initial process for every new virtual deployment. Finally, chapter 12 provides a quick and interesting introduction to digital forensics (and data recovery) on virtual enviroments, mainly focused on how to deal with virtual file systems, such as VMFS, VMDKs, and raw disks. A quick recommended read for forensic analysts interested on expanding their skills to virtual victims.

There are a few things I feel will improve the book contents. Unfortunately, due to the publication deadline, its coverage of the latest VMware vSphere virtual architecture is pretty limited, as the author clarifies. Besides that, considering the frequent security updates and patches released by virtualization vendors, I would have liked to find a better coverage of best practices to update the virtual infrastructure itself. Finally, as mentioned previously, about half of the book includes detailed how-to sections describing how to apply the recommended settings, but the other half misses that how-to portion. I understand this may be a limitation to make the book size manageable (it's over 500 pages now).

This book is highly recommended for IT and security architects, involved in the design of new virtual solutions, as well as virtualization administrators and anyone in charge of the maintenance of a virtual infrastructure. From a security perspective, people evaluating, assessing, and suggesting improvements for virtual solutions should read the book in order to have a full overview of all the security threats and possible countermeasures. Overall, the book is a must read for anyone already involved, or planning to get involved, in virtualization. It really helps to acquire a very broad and extensive knowledge of the security considerations that apply to such a complex and modern IT architectures.

UPDATE: Slashdot review, Amazon review.

Samurai Web Testing Framework (WTF) Firefox Add-ons Collection

On June 2009 Mozilla released the add-ons collections feature on their add-ons web site. As a member and contributor to the SamuraiWTF project, I would like to announce the release of the SamuraiWTF Firefox add-ons collection!

The Samurai Web Testing Framework (WTF) is a LiveCD focused on web application testing. It contains a pre-installed collection of the top web application penetration testing tools, becoming the perfect environment for testing applications.

The goal of this Firefox collection is to include the best add-ons for web application penetration testing and offensive security analysis, to convert your browser in the ultimate pen-testing tool. It is aligned with the Samurai Web Testing Framework (WTK) LiveCD distribution. I plan to keep the collection updated with new web-app pen-testing add-ons, but I would like to carefully evaluate new additions (or replacements) so that the list doesn't grow to limits where it becomes unmanageable. It includes 19 add-ons at this time.

As of today, it seems it is not possible to install all add-ons from a collection with a single click. The current SamuraiWTF add-ons collection can be installed on the latest Firefox version, v3.5, with the exception of the "Add N Edit Cookies" add-on. Although this add-on works in Firefox 3.5.*, it cannot be directly installed. There is a quick hack you can apply to install it on Firefox 3.5 until the official version is updated by its developer:

• Go to the "Add N Edit Cookies" add-on webpage with a compatible old Firefox version, or with a different browser like Internet Explorer, and download the add-on (XPI file).
• Change the XPI extension on the file to ZIP.
• Extract the "install.rdf" file from the ZIP archive.
• Edit the "install.rdf" file and replace the following line (maximum version):

        <em:maxversion>3.0.*</em:maxversion>

• by:

        <em:maxversion>3.5.*</em:maxversion>

• Put (drag & drop) the new "install.rdf" file back into the ZIP archive, and it will automatically replace the old version of the file.
• Change back the ZIP extension on the file to XPI.
• At this point, you can install the recently modified XPI add-on in Firefox 3.5.
  

Once you install all the add-ons within the SamuraiWTF collection, one by one, the look and feel of your Firefox browser will notably change. I recommend you to hide the add-ons toolbars visible by default. You can individually enable them at any time, such as when you are going to use each specific add-on:

• Go to the "View" menu and select "Toolbars".
• Deselect "Access Me Toolbar", "Web Developer Toolbar", and (specially) "HackBar".
  

Finally, the "DOM Inspector" add-on has been added to the collection as it is a requirement to enable all the capabilities of the "Web Developer" add-on.

Please, take a look at the collection, feel free to share your thoughts/comments (send me an e-mail), vote for this collection if you find it useful, and enjoy it!

Prison Break - Breaking, Entering & Decoding - Challenge Answers & Winners

The answers and winners for the EH-net "Prison Break (Breaking, Entering & Decoding)" challenge (August 2009) have been published today.

The answers for this challenge were released in scoop to The Informer subscribers a few days ago. In Johnny Long words, "The Informer is a fund raising effort run by Hackers For Charity. It is designed to give subscribers a "backstage pass" to the world of Information Security. For $54 per year, subscribers get early, exclusive access to all sorts of goodies donated by the top names in the INFOSEC world. The industry's most recognized names will post blog entries here before they even post them to their own sites." The EH-Net contribution will be the answers to the Skillz H@ck1ng Challenges a few days before they are revealed on EH-Net.

It is an honor for me to drive this initiative, with the support of Don Donzal (EH-Net) and Ed Skoudis (Challenge Master), and start posting the official answers of this challenge on The Informer.

The “Prison Break – Breaking, Entering & Decoding” challenge winners have been announced on EH-net, and the answers are contained in a single PDF file (27 pages) plus three associated screencasts:

• "BTv4 802.1q (VLAN) setup"
  
  

  BTv4 802.1q (VLAN) setup from siles on Vimeo.
  
  

• "Metasploit meterpreter Windump/Winpcap sniffer"
  
  

  Metasploit meterpreter Windump/Winpcap sniffer from siles on Vimeo.
  
  

• "Metasploit meterpreter built-in sniffer module"
  
  

  Metasploit meterpreter built-in sniffer module from siles on Vimeo.
  
  

Thanks to everybody for participating on the challenge, and to Ed and Don for the opportunity. I hope you enjoyed working on it as much as I enjoyed designing and writing it!

Sqlninja & Metasploit Demo

Last week I run the "Web App Pen-Testing" SANS webcast to provide a sneak preview of the SEC542 "Web Penetration Testing and Ethical Hacking" course I will be teaching in London later this year. At the end of the webcast I run a Sqlninja & Metasploit demo over the Hacme Bank vulnerable site using the recently released sqlninja patch.

This post includes a screencast of that demo (15:40 minutes):



You can access the archived version of the full SEC542 webcast from the SANS portal. Hope to see some of you, RaDaJo readers, in London!

Sqlninja & Metasploit

Sqlninja is one of the best open-source tools to automate SQL injection exploitation against MS SQL Server databases. If you combine it with the best open-source network penetration testing framework, Metasploit, you get an extremely powerful web application pen-testing toolkit for total database p0wnage!

This week I have been preparing a sqlninja demo focused on its integration with Metasploit for next week "Web App Pen-Testing" SANS webcast, scheduled for October 1. During the webcast I'll cover a sneak preview of the SEC542 "Web Penetration Testing and Ethical Hacking" course I will be teaching in London later this year, and run a demo using the latest publicly available sqlninja version, 0.2.3-r1, including the quick fix detailed below (0.2.3-r1p).

Sqlninja is a Perl-based tool that can make use of Metasploit capabilities to upload and run a Meterpreter or VNC server payload on the target MS SQL server through SQL injection flaws on the target web application. The integration of these tools accepts both direct and reverse TCP connections to/from the database server and the pen-tester system. It uses the "msfpayload" tool to generate the payload that will be executed on the database server (metxxxxx.exe), and the "msfcli" tool to establish (or wait for) a connection with that payload.

Due to the extensive number of modules available in Metasploit nowadays, the msfcli execution takes around 20 seconds in a BTv4 virtual machine to load the whole Metasploit module tree:

# ./msfconsole -v
Framework Version: 3.3-dev
# ./msfconsole
=[ msf v3.3-dev [core:3.3 api:1.0]
+ -- --=[ 404 exploits - 248 payloads
+ -- --=[ 21 encoders - 8 nops
=[ 188 aux
...

# time /pentest/exploits/framework3/msfcli
[*] Please wait while we load the module tree...
...
real    0m18.568s
user    0m13.402s
sys     0m4.683s
#

As a result, the current sqlninja Metasploit module may fail due to a race condition mainly on reverse mode, and specifically, due to timing issues of when the module initiating the connection(client role) executes versus the module listening for the connection (server role).

The patch released in this post fixes this race condition by adding specific (client & server) timeouts to the bind_tcp and reverse_tcp connections. The timers for the bind case try to ensure that the server (msfpayload) starts before the client (msfcli) tries to establish a connection. The timers for the reverse case try to ensure that the server (msfcli) starts before the client (msfpayload) initiates the reverse connection. The values for the timers are conservative and set a difference of 25 seconds between the server and the client initialization.

Due to the fact the reverse Metasploit payload does not retry the client connection, if the other end is not listening when the connection is initiated, the connection never succeeds and cannot be established. Additionally, I've seen the CPU of the target DB system going up to near 100% (a non-desirable DoS condition during a professional pen-test).

The patch can be applied by renaming the original 0.2.3-r1 "sqlninja" Perl file to "sqlninja.original" and running:

$ patch sqlninja.original -i sqlninja.patch -o sqlninja

The new timers ($client_delay (30 secs) and $server_delay (5 secs)), defined at the beginning of the sqlninja main file, can be changed to accommodate future Metasploit startup delays, or even be converted into sqlninja configuration options within the sqlninja.conf file. The patch changes the sqlninja version to 0.2.3-r1p, to indicate the patch has been applied.

Enjoy it, (sql) ninja pen-testers!

Looking for the right event

Not so long ago, during an incident investigation, I needed to reconstruct a series of events from several Windows systems. I needed to do so from the system that I was using to conduct the whole investigation which had Linux installed in it. That didn't make things easier because, as you will already know, Windows event logs are binary.

Two Google minutes later, I had downloaded a perl script written by Christophe Monniez that was able to do the work. The script turned out to be quite useful (Thanks Christophe!) but I need more. I had lots of events from several systems that were interrelated and needed to be interpreted to be able to understand the way the attack had been conducted, in order to add only the relevant stuff to the timeline. Going back and forward with such a big amount of events searching for the right one wasn't an option, so I decided to provide me with some search capabilities and add my own perl script to do so. The concept is trivial, I wanted to be able to search for some string with in the event, but I want the output to show the complete event instead of the line that matched the string only. You can do this easily with awk, but I rather use perl. Here is my little script in case it can also be helpful to you.

#!/usr/bin/perl
$/ = "\n\n\n";

die "Error: search string missing." if (@ARGV < 1);

while ($line = <stdin>) {

   if ($ARGV[0] eq "-v") {
       print $line if ($line !~ /$ARGV[1]/i);
   } else {
       print $line if ($line =~ /$ARGV[0]/i);
   }
}

This incident investigation was fairly successful and we had access to one laptop involved in the attack. However the system had been reformated and reinstalled, but some information could be recovered using the usual forensic tools. The event file was partially corrupted and I needed to recover the events that were still available. I rewrote the Christophe's code, that was available under the GPL license, and ended up with the following script that does exactly that.

#!/usr/bin/perl -w 

# Process Microsoft event file fragments.
#
# Copyright (c) Jorge D. Ortiz Fuentes, 2009
# Based on Monniez Christophe's code.
# - Added hability to process a fragmented event files.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.

# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
#

use strict;
use Getopt::Std;

#
# Help information
#
sub usage {
    print STDERR "\nUsage:\n\t$0 [-] \n";
    print STDERR "Options:\n";
    print STDERR "\t-d\tDebug information.\n";
    print STDERR "\t-l\tUse localtime instead of GMT.\n";
    print STDERR "\t-u\tPreserve unicode.\n";
    print STDERR "\t-h\tPrint this help and exit.\n";
    print STDERR "\nfile\tThe evt file to be analyzed.\n\n";
    exit 1;
}

#
# Search for the first record inside the file.
# It doesn't require that the signature is DWORD aligned 
#
sub next_signature {
    (my $debug, my $file) = @_;

    my $bytes_read;
    my $signature;
    my $sig_found = 0;

    do {
 $bytes_read = read($file, $signature, 1);
 die("End of file reached.\n") if ($bytes_read <= 0);
 if ($signature eq "L") {     
     $bytes_read = read($file, $signature, 3);
     die("End of file reached. Signature not found.\n")
  if ($bytes_read <= 0);
 }
 $sig_found = 1 if (($signature eq "fLe") && (tell($file) >=8));
    } while ($sig_found == 0);
    # Move the position in the file 8 bytes backwards, to 4 bytes before
    # the signature that is where the length of the record is stored.
    seek ($file, -8, 1);
    if ($debug) {
 print "Record starts in position ", tell($file), "\n";
    }
}


#
# Extract record information.
#
sub process_record {
    (my $debug, my $file, my $length, my $localtime, my $unicode) = @_;

    # Local variables
    my $record;
    my $t_gen;
    my $t_writ;
    my $rest;
    my $rest_reencoded;

    # Process the fixed part of the record (at least 56 bytes and
    # it is in position 4):
    read($file, $record, 52);
    $length -= 56;

    # Extract the data from the structure
    (my $reserved, my $record_nb, my $time_gen, my $time_writ,
     my $event_id, my $event_type, my $nb_strings, my $evt_category,
     my $reserved_flag, my $cl_record, my $string_offset,
     my $SID_leng, my $SID_offset, my $data_len, my $data_offset) =
  unpack "LLLLLSSSSLLLLLL" , $record;
    # The reserved field must be 1699505740 otherwise skip this record
    if ($reserved == 1699505740) {
 # Convert dates into strings
 if ($localtime) {
     $t_gen  = localtime($time_gen) . " localtime";
     $t_writ = localtime($time_writ) . " localtime";
 } else {
     $t_gen  = gmtime($time_gen) . " GMT";
     $t_writ = gmtime($time_writ) . " GMT"; 
 }
 # Print data 
 print "Record number:  $record_nb\n";
 print "Time generated: $t_gen\n";
 print "Time written:   $t_writ\n";
 print "Evt ID: $event_id  Evt type: $event_type Evt category: $evt_category\n";
 if ($debug) {
  print "* Reserved: $reserved\n"; 
  print "* $nb_strings strings\n";
  print "* String offset: $string_offset\n";
  print "* SID Len: $SID_leng SID offset: $SID_offset\n";
  print "* Data len: $data_len Data offset: $data_offset\n";
 }

 # Process the rest of the record: Source program, computer name, SID
 # and other strings
 if (read($file, $rest, $length) < $length) {
  die ("End of file reached while reading strings\n");
 }

 $rest_reencoded = pack "C*" , unpack "U0C*" , $rest;

 # Split into several strings
 my @strings = split(/\0\0/, $rest_reencoded);
 my $str;
 $str = $strings[0];
 # hack to suppress unicode
 $str =~ s/\0//g unless ($unicode);
 print "Program: $str\n";
 $str = $strings[1];
 # hack to suppress unicode
 $str =~ s/\0//g unless ($unicode);
 print "Computer: $str\n";
 my $i=0;
 while ($i < $nb_strings) {
  $str = $strings[$i+2];
  $str =~ s/\0//g unless ($unicode);
  print "String $i: $str\n";
  $i++;
 }
 print "\n\n";
    } else {
 print "Reserved: $reserved\n" if ($debug);
 print STDERR "RECORD REJECTED: reserved value fails to match!\n\n\n";
 # Searching continues from where it is since this is a corrupted record
    }
}


#
# Main program
#

# Variable declarations
my $evt_file = "";
my $record_sig;
my $record;
my $length;
my $dword;
# Option declarations
our ($opt_d, $opt_h, $opt_l, $opt_u);

# Process the command line parameters
getopts('dhlu');

# Debug option
print "\$opt_d:$opt_d\n" if (defined($opt_d));
# Help option
print "\$opt_h:$opt_h\n" if (defined($opt_d) && defined($opt_h));
# Localtime option
print "\$opt_l:$opt_l\n" if (defined($opt_d) && defined($opt_l));
# Unicode option
print "\$opt_u:$opt_u\n" if (defined($opt_d) && defined($opt_u));

# Obtain the file name
$evt_file = shift(@ARGV);
print "Event file: $evt_file\n" if (defined($opt_d) && defined($evt_file));

if ($opt_h) {
    &usage();
}

# Open the selected file in binary mode.
open(FILE, $evt_file) or die "ERR: Couldn't open file $evt_file: $!";
binmode(FILE);

do {
    &next_signature($opt_d, *FILE);

    # The following condition should never be met, because:
    # - A record has been found and the file has been rewinded 8 bytes
    # - Or EOF was reached and next signature ended the program
    die("End of file reached: Incomplete record.\n")
 if (read(FILE, $dword, 4) <= 0);
    # Obtain the length of this record
    $length = unpack "L", $dword;
    # A record should be at least 56 bytes long
    if ($length > 51) {
 # Read the record and process it
 &process_record($opt_d, *FILE, $length, $opt_l, $opt_u);
    } else {
 # Probably corrupted record
 print SDTERR "Record too short found and discarded! (Corrupted?)\n";
 if ($opt_d) {
     print "Record length was: $length\n";
 }
 # skip current signature to avoid infinite loop.
 seek(FILE, 4, 1);
    }
} while (!eof(FILE));
close FILE;

exit(0);

Enjoy!

Ethical Hacker Challenge: Prison Break - Breaking, Entering & Decoding

DISCLAIMERSince our last post half a year ago, we have not forgotten, RaDaJo readers! No excuses :( It has been very hard for us to find time to publish new posts, as we have been involved in three very large projects, plus a few extra security services, during the first half of the year. We hope one of the projects becomes a relevant step towards the security of embedded devices and service provider infrastructures. It is just the beginning... "That's one small step for a man, a giant leap for mankind." The other two projects have been large, really enjoyable, and interesting penetration tests. Meanwhile, we had to deal with some presentations, training events, collaborations, new discovered vulns, ISC shifts, and small pen-tests. In the background, we have also found time to work out things like the one we present you in this new and long time awaited RaDaJo post...

A few months back, by the time I sent my submission to the "Santa Claus is Hacking to Town" challenge, Ed Skoudis gave me the opportunity to write one of his famous and always interesting security challenges. I couldn't say no ;)

As a result, a new challenge has been published on The Ethical Hacker Network. The challenge is adapted from the Prison Break TV show, and it has two main goals. On the one hand, the offensive one, improve your penetration testing skills, tool set, and force you to solve various real-world scenarios I have found along my pen-testing activities. On the other hand, the defensive one, make you think like an attacker, analyze some of the tools and offensive capabilities available today, and figure out ways to put in place countermeasures to mitigate this type of attacks.


I hope you enjoy thes new "Prison Break - Breaking, Entering & Decoding" security challenge during summer. It is ready right before BlackHat & Defcon, so you can try to solve it after the common depression following these two conferences. Go to the Ethical Hacker Network website, digest the challenge and... participate! (Submit your answer by August 31, 2009)
--
Raul Siles
www.raulsiles.com

Prison Break image obtained from “http://www.shockya.com/news/wp-content/uploads/prison_break_ver4_poster.jpg”.

Penetration Testing Challenge: Santa Claus is Hacking to Town

This past holidays Ed Skoudis published one of his always interesting, amusing, and educational thematic security challenges at the Ethical Hacker Network: "Santa Claus is Hacking to Town". The last one I participated in was in mid-late 2006, although I was a huge fan of them since 2003. This time the challenge was penetration testing focused, rather than incident handling based, so I decided to play and enjoy it. Honestly, from all the security services I offer, penetration testing has taken an increasingly significant percentage of my time during the last years. There is a clear need in the industry for more pen-testers.

I suggest you to read the challenge wording and try to solve it before reading the official solution and answers. You can get some hints by reading the first paper referenced at the end of this post (Ed told me he published them there on purpose to help people out with the challenge), although it is funniest to solve it from scratch :)

You can find my submission here. I also generated (out of the contest) a second version of the paper that includes the challenge text, my official solution, and an appendix with a simpler and direct solution to the challenge, plus the reasons why it was not included as my final submission. Definitely, I could have been stealthier by providing the "-n" option to all the netcat relay instances in order to disable DNS resolution.

Complementary, my Inguardian's friends recently released two penetration testing papers you might be interested in: "Secrets of America's Top Pentesters" (Ed) and "Vista Wireless Power Tools for the Penetration Tester" (Joshua). I strongly recommend both!
--
Raul Siles
www.raulsiles.com

NMAP Trivia ANSWERS: Mastering Network Mapping and Scanning

Three weeks ago I published the NMAP Trivia challenge. Thanks to all ISC readers that submitted their responses! A special mention goes to the winning entry from Jason DePriest, an extensive and elaborated submission, available here. Congratulations! The prize (technical book) is on his way! ;)

Jon Kibler provided an in-progress nmap idea for a new features, a scan proxy engine equivalent to the FTP bounce scan to scan through HTTP or SOCKS.

Now... it is time for the answers:

1. What are the default target ports used by the current nmap version (4.76)? How can you change the target ports list? What (nmap) options can be used to speed up scans by reducing the number of target ports and still check (potentially) the most relevant ones? How can you force nmap to check all target ports?

Fyodor performed a thorough port scan research this last summer to identify the most common ports available on the Internet [1]. The current nmap version scans by default the 1000 most popular ports. The popularity of each port is coded inside the nmap-services configuration file (by default under /usr/local/share/nmap).

...
unknown 4/tcp 0.000477
rje 5/udp 0.000593 # Remote Job Entry
unknown 6/tcp 0.000502
echo 7/tcp 0.004855
echo 7/udp 0.024679
unknown 8/tcp 0.000013
...

Nmap provides an option for quick scans, "-F". It scans the 100 most popular ports, reducing the default load in one order of magnitude. Additionally, you can decide how many popular ports you want to scan through the "--top-ports N" option, where "N" is the top number of ports.

# ./nmap -F scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 10:44 GMT
Interesting ports on scanme.nmap.org (64.13.134.52):
Not shown: 95 filtered ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp closed smtp
53/tcp open domain
80/tcp open http
113/tcp closed auth

Nmap done: 1 IP address (1 host up) scanned in 4.04 seconds

# ./nmap --top-ports 5 scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 10:44 GMT
Interesting ports on scanme.nmap.org (64.13.134.52):
PORT STATE SERVICE
21/tcp filtered ftp
22/tcp open ssh
23/tcp filtered telnet
80/tcp open http
443/tcp filtered https

Nmap done: 1 IP address (1 host up) scanned in 8.56 seconds

Finally, nmap allows you to define the specific set of ports to scan through the "-p" option, as in "-pT:22,80,443,U:53,69,514". All ports, including port 0, can be scanned by providing the "-p0-" option, meaning from 0 till the end of the range, that is, port 65535. You need to specify if they are TCP or UDP ports, or both ("-sSU").

# nmap -p0- scanme.nmap.org

[1] http://insecure.org/presentations/BHDC08/

2. How can you force nmap to scan a specific list of 200 target ports, only relevant to you?

If you don't want to scan the most popular ports, you can tell nmap what particular list of ports to scan by specifying them with the "-p" option, one by one or in ranges, like in "-p 20-23,25,80,443". Because this can be too tedious for long lists of ports, the recommended way is to copy and edit the "nmap-services" file and create a custom version containing your list of interesting ports. The new custom file can be referenced using the "--servicedb" (for individual files) or "--datadir" (for the configuration files directory) options, as in:

# nmap --datadir ./myconfig scanme.nmap.org

If your custom file contains more than 200 target services, then you can use the "--top-ports 200" option again. The specific file and directory search order followed by nmap is detailed on page 370 of the nmap book: http://nmap.org/book/data-files-replacing-data-files.html.

3. What is the default port used by nmap for UDP ping discovery (-PU)? Why? If you don't know it from the top of your head ;), how can you easily identify this port without using other tools (such as a sniffer) or inspecting nmap's source code?

By default, nmap sends an empty UDP packet to port UDP/31338 for the UDP ping discovery method ("-PU"). The reason is that there is a high chance this random high port is closed. This is the preferred state expected by nmap trying to elicit an ICMP port unreachable packet in return and, as a result, identify the existence of a new host. The port number is defined in nmap.h, specifically in the DEFAULT_UDP_PROBE_PORT_SPEC constant. Did you notice it is 31337 plus 1, the elite port (31337 in haxor speech) plus one.

Currently, nmap provides the "--packet-trace" option to gather detailed information about the network traffic and individual packets sent and received during its operations. Effectively, this option acts as a built in sniffer, very useful to get details about what nmap is doing on the backstage.

# nmap -PU --packet-trace scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 10:58 GMT
SENT (0.6580s) UDP 192.168.166.166:59676 > 64.13.134.52:31338 ttl=58 id=45958 iplen=28
SENT (1.6560s) UDP 192.168.166.166:59677 > 64.13.134.52:31338 ttl=59 id=46599 iplen=28
Note: Host seems down. If it is really up, but blocking our ping probes, try -PN
Nmap done: 1 IP address (0 hosts up) scanned in 2.68 seconds

4. When nmap is run, sometimes it is difficult to know what is going on the backstage. What two (nmap) options allow you to gather detailed but not overwhelming information about nmap's port scanning operations? What other extra (nmap) options are available for ultra detailed information?

The first of the options has been mentioned and used on the previous question, "--packet-trace". It allows to get a tcpdump-like output about packets sent and received. Additionally, nmap provides the "--reason" option to display the reason why a port has been clasiffied on an specific state: open, closed, filtered, etc.

# nmap -F -sSU --reason scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 11:00 GMT
Interesting ports on scanme.nmap.org (64.13.134.52):
Not shown: 99 open|filtered ports, 96 filtered ports
Reason: 194 no-responses and 1 admin-prohibited
PORT STATE SERVICE REASON
22/tcp open ssh syn-ack
25/tcp closed smtp reset
53/tcp open domain syn-ack
80/tcp open http syn-ack
113/tcp closed auth reset

Nmap done: 1 IP address (1 host up) scanned in 7.95 seconds

# nmap -F -sU --reason scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 11:02 GMT
Interesting ports on scanme.nmap.org (64.13.134.52):
Not shown: 99 open|filtered ports
Reason: 99 no-responses
PORT STATE SERVICE REASON
520/udp filtered route admin-prohibited from 192.168.15.1

Nmap done: 1 IP address (1 host up) scanned in 15.90 seconds

For those interested on gathering as much information as possible about nmap's operations, the "-v" verbosity option, or the "-dN" debugging option are available. These options specify nmap to be verbose (multiple verbosity levels are allowed), or the nmap debug level for troubleshooting purposes, where N can have a value between 1 and 9. Be careful when you use it! Try it and be ready for a Matrix-like output 8-)

# nmap -p80 -sS -v scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 11:07 GMT
Initiating Ping Scan at 11:07
Scanning 64.13.134.52 [2 ports]
Completed Ping Scan at 11:07, 0.24s elapsed (1 total hosts)
Initiating Parallel DNS resolution of 1 host. at 11:07
Completed Parallel DNS resolution of 1 host. at 11:07, 0.24s elapsed
Initiating SYN Stealth Scan at 11:07
Scanning scanme.nmap.org (64.13.134.52) [1 port]
Discovered open port 80/tcp on 64.13.134.52
Completed SYN Stealth Scan at 11:07, 0.26s elapsed (1 total ports)
Host scanme.nmap.org (64.13.134.52) appears to be up ... good.
Interesting ports on scanme.nmap.org (64.13.134.52):
PORT STATE SERVICE
80/tcp open http

Read data files from: .
Nmap done: 1 IP address (1 host up) scanned in 6.13 seconds
Raw packets sent: 3 (112B) | Rcvd: 2 (72B)


# nmap -p80 -sS -d1 scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 11:08 GMT
--------------- Timing report ---------------
...
---------------------------------------------
Initiating Ping Scan at 11:08
Scanning 64.13.134.52 [2 ports]
...
Nmap done: 1 IP address (1 host up) scanned in 0.74 seconds
Raw packets sent: 3 (112B) | Rcvd: 2 (72B)

Try it by your own! ;)

5. What are the preferred (nmap) options to run a stealthy TCP port scan? Particularly, try to avoid detection from someone running a sniffer near the person running nmap and focus on the extra actions performed by the tool (assuming the packets required to complete the port scan are not detected)?

Most current network IDS can detect the default packets generated by nmap when port scanning a target. We are assuming here these cannot be detected, so a stealthier scan can be launched by using the "-n" option (not used in any of the Nmap Trivia examples), that is, disable all reverse DNS resolution at the nmap level. Most Unix-based security tools provide this same option for the same purpose.

# nmap -F -n scanme.nmap.org

However, this way you lose the sometimes valuable DNS information. You can use the "--dns-servers" option to indicate the DNS recursive servers to use as DNS proxies when analyzing the target IP address.
More stealthier details on answer number 12.

6. Why port number 49152 is relevant to nmap?

Port 49152 is the first of the ephemeral ports for dynamic usage based on IANA. However, the port assignment depends on the implementation of your tools or operating system. See http://www.iana.org/assignments/port-numbers:
- The Well Known Ports are those from 0 through 1023
- The Registered Ports are those from 1024 through 49151
- The Dynamic and/or Private Ports are those from 49152 through 65535

7. What is the only nmap TCP scan type that classifies the target ports as "unfiltered"? Why? What additional nmap scan type can be used to discern if those ports (previously identified as "unfiltered") are in an open or closed state?

The only nmap scan type that can show a port in the "unfiltered" state is the TCP ACK scan, "-sA" option. The reason is because this scan cannot differentiate between an open and closed port, as a target hosts (if unfiltered) will always reply with a RST packet. This is the standard behaviour for a closed port, and is also standar for an open port for which there is not a previously established connection to map the ACK packet to. Therefore, nmap's ACK scan cannot be considered a port scan, as it cannot differentiate between port states, but a host discovery scan.

The TCP Window scan, "-sW" option, is similar to the TCP ACK scan, but it can differentiate between open and closed ports is some scenarios.

8. When (and it what nmap version) the default state for a non-responsive UDP port was changed on nmap (from "open" to "open|filtered")? Why?

The default state for a non-responsive UDP port was changed (from "open" to "open|filtered") on nmap version v3.70 in 2004. The reason was accurancy, as extensive use of filtering devices by that time made filtered UDP ports always appear as open in previous nmap versions.

9. What is the default scan type used by nmap when none is specified, as in "nmap -T4 scanme.nmap.org"? Is this always the default scan method? If not, what other scan method does nmap default to, under what conditions, and why?

The current nmap version performs a TCP SYN scan ("-sS" option) by default when no scan type is specified. However, this is only the default behavior when nmap is launched as a privileged user (eg. root in Linux). The TCP connect scan, "-sT" option (connect() syscall), is used by default with non-privileged users as these cannot send raw packets (used by the SYN scan) or if there are IPv6 targets.

# ./nmap -PN -p80,81 --packet-trace scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 11:22 GMT
...
SENT (0.3730s) TCP 192.168.166.166:56464 > 64.13.134.52:80 S ttl=50 \
id=8102 iplen=44 seq=1698869517 win=3072
SENT (0.3740s) TCP 192.168.166.166:56464 > 64.13.134.52:81 S ttl=43 \
id=48226 iplen=44 seq=1698869517 win=4096
RCVD (0.6120s) TCP 64.13.134.52:80 > 192.168.166.166:56464 SA ttl=48 \
id=0 iplen=44 seq=2849983456 win=5840 ack=1698869518
RCVD (1.9570s) TCP 64.13.134.52:80 > 192.168.166.166:40972 SA ttl=48 \
id=0 iplen=44 seq=2805666242 win=5840 ack=2103880733
SENT (2.5730s) TCP 192.168.166.166:56465 > 64.13.134.52:81 S ttl=55 \
id=14744 iplen=44 seq=1698935052 win=4096
Interesting ports on scanme.nmap.org (64.13.134.52):
PORT STATE SERVICE
80/tcp open http
81/tcp filtered hosts2-ns

Nmap done: 1 IP address (1 host up) scanned in 3.79 seconds

$ ./nmap -PN -p80,81 --packet-trace scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 11:25 GMT
...
CONN (0.1290s) TCP localhost > 64.13.134.52:80 => Operation now in progress
CONN (0.1290s) TCP localhost > 64.13.134.52:81 => Operation now in progress
CONN (2.3510s) TCP localhost > 64.13.134.52:81 => Operation now in progress
Interesting ports on scanme.nmap.org (64.13.134.52):
PORT STATE SERVICE
80/tcp open http
81/tcp filtered hosts2-ns

Nmap done: 1 IP address (1 host up) scanned in 3.57 seconds

10. What nmap features (can make or) make use of nmap's raw packet capabilities? What nmap features rely on the OS TCP/IP stack instead?

Nmap makes use of the raw packet capabilities by default, "--send-eth" option, as demonstrated in the previous question for some features, such as TCP and UDP port scans launched by privileged users (except for the connect scan and the FTP bounce scan), or fragmentation probes. Other features like the Nmap Scripting Engine and version detection relay on the OS TCP/IP stack.

11. Nmap's performance has been sometimes criticized versus other network scanners. What (nmap) options can you use to convert nmap into a faster, stateless scanner for high performance but less accurate results?

If the congestion controls and packet loss detection algorithms are omitted, a scanner will run faster. Nmap can achieve a similar behaviour as stateless scanners, no code to track and retransmit probes, using the following options:

# ./nmap --min-rate 1000 --max-retries 0 ...

These indicate nmap to send at least 1000 packets per second (if your system or wire can) and disable retransmission of timed-out probes. However, take into account the impact this might have in the accurancy of the results.

# ./nmap -PN -n --min-rate 1000 --max-retries 0 -F scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 12:08 GMT
Warning: Giving up on port early because retransmission cap hit.
Interesting ports on 64.13.134.52:
Not shown: 95 filtered ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp closed smtp
53/tcp open domain
80/tcp open http
113/tcp closed auth

Nmap done: 1 IP address (1 host up) scanned in 1.06 seconds

12. What relevant nmap feature does not allow an attacker to use the decoy functionality (-D) and might reveal his real IP address?

Apart from the previously mentioned "-n" option to run stealthier scans and avoid IDS detection, there are other related options, such as "--data-length", to change the default empty packet used for some probes, "--ttl" to modify the TTL on the sent packets, timing options ("-T"), "--randomize-hosts" to change the order the target hosts are scanned, or "-D" to launch a decoy scan (simulate the scan is coming from multiple hosts).

Decoys are used in the ping discovery, port scanning, and remote OS detection phases. However, this feature does not apply when DNS queries or service version detection ("-sV" or "-A") are used, being the source IP address disclosed.

13. What are the (nmap) options you can use to identify all the steps followed by nmap to fingerprint and identify the Web server version running on scanme.nmap.org?

# ./nmap -sSV -p80 --version-trace scanme.nmap.org

Starting Nmap 4.76 ( http://nmap.org ) at 2009-01-21 12:17 GMT
...
SCRIPT ENGINE: Initiating script scanning.
SCRIPT ENGINE: Script scanning scanme.nmap.org (64.13.134.52).
SCRIPT ENGINE: Initialized 4 rules
SCRIPT ENGINE: Matching rules.
SCRIPT ENGINE: Running scripts.
SCRIPT ENGINE: Script scanning completed.
Scanned at 2009-01-21 12:17:57 GMT for 8s
Interesting ports on scanme.nmap.org (64.13.134.52):
PORT STATE SERVICE VERSION
80/tcp open http Apache httpd 2.2.2 ((Fedora))
Final times for host: srtt: 238764 rttvar: 179294 to: 955940

Read from .: nmap-rpc nmap-service-probes nmap-services.
Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 8.17 seconds

The "-sSV" option allows you to focus on a TCP scan type (SYN scan in this case, "-sS"), and fingerprint the service ("-sV"). In order to just target the web server (supposing HTTP (TCP/80) is the target port, and not HTTPS (TCP/443)), the "-p80" option must be used.

The "--version-trace" option is similar to the "--packet-trace" option, but instead of dumping the network traffic, it dumps all the actions or steps performed by nmap during the execution of the service fingerprinting modules. Additionally, other debug options ("-dN") can be added to gather further details.

14. As an attacker, what port number would you select to hide a listening service backdoor trying to avoid an accurate detection by nmap's default aggressive fingerprinting tests? Would it be TCP or UDP? Why? What additional (nmap) options do you need to specify as a defender to fingerprint the hidden service backdoor?

If a port in the range of TCP/9100-9107 is selected for a backdoor, due to the fact these are common ports for printer services, nmap won`t fingerprint the service. These ports are excluded by default on the service fingerprinting tests ("-sV") or aggressive scan options ("-A") trying to save the planet, trees and forests specifically, by not making printers dump dozens of pages full of nmap probes and garbage as a result of the stimulous received from the scan.

If you want to enable service fingerprinting on all ports, there are two options. The "--allports" option can be specified, as in "nmap -A --allports", or the nmap-service-probes file can be modified to enable these ports by removing the "Exclude" directive.

15. What is the language used to write NSE scripts, and what two other famous open-source security tools/projects currently use the same language?

Nmap uses the LUA (www.lua.org) programming language. LUA (pronounced LOO-ah) means "Moon" in Portuguese, or "Luna" in Spanish ;) Other famous open-source security tools, like Wireshark and Snort use LUA to extend their capabilities.

16. What Linux/Windows command can you use to identify the list of NSE scripts that belong to the "discovery" category and will execute when this set of scripts is selected with the "--script discovery" nmap option?

By default, NSE scripts are available under the "scripts" directory (however, nmap searched in other locations too: --datadir, $NAMPDIR, etc), with the ".nse" file extension. All NSE scripts belong to one or more categories, define inside the script, and indexed by the scripts/script.db database (if updated through the "--script-updatedb" option).

Therefore a couple of options to search for discovery scripts in Linux are:

# grep discovery scripts/*.nse
scripts/ASN.nse:categories = {"discovery", "external"}
scripts/HTTP_open_proxy.nse:categories = {"default", "discovery", "external", "intrusive"}
scripts/HTTPtrace.nse:categories = {"discovery"}
...

# grep discovery scripts/script.db
Entry{ category = "discovery", filename = "HTTPtrace.nse" }
Entry{ category = "discovery", filename = "rpcinfo.nse" }
Entry{ category = "discovery", filename = "SMTPcommands.nse" }
...

You can perform a similar search in Windows using the built-in search capabilities (searching by "A word or phrase in the file" to look inside the directory) or the find or findstr commands (to search within a file or set of files).

17. How can you know the specific arguments accepted by a specific NSE script, such as those accepted by the whois.nse script?

In order to identify the arguments that can be passed through the "--script-args" option to a NSE script, eg. whois.nse, check the documentation or code within the script file. If it is properly documented, search by "-- @args" to go to the arguments documentation section.

Finally, a couple of extra questions for the real nmap-lovers:

1. How can you get in real-time the open ports discoverd by nmap before the final report is displayed?
2. What happens when you run nmap in verbose mode on September 1?

That's all folks! Happy nmap discovery and scanning!

NOTE: This challenge has been published on the Internet Storm Center (ISC) diary too.

--
Raul Siles
www.raulsiles.com