OWASP China Research Group

To better facilitate the activities of OWASP in China for consistent and perpetual continuity, OWASP China has formed regional groups mainly tasked to support the regional sharing and discussion. We welcome you to recommend an individual to take the lead. OWASP China Research Group currently aims to build upon and go into the depths of the foundation laid out by the OWASP Foundation, plus translation of the OWASP resources ectera. There will be activities such as training in different regions. OWASP China QQ Discussion Group 78238096

(My translation above)

I hope to improve China’s internet security. I succeeded Frank and Rip on the last iteration of this project, and that is why my December has been busy all along, and took much of my time.

Thanks a lot to the people below, and especially the many Microsoft people who worked so hard even during Christmas to produce this testing guide. Sorted from last name (Mandarin) :

• Aaron (DBAPPSECURITY)
• Joanne Cheng (Microsoft)
• Frank Fan (DBAPPSECURITY)
• Karin He (Microsoft)
• Adams Li (Microsoft)
• RIP (OWASP China Chair)
• Will Shen (Microsoft)
• Chao Wang (Microsoft)
• Wei Wei (Microsoft)
• Pak Ming Cheung (Microsoft)
• Eric Chio (Microsoft)

Hope that readers of the guide will benefit much from it!

For the whole December and some January, I’ve been working for OWASP China on some projects – thus taking my full attention. And I have been busy on picking up some bits of life and my side project – yes! Working on it! It’s coming in this January!

The 2009 is a fantastic year! I am aiming well for 2010 and will aim to advance fully into my interests. More to that next time… meanwhile, stay tooned. =)

You might have noticed that I used md5 as filenames in the previous (old!) post. In most cases, it is fine.

However, what if the malware depends on a file called hgz.dll? You can calculate hgz.dll as md5, then find the filename out, now put that in the VM again – fine. But you see it is a troublesome process… that you can’t easily automate. There are other cases… of course.

Well, you get the point!

For this problem, we assume all the files are malicious as honeypots do not collect innocent software.

One way is to use virus scanners to scan and classify the files. After a scan, group together all the files that are detected as “Conficker.B” for example. As Conficker family is quite prevalent, such duplication identification can save a lot of time and effort. This way, just analyzing one or two of them is sufficient. However, the drawback is that all the undetected samples will be left as a big group which you must analyze one-by-one.

Extract of a clamscan result…

/tmp/4c71b97435a24ffb8fd7fedd1b1790e1: OK
/tmp/82dd3a3d386d4ea09870dcee4a75a531: OK
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin: OK
/tmp/24bd1722b994f7daa193458348108bfc.bin: OK
/tmp/39960c5ff1922466ded71a4a2799c295: Trojan.VanBot-366 FOUND
/tmp/33f5f14c33bf2f71556204705407a885: W32.Virut-54 FOUND
/tmp/880ce6df69aaeb1d3c57e756f53dd158.bin: Trojan.Delf-911 FOUND
/tmp/7e0ce66bb299370010016f4522152969: Trojan.VanBot-366 FOUND
/tmp/4f2d9f8129e7d7fd9b37f700aacdc9aa.bin: Trojan.Hupigon-25647 FOUND
/tmp/5b69ff6f331ece36558516f66306f969: Trojan.Small-4287 FOUND
/tmp/078aedb8630339487cf39d028b0156bd.bin: OK
/tmp/417bdef0688996a845701da9dcf1b145: Trojan.VanBot-366 FOUND
/tmp/eda3b7766c23dfffc0b85d0ba546b0c1: W32.Virut-54 FOUND
/tmp/86f22ff53382dbb54e2c22560a3db373: Trojan.VanBot-366 FOUND
/tmp/a4a41d2122c4d3552e3d59315f42d4e3: W32.Virut-54 FOUND

In the above, without signatures, how can you tell if 4c71b97435a24ffb8fd7fedd1b1790e1 and 82dd3a3d386d4ea09870dcee4a75a531 is not the same family? How can you tell which malware is unique? You have to analyze them. Now scale the problem to perhaps 600, for yourself only.

The other way is to use ssdeep, a fuzzy hashing tool. It is used to match inputs that are similar, perhaps only some bytes and length. It will produce a hash signature like md5 but unlike md5, a single change of byte will not create a wildly different signature. The concept of ssdeep is to chop the files into many sections, and calculate the hash for each section.

Below I take a sample of an exe file (“file1.exe”). I copied the file and concatenates a byte after it (“file2.exe”), and computes the md5 sum of the two files.

$ cp file1.exe file2.exe
$ echo 1 >> file2.exe

$ md5sum file1.exe file2.exe
72bdd3bd37a0b5d1dd5f1be80cb29639  file1.exe
a626b78fa6ba13fdd9cfddb9f55ee7c6  file2.exe

Just a difference in one byte, and the md5 hash is completely different. Let us do the ssdeep sum of the two files.

(broken into lines for clarity)

$ ssdeep -b file1.exe file2.exe
ssdeep,1.0–blocksize:hash:hash,filename
768:my+qxlsz7yiV0+7YUaFhLFAtVI0xbM
LvzEg1B1Ki8nJ78:R+qxlsHvGhLFyI0l8tC5J78,”file1.exe”
768:my+qxlsz7yiV0+7YUaFhLFAtVI0xbM
LvzEg1B1Ki8nJ7V:R+qxlsHvGhLFyI0l8tC5J7V,”file2.exe”

Separated by colon, the first (768) is the blocksize, then two ssdeep hashes (my+qxlsz7yiV0+7YUaFhLFAtVI0xbMLvzEg1B1Ki8nJ7V and R+qxlsHvGhLFyI0l8tC5J7V) , then the last is the file path name (“file2.exe”). The main point are the two hashes – the signatures of the file. Both file hashes of the two files are really alike except for the last byte ( “8″ vs “V” ).

If you have a large number of unidentified malware, antivirus scanners will not help to classify, but ssdeep can try. Below is extracted output of file matching with ssdeep. Each file name is the md5 of the file itself.

$ ssdeep -dr .

…
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin matches /tmp/fa7c91b738e763eccf69676bd393925e.bin (88)
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin matches /tmp/ae142ce3b35cc04f5648a0c17c37ea30.bin (82)
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin matches /tmp/794b74fc4e833d245eb005e078dc21da.bin (82)
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin matches /tmp/46fb9678675df8dc83d38761a76c7950.bin (99)
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin matches /tmp/f412d41aacb4b16ded7b158b89fd3552.bin (90)
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin matches /tmp/4bfba885ed3dc4ba800446df49051af0.bin (82)
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin matches /tmp/13776c2b604290906305a56c4e7c61e5.bin (99)
/tmp/72bdd3bd37a0b5d1dd5f1be80cb29639.bin matches /tmp/5a8424f4e1504b5823ca8742e2b1ce8d.bin (82)
…

In the above, all of them are undetected malware and gives wildly different md5 signature. Yet, ssdeep can relate them. For malware that does not match any other files, it can be assumed to be a unique malware in your collection, and you should pay more attention to it. Moreover, even packed executables (tested on UPX) still can be matched since packers are just compressors – the similar code will be compressed into a similar binary pattern.

There are a few culprits. First, remembering that ssdeep just does mini-hashes, if some bytes vary a little throughout the file ( by some obfuscation, etc, every 1 byte change at 100 byte intervals, i.e. no-ops) will cause the ssdeep to fail to identify matches. Then, for botnets credentials identification, similar files could contain very different login credentials and wrongly discarded due to highly similar binary structure. However, you can analyze the access control logic through such duplicated samples, then you can generalize the login credentials.

With ssdeep, you can now group duplicated undetected malware into groups for more efficient analysis.

===

ssdeep – http://ssdeep.sourceforge.net/

UPX – http://upx.sourceforge.net/

Goal

The goal of running a spampot (honeypot which only care about spam) is to collect spam and analysis the trend of them, hopefully we can find some interesting techniques that spammers/ hackers use in junk and phishing emails.

Approach
So far, there are at least two types of spampot hosting method that I know. The names of them are designed by me, if there are formal names for them, please let me know.

Open Relay Spampot: This kind of honeypot is running as an open mail relay server. In case you are not familiar with, open relay means users can send message through the server anonymously.

Close Relay Spampot: The spampot is running as a close mail relay server. To expose the server to spammers, you need to have your own domain binding to this server with email address(es) exposing to spammers/ hackers. For example, we can have onhacks.org binding to a spampot and spam@onhacks.org is one of the email address we want to expose to spammers. However, about the methods to increase the exposure of an email addresses is out of scope, we can discuss more on it later.

In my setup, I decided to run spampot as open mail relay server.

Setup
I have VirtualBox installed on top of Windows 7. I am using Ubuntu as the guest OS, this is because it seems the implementation was done in *nix system. Since port 25 is the default port for SMTP service, we need to forward packets from host (Win7) to guest (Ubuntu) so that the spampot in guest OS can react to incoming connection at host port 25.

(Assuming that you are using NAT for VirtualBox)
To enable port forwarding, you need to set the HostPort 25 forwarding to GuestPort 25. For more detail around port forwarding in VirtualBox, please refer to this article.

However, you will soon discover that it is not possible to perform port forwarding if the port is reserved (< 1024). This can easily be resolved by running VirtualBox with admin credential (ie. Run As Administrator).

The spampot.py requires Sendmail being installed in Linux. Since sendmail actually is a service listening to port 25, I will do the follow to switch to spampot.py:

sudo /etc/init.d/sendmail stop
sudo spampot.py 0.0.0.0

Surely you can set this automatically run when the system is started.

The last thing is to add a DNS record pointing to my machine. I have smtp.onhacks.org. pointing to it. Since it is still under experiment, the machine is running at home and IP is dynamic, I need to change it often.

Result
Currently, I got 0 message after running the spampot for few days. I have google around and looks like open relay spampot is not that popular anymore because many server admins aware that spammers were abusing open mail relay servers, they don’t allow open relay anymore. As a result, submitting spams to open relay servers is not efficient anymore.

I will continue running the spampot these days and see if we can get more spam through open relay honeypot. Afterward, I will work on close relay spampot.

Reference

1. Open mail relay – Wikipedia
2. spampot.py – written by Neale Pickett
3. Configure Port Forwarding to a VirtualBox Guest OS – Tombuntu
4. SpamPots Project – Cert.org
5. Brazilian Honeypots Alliance

After a deeper investigation, I discovered that one of the user was preparing questions on the machine, and generated a 12GB test data unexpectedly. Since this is a very old machine, it only has a 14GB hard disk for data storage and it already had 2GB data on it. This is kind of DoS attack since no one can submit sources to the judging system even though they can navigate to it.

Lesson learned: We should have restriction on storage usage of each user instead of unlimited.

Any other suggestion to prevent this happen again?

Log0 is quite busy lately.

This tool was not to be released when I first built it, however it becomes more useful. It has a lot of functions to include in the future such as adopting general botnet communication, and making it easier to use and automate, and even scriptable.

I will post the source code to the tool after updated with HELP document. Feel free to email me =)

BTW, hac.ka is my friend and the otherOnHacks teammate whom I mentioned during my final speech. He works on Email and DNS related items.