onHacks

Last night, I was waken by a call that a server was not working. This server is hosting an online judging system (similar to uva.onlinejudge.org, which has algorithmic problems that users can solve). I took a quick look at the compilation process and web pages, everything looked good except it always return “Compilation Error” no matter what was the content in source code (even a HelloWorld!). By manually compiled the source code, the compilation error message gave more detail information about the root cause…Not enough space to link the object files! When I did a “df”, it said that the data partition was used 100%!!

After a deeper investigation, I discovered that one of the user was preparing questions on the machine, and generated a 12GB test data unexpectedly. Since this is a very old machine, it only has a 14GB hard disk for data storage and it already had 2GB data on it. This is kind of DoS attack since no one can submit sources to the judging system even though they can navigate to it.

Lesson learned: We should have restriction on storage usage of each user instead of unlimited.

Any other suggestion to prevent this happen again?