Looking for hacking activity in Apache Logs

This is my first post with Ghost and since it contains code snippets and command line goodies I thought it would be a good test for Ghost’s markdown language. Let’s see how it goes.

The sheer number of bad people on the planet mean that there’s a really good chance your website has at least been probed to see if it is a good attack platform. It may also mean that your website has already been compromised and is doing bad things for some other person as we speak. Some people I talk to say things like “well, if I get hacked, I’ll deal with it then”. But that’s dumb. It’s dumb because when someone compromises your website, they’re not going to put a big banner on it letting you know. It may be days, weeks or months before you notice.

Without trying to sort your way through the myriad of ‘security services’, what can a lowly sysadmin do to protect herself? Access logs are a treasure trove of information and are an often overlooked resource. A fun activity is to take a gander through your Apache access logs now and again to see what people are hitting.

First, check out the referrers. Where are people coming from?

So far so good. Most of the requests are coming from links on my own site. I don’t know why buttons-for-website.com would have a link to me, but there’s more to look at before we can start drilling down. For example, what pages are these guys looking for and what IP addresses are they coming from?

Probes generally result in 404 response codes. This is because the bad guy is hitting a specific URL in order to determine if you’re running a piece of software, or if you are running a specific version of some software. So check out any 404s and what the request was for:

The robots.txt file is not a smoking gun. All search engines are supposed to look for it. However, the random request for an xmlrpc.php end point is curious. That is the xml endpoint for a WordPress blog and this domain has never run an WordPress installation. This smells like someone sniffing around. Let’s look more closely at that one.

A Russian IP. There is a ton of really bad traffic coming out of the eastern block these days so a probe for a non-existent URL from a Russian IP raises eyebrows.

It’s also worth checking out all the non-404’s in the log for two reasons:

  • If this WAS a WordPress installation, I would not see the xmlrpc.php file 404’ing. That means someone could be exploiting my xml endpoint under my radar.
  • Bad guy may have successfully placed a bad file on your server and is hitting it to do her nefarious bidding. Those also would likely be returning non-404 responses.

There’s not enough in this log to worry me, but if there were more suspicious entries, I’d start looking at how much traffic this IP was using to get a sense if he was successfully exploiting the site and then block it.

Checks like this are not very in-depth, but they take about 5 minutes to do and can really shine a light on anomalies worth investigating further.