menu

Category: Server Logs Explained

Server Logs Explained is a series that started in 2016 as a way to dissect and explain excerpts from real server logs on production servers. Posts in this series may also include configuration excerpts and how-to guides for configuring and securing your own server.

Updated Posted by Arnon Erba in Server Logs Explained on .

In today’s world, the exhaustion of IPv4 addresses and the slow adoption of IPv6 means that publicly routable IPv4 addresses are in high demand. It also means that when you spin up a cloud-based virtual private server using a service like Digital Ocean, Linode, or Amazon Web Services (AWS), you’ll almost certainly get an IPv4 address that was previously in use by someone else. In the worst case, your new IP address might be on some blacklists, but the most likely situation is that you’ll get some extra “background noise” in your server logs.

The Logs

167.114.0.63 - - [06/May/2018:03:10:08 +0000] "GET /0f0qa0a/captive_portal.html HTTP/1.1" 404 152 "-" "Go-http-client/1.1"
37.187.139.66 - - [06/May/2018:03:10:41 +0000] "GET /0f0qa0a/captive_portal.html HTTP/1.1" 404 152 "-" "Go-http-client/1.1"

These Nginx logs were pulled from a fresh virtual private server that I created with a new-to-me IPv4 address. If you’re curious, my original Server Logs Explained post contains a breakdown of the log format I’m using, but I’ll cover what these log excerpts mean in this post as well.

Essentially, two completely different IP addresses performed an HTTP GET request for the same resource, /0f0qa0a/captive_portal.html. Unable to provide this mysterious file, my server responded with 404 Not Found. This pair of log entries became much more interesting when I noticed that my server kept getting the same two requests from the same two remote IP addresses every few seconds.

Some Detective Work

First of all, these log entries are completely harmless. Anyone can request any random page from a web server, and the server should return a 404 response if the page does not exist. At this point, curiosity is the only reason to continue exploring the source of the two requests.

A cursory WHOIS lookup on both IPs reveals that they are owned by OVH Hosting, a French company that provides cloud-based hosting services. It’s similar to the other cloud hosting companies I mentioned at the beginning of this post. It isn’t a big leap to assume that both IPs belong to virtual servers hosted by OVH, so with that assumption in mind, let’s move forward.

Next, a reverse DNS (rDNS) lookup on each IP address yields the following interesting results:

167.114.0.63 maps to prometheus-nodes-ca-ovh.afdevops.com
37.187.139.66 maps to prometheus-nodes-eu-ovh.afdevops.com

Right away, there’s “ovh” in the hostnames for the two servers, which seems to confirm that OVH Hosting was a good guess. There’s also something else interesting about these results: 167.114.0.63 is a Canadian IP address, and it has “ca” in its hostname. On top of that, 37.187.139.66 is a French IP address, and it has “eu” in its hostname. It’s beginning to look like these two servers are part of some company’s public-facing server infrastructure.

Let’s look at the far left section of the hostnames, prometheus-nodes. There’s no law that dictates how you should name your servers, but it is pretty common to give them logical names that correspond to the software that runs on them. With that in mind, what is “prometheus”?

Prometheus is a real piece of software. Its GitHub page describes it as “a systems and service monitoring system”, which would explain the persistent requests in my server logs. Monitoring solutions work by constantly checking a service, evaluating the responses they receive, and notifying administrators if something looks wrong. It seems reasonable to conclude that some important service was being hosted by the previous owner of my IP address, and someone forgot to reconfigure their monitoring solution after decommissioning the server.

There’s still more we can learn from the reverse hostnames. A simple Google search for one of the hostnames turns up this GitHub issue from February, where a bunch of information is listed that confirms that the server runs Prometheus. On top of that, the user who opened the issue claims to work for a company called “AnchorFree”.

AnchorFree — could that have anything to do with the afdevops.com portion of the hostnames? Even though there’s no public-facing website associated with that domain, Googling for “afdevops.com” turns up this user profile on Docker Hub. Guess what’s listed in the bio for that user? Anchorfree, Inc.

There’s something else, too. A WHOIS lookup on “afdevops.com” reveals that it is registered under the real name of one of the co-founders of AnchorFree. That was easy. Too bad GDPR is killing WHOIS at the end of this month. (Ed: This was accurate when this post was drafted in 2018.)

Now that we know that both servers are owned by AnchorFree, let’s figure out what AnchorFree actually is. Wikipedia and AnchorFree’s actual website both confirm something interesting — AnchorFree is the parent company behind Hotspot Shield, a fairly well-known “freemium” VPN app.

The Conclusion

Nothing in this post is particularly revealing, but it’s interesting to consider what happens when IP addresses owned by cloud service providers get reused. If you incorporate cloud service provider IP addresses into your server infrastructure, it’s important to remember that those addresses may be recycled for other customers in the future. If you set up important services or access control lists based on IP addresses you don’t own, it’s possible to introduce problems that won’t become apparent until later. In cases like this, it might be as simple as having your monitoring solution check the wrong IP address for a few days.

On the other hand, consider what might happen if you decommissioned a server but forgot to remove its forward DNS entry. If a malicious actor gained control of its old IP address, they could set up a phishing website that would appear to be hosted on your domain. Change control and detailed documentation are important when it comes to using public cloud services.

Update (1/9/20)

A reader contacted me in October 2019 to let me know he had been hit by similar requests from two additional IP addresses:

167.114.65.142, or prometheus-longterm-ca-ovh.afdevops.com
167.114.119.75, or prometheus-dev-ca-ovh.afdevops.com

I also happened across a completely unrelated blog post from 2018 that documents the same strange requests for /0f0qa0a/captive_portal.html that were the basis for this post. In that case, the author chose to block the repetitive requests with his firewall.

Finally, on a whim, I tried accessing one of the IP addresses in a web browser and was taken to a login page asking for AnchorFree SSO credentials. By itself, that isn’t particularly surprising, since we already know that AnchorFree runs those servers. In any case, it would be great if AnchorFree did a little auditing of their public cloud infrastructure.

Updated Posted by Arnon Erba in Server Logs Explained on .

Some log entries are particularly bizarre, like the one we’ll be looking at today:

169.229.3.91 - - [26/Jun/2016:00:35:26 -0700] "\xD5H\xC5p*\xB7:\x8F\x91\x8A\xE1\xAA\xE0p\xD9\xF2[;\xAE\xE7c\xF7\x9C\xAB~\x98\xCB\xAD\xCB\xBE\xCE\xED\xAF\xEC\x8B\x19\xC6\x08D\xEB\xA8\x91\x1De\x10\x18 u\x01zHj\x00\x8D|\x15\x8B;\x98\x08RaSH" 400 166 "-" "-"

My server responded with 400 Bad Request, but the most interesting part is the giant $request portion, which doesn’t include any of the normal components you would expect in an HTTP request:

\xD5H\xC5p*\xB7:\x8F\x91\x8A\xE1\xAA\xE0p\xD9\xF2[;\xAE\xE7c\xF7\x9C\xAB~\x98\xCB\xAD\xCB\xBE\xCE\xED\xAF\xEC\x8B\x19\xC6\x08D\xEB\xA8\x91\x1De\x10\x18 u\x01zHj\x00\x8D|\x15\x8B;\x98\x08RaSH

Note: See my first Server Logs Explained post for an example of how to interpret the entire log entry.

Wait, What?

If your first thought was that this looks like 64 bytes of garbage, then you’d be exactly right. As it turns out, I wasn’t the first person to see one of these bizarre log entries. According to this Information Security StackExchange question and answer, this server request is from an Internet-wide research scan led by the Electrical Engineering and Computer Sciences (EECS) department at the University of California at Berkeley.

A reverse DNS lookup of the IP address led to an illuminating hostname, researchscan1.EECS.Berkeley.EDU. It turns out there’s actually several machines related to the project:

  • 169.229.3.90, or researchscan0.EECS.Berkeley.EDU
  • 169.229.3.91, or researchscan1.EECS.Berkeley.EDU
  • 169.229.3.92, or researchscan2.EECS.Berkeley.EDU
  • 169.229.3.93, or researchscan3.EECS.Berkeley.EDU
  • 169.229.3.94, or researchscan4.EECS.Berkeley.EDU

If you access any of those IP addresses or hostnames in a web browser, you’ll see a brief description of the project. However, according to this answer on StackExchange, the text on those webpages has changed over time, so the most concise explanation comes in the form of a quote from the project leaders at Berkeley that’s reproduced in that answer:

We are performing a measurement study of a particular phenomenon on the Internet. To accurately asses the behavior we’re performing a daily scan of the IPv4 space by sending a single benign packet to every IP on port 80 consisting of 64 random bytes of data. […] No, we are not attempting to gain unauthorized access. […] It’s simply randomly generated data that conforms to a certain set of criteria.

I contacted the project team a few months ago as well, but have not heard anything back. Given that the StackExchange answer and my log entry both date back to 2016, it’s possible that the research project is already over and is now just Internet history. Either way, it’s interesting to finally know what it is.

Updated Posted by Arnon Erba in Server Logs Explained on .

Since WordPress is such a popular website platform (a.k.a. Content Management System, or CMS), there’s a multitude of different ways to misconfigure it. Consequently, there’s also someone out on the Internet who will attempt to exploit every possible misconfiguration in your WordPress installation. This time on Server Logs Explained we’ll be looking at one such attempt, albeit a fairly basic one.

The Logs

WordPress sites get random requests all the time for pages that should normally be off-limits to the outside world, which brings us to today’s log excerpt:

203.0.113.42 - - [19/Jun/2016:07:37:57 -0400] "GET /wp-admin/post-new.php HTTP/1.1" 302 5 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"

This hardly borders on an attempted exploit, since it’s just a plain HTTP GET request for the post-new.php page. However, if you’re still getting familiar with WordPress’s file structure, it can be a good learning experience to examine some of the unexpected requests your server gets.

Call and Response

First, let’s see what happened next, since the server returned a 302 Found (moved temporarily) response code. A 302 response generally provides the client with an alternate URL it should try instead:

203.0.113.42 - - [19/Jun/2016:07:37:57 -0400] "GET /wp-login.php?redirect_to=http%3A%2F%2Fwww.example.com%2Fwp-admin%2Fpost-new.php&reauth=1 HTTP/1.1" 200 1307 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"

In plain English, the server politely asked the client to try wp-login.php instead, and the client followed the redirect.

This is exactly what’s supposed to happen, because post-new.php is the admin page that allows a site author to create a new post. Like the other admin pages in the /wp-admin/ directory, it should only be accessible to users who are logged in and have the permission to create new posts. If an unauthenticated user tries to access an admin page, WordPress is supposed to redirect them to the login page so that they can authenticate (or not).

Bonus Section: A Little Information About Query Strings

The most interesting part of this exchange is the request to wp-login.php, so let’s examine that. Stripping away the extra information from the second log excerpt leaves us with the request the client made to wp-login.php:

GET /wp-login.php?redirect_to=http%3A%2F%2Fwww.example.com%2Fwp-admin%2Fpost-new.php&reauth=1 HTTP/1.1

The first part, GET, is the HTTP method used (as opposed to POST, etc.). Check out Mozilla’s HTTP request methods documentation if you are curious about other methods defined in the HTTP standard. The last part, HTTP/1.1, is the version of HTTP that was used between the client and the server, so we can skip that as well.

The middle part is the request the client made to wp-login.php, and it includes an interesting query string. The query string begins with ? and looks like this:

?redirect_to=http%3A%2F%2Fwww.example.com%2Fwp-admin%2Fpost-new.php&reauth=1

Query strings are one way to allow data to be passed to a webpage. This is useful if your webpage is coded in a dynamic language, like PHP, and can accept input. This particular query string contains two pieces of data, separated by the & character:

  1. redirect_to, a string, and
  2. reauth, a value.

The data contained in the request is highlighted in the following example:

?redirect_to=http%3A%2F%2Fwww.example.com%2Fwp-admin%2Fpost-new.php&reauth=1

Let’s look at each piece of data individually:

redirect_to

The URL in redirect_to is where wp-login.php will send the client to after it has successfully authenticated. If you examine the WordPress source code, you can locate the section in the wp-login.php that handles the redirection logic.

If we clean up the URL by replacing the escaped characters with their human-readable counterparts, we can see what the full URL looks like:

http://www.example.com/wp-admin/post-new.php

In other words, after successfully authenticating, the client would have been redirected to post-new.php, which was where it was trying to go in the first place.

reauth

ReAuth is a special WordPress feature that forces the client to re-authenticate no matter what. Normally, when you sign into your WordPress site (and you have cookies enabled in your browser), you’ll stay logged in to WordPress for a certain amount of time. This is because WordPress is able to store cookies locally in your browser that indicate that you’ve already authenticated properly and shouldn’t need to log in again. However, as a safety method, if WordPress fails to detect a proper login cookie, it redirects the client to wp-login.php and forces it to re-authenticate and acquire new cookies by setting reauth=1.

Updated Posted by Arnon Erba in Server Logs Explained on .

A couple weeks ago, I covered what a WordPress brute-force attack looks like. However, you may have realized that trying an unlimited number of passwords is futile if you don’t know any valid usernames to guess passwords for. Fortunately for crackers, there’s a simple way to abuse the WordPress “pretty permalinks” feature to obtain valid usernames for a WordPress installation. Fortunately for us, there’s a simple way to block this with Nginx.

The Logs

Like a brute-force attack, a user enumeration attempt is usually pretty easy to spot. The logs usually start out like this:

203.0.113.42 - - [23/Jun/2016:17:04:11 -0700] "GET /?author=1 HTTP/1.1" 302 154 "-" "-"

And then continue like this…

203.0.113.42 - - [23/Jun/2016:17:04:12 -0700] "GET /?author=2 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:13 -0700] "GET /?author=3 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:15 -0700] "GET /?author=4 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:16 -0700] "GET /?author=5 HTTP/1.1" 302 154 "-" "-"

…until the cracker gives up.

203.0.113.42 - - [23/Jun/2016:17:04:18 -0700] "GET /?author=6 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:19 -0700] "GET /?author=7 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:20 -0700] "GET /?author=8 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:22 -0700] "GET /?author=9 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:23 -0700] "GET /?author=10 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:25 -0700] "GET /?author=11 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:26 -0700] "GET /?author=12 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:28 -0700] "GET /?author=13 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:29 -0700] "GET /?author=14 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:30 -0700] "GET /?author=15 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:32 -0700] "GET /?author=16 HTTP/1.1" 302 154 "-" "-"
203.0.113.42 - - [23/Jun/2016:17:04:33 -0700] "GET /?author=17 HTTP/1.1" 302 154 "-" "-"

Why This Works

By default, WordPress uses query strings as permalinks, such as:

http://example.com/?p=123

This example permalink would display a post with the ID number “123”. The post ID is generated when a new post is created. Query strings make for “ugly” permalinks, however, so WordPress allows you to enable “pretty permalinks” using Apache mod_rewrite or a custom try_files directive in Nginx. With “pretty permalinks” enabled, WordPress performs an HTTP 301 redirect from the “ugly” permalink to the “pretty permalink” configured on the Settings>Permalinks screen.

WordPress doesn’t just have IDs for posts, though. Every WordPress user, or author, has a unique ID that maps to their archive page, which is a list of all the posts that they have created. “Ugly” author permalinks look like:

http://example.com/?author=1

When pretty permalinks are enabled, the author archive page looks like:

http://example.com/author/username

This reveals the author’s WordPress username. A simple script can easily enumerate all the usernames on a WordPress site by trying ?author= with sequential numbers, as we saw in the log excerpts above.

Mitigating WordPress User Enumeration Attempts

We can block user enumeration on two levels: by redirecting the “ugly” permalinks, or by redirecting the /author/ pages entirely. Keep in mind that even if you disable the /author/ pages, your username can be discovered through other methods, and you should assume it is publicly available knowledge. However, we can make it difficult for the public to obtain that knowledge.

Disable query-string based user enumeration

A simple if statement works to disable user enumeration using query strings (or “ugly” permalinks). This is a “safe” if statement in Nginx (see the infamous If Is Evil page) since we are using it with a return statement.

if ($args ~ "^/?author=([0-9]*)") {
        return 302 $scheme://$server_name;
}

This code uses a simple regex, or regular expression, to match any URIs that end in /?author= plus a number. Here’s how it works:

$args is an Nginx variable for the query string
~ indicates that we want Nginx to perform a case-sensitive regex match using the regular expression inside the double quotation marks
^ (the carat) indicates the beginning of the path
/?author= is the fixed part of the path
([0-9]*) is a capturing group that matches any combination of numbers between 0 and 9

The return statement then redirects any URIs that fit the pattern.

Disable WordPress author pages entirely

We can add a simple location block to disable the author pages entirely. This solution is a bit redundant, because you would have to already know the author’s username to access their /author/ archive page, but this is useful if you don’t want author archive pages on your blog for some reason.

Note: this solution, by itself, does not prevent user enumeration, because the intermediary step between the query string and the author archive page pretty permalink will not be hidden. In other words, the query string will redirect to the archive page, revealing the username, and then will redirect based on the code below.

location ~ ^/author/(.*)$ {
        return 302 $scheme://$server_name;
}

~ starts a case-sensitive regex match, like above
^ starts the path we want to match
/author/ indicates we want paths beginning with /author/ to be matched
(.*) is a capturing group that matches any character except newlines
$ marks the end of the path

If a URI is matched, it is redirected to the root server name using return, like above.

Updated Posted by Arnon Erba in Server Logs Explained on .

There are a couple different ways that crackers will try to get into your WordPress installation, and one of them is by using a plain old brute-force attack. This kind of attack requires nothing more than a freely available exploit toolkit, and is not difficult to detect in the server logs. In the first section of this post, I’m going to give an example of what a brute force attack looks like, and then to make things more interesting I’ll discuss some techniques used to mitigate them using Nginx.

The Logs

As you would guess, when one computer makes hundreds of requests for a resource in quick succession, it leaves some pretty serious traces in the server logs (these are real logs, but I removed the server name):

203.0.113.42 - - [22/Jun/2016:19:18:58 -0700] "POST /wp-login.php HTTP/1.1" 200 3848 "http://www.example.com/wp-login.php" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; 125LA; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)"
203.0.113.42 - - [22/Jun/2016:19:18:59 -0700] "POST /wp-login.php HTTP/1.1" 200 3848 "http://www.example.com/wp-login.php" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; 125LA; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)"
203.0.113.42 - - [22/Jun/2016:19:18:59 -0700] "POST /wp-login.php HTTP/1.1" 429 0 "http://www.example.com/wp-login.php" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; 125LA; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)"
203.0.113.42 - - [22/Jun/2016:19:19:00 -0700] "POST /wp-login.php HTTP/1.1" 429 0 "http://www.example.com/wp-login.php" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; 125LA; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)"
203.0.113.42 - - [22/Jun/2016:19:19:00 -0700] "POST /wp-login.php HTTP/1.1" 429 0 "http://www.example.com/wp-login.php" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; 125LA; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)"
203.0.113.42 - - [22/Jun/2016:19:19:00 -0700] "POST /wp-login.php HTTP/1.1" 429 0 "http://www.example.com/wp-login.php" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; 125LA; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)"

Here’s a couple things about these requests that make it obvious that this is a brute-force attack (other than the fact that they go on for about half an hour).

  1. The HTTP method is POST, which indicates data is being sent to the server (i.e. the actual password guesses).
  2. The resource requested is /wp-login.php, which is the default WordPress login page and should rarely be requested, even by legitimate users.

If you look more closely, however, you’ll see something interesting: the HTTP response code that the server returns starts off as 200 OK, but quickly transitions to 429 Too Many Requests. This is one method of fending off brute force attacks with Nginx.

Mitigating WordPress Brute-Force Attacks

Fortunately, WordPress brute-force attacks are not that difficult to defend against without the use of plugins or additional software. We can:

  1. Restrict access to the login page to a curated list of IP addresses,
  2. Explicitly block the IP addresses of known brute-force offenders with Nginx or with a firewall,
  3. Password-protect the login page using HTTP Basic Authentication,
  4. Or, my personal favorite: set up rate-limiting with Nginx to cut down on how many requests attackers can make in a certain period of time.

Restrict Access to Certain IP Addresses

Arguably, the best way to mitigate brute-force attacks is to restrict access to the WordPress login page to only known good IP addresses. Here’s what that looks like with Nginx:

location = /wp-login.php {
    allow 192.168.1.2;
    allow 192.168.1.50;
    deny all;
    # add your PHP fastcgi config here
}

This location block explicitly targets the /wp-login.php page and only allows clients using the IP addresses 192.168.1.2 and 192.168.1.50 to access it. All other requests will be met with a 403 Forbidden error message. Keep in mind you will need to add your PHP fastcgi config to this location block as well so that Nginx knows to pass legitimate requests back to PHP. If you’re not familiar with how to do this, either consult the Nginx docs regarding PHP or keep an eye out for a newer post.

This method ensures that attackers will never get access to the login page, but is difficult to maintain if legitimate WordPress users do not have static IP addresses.

Deny Access from Certain IP Addresses

Another solution is to explicitly block brute-force offenders. You can block certain IP addresses from accessing the login page with:

location = /wp-login.php {
    deny 203.0.113.42;
    # add your PHP fastcgi config here
}

If you are familiar with configuring firewalls, you can use firewall commands to block the IP address from accessing anything on your server at all.

While blocking specific IP addresses can be useful, I don’t recommend using this as your only line of defense. For one, any IP address used in a brute force attack is almost certainly a VPN, proxy, or bot IP address. By blocking these, you risk denying access to legitimate users, even if that risk is slight. The main concern is that maintaining a list of IP addresses is tedious and unwieldy and is not a good long-term solution. That’s not to say this approach is useless, however, as you may want to use it in tandem with another one.

With that in mind, the next possible solution is adding a second layer of protection to the WordPress login page with HTTP Basic Auth.

Restrict Access Using HTTP Basic Auth

There are two steps to using HTTP Basic Auth with WordPress and Nginx.

  1. Create the password file
  2. Configure Nginx

I am going to skip the first step in this post, as there are many good existing guides on using openssl or apache2-utils to create a password file (see here or here).

The second step, configuring Nginx, is fairly simple. Just add two lines to your wp-login location block:

location = /wp-login.php {
    auth_basic "Restricted Content";
    auth_basic_user_file /path/to/.password_file;
    # add your PHP fastcgi config here
}

You can change “Restricted Content” to any phrase you want, as it will be the message that end-users see when they attempt to access the login page. Make sure you enter the correct path to your password file you created as well.

While password-protecting the login page is a valid solution, it has the potential to overly complicate the login process for legitimate users.

Using Rate Limiting in Nginx

Nginx has some great documentation on how to implement rate limiting, but I am going to provide an example of how to optimize it for WordPress. Setting up rate limiting in Nginx is simple, and only requires two components:

  1. We must define a zone in the main nginx.conf file.
  2. We must implement that zone in the WordPress login location block.

To define the zone, we use limit_req_zone and, optionally, limit_req_status. These directives go inside the http block of the main nginx.conf configuration file.

http {
     limit_req_zone $binary_remote_addr zone=wordpress:10m rate=15r/m;
     limit_req_status 429;
}

The above snippet defines a 10 MB zone named “wordpress” that allows a maximum of 15 requests per minute from any one IP address. The limit_req_zone requires a variable, or key. In this case, the key is $binary_remote_addr, or the IP address of the client. Nginx will use a maximum of 10 MB of memory to store the keys, and if a key exceeds the maximum number of allowed requests, Nginx will terminate the connection and return the status code defined in limit_req_status. The default code is 503 Service Unavailable, but I prefer the more specific 429 Too Many Requests response. Keep in mind that Nginx will display a blank page to the client for non-standard HTTP codes if you have not set a custom error page using the error_page directive.

You can name the zone anything you want (it is named “wordpress” in the example above) and you can also define any rate limit you feel is appropriate. I found that allowing a maximum of 15 requests per minute is restrictive enough to hamper a brute-force attack but is permissive enough not to interfere with end-users who legitimately mistyped their passwords.

To actually use the zone, we must implement it by adding this code to the WordPress login location block:

location = /wp-login.php {
    limit_req zone=wordpress;
    # add your PHP fastcgi config here
}

This tells Nginx to limit requests to the /wp-login.php page using the parameters specified in the zone we defined above. Make sure you replace “wordpress” with whatever you named your zone in the previous step. Restart or reload Nginx and rapidly refresh your login page to test if the new brute-force protection is working. If you refresh faster than the rate you defined in limit_req_zone, the server will return the status code defined in limit_req_status.

Obligatory note: if you’ve read other guides on how to set up rate limiting with Nginx, you may have seen other syntaxes used, such as limit_req zone=one burst=1 nodelay. The burst and nodelay options are more complex and allow you to control what happens to excess requests. They are not necessary in this context, since we want any excess brute-force attempts to be immediately rejected, but I would highly encourage you to read the documentation for them here.

Conclusion

This is by no means an exhaustive list for preventing brute-force attacks. Other solutions exist in the form of WordPress plugins or intrusion prevention systems such as Fail2ban. However, a lot can be accomplished by correctly configuring Nginx, and the less WordPress plugins you have installed, the better.

Updated Posted by Arnon Erba in Server Logs Explained on .

(Editor’s note: The IP addresses in this post have been replaced with reserved IP addresses for documentation purposes. This post has been updated since publication to include more information about the w00tw00t scan.)

What is w00tw00t.at.blackhats.romanian.anti-sec?

If you read last week’s post, you’ll remember that I promised to post a more interesting log excerpt this week. This one is from a pretty common bot scan that you’ll see if you’re running a web server for any length of time, and while it looks scary at first, you likely don’t need to worry if your server is configured properly.

203.0.113.42 - - [21/Jun/2016:06:35:55 -0400] "GET /w00tw00t.at.blackhats.romanian.anti-sec:) HTTP/1.1" 400 0 "-" "ZmEu"

In this log excerpt, we see that an IP address that maps to the Netherlands made a GET request for /w00tw00t.at.blackhats.romanian.anti-sec:), a nonexistent resource. However, the server returned a 400 Bad Request error rather than a 404 Not Found.

What This Means for You

Because I didn’t grab the accompanying error log entry that explains why Nginx returned a 400 error, I’m going to skip right to the explanation (spoiler alert). The w00tw00t entries are created by the ZmEu or DFind vulnerability scanners as part of an attempt at banner grabbing. Banner grabbing is an enumeration technique, and in this case the scanner was searching for information about my server that could reveal possible exploits. The process goes something like this: a bot, possibly an infected computer or a proxy server, sends an HTTP GET request with a bogus URI in the hope that the targeted server will respond with some information about its configuration. In my case, Nginx determined that the HTTP request was malformed in some way, so it rejected it with a 400 Bad Request status code. Most likely, the request was missing the Host header, in the hope that my server would fill it in or provide some other information.

The bottom line is that if you’re running a web server, you’re going to come across these requests in your server logs at some point. The Internet is frequently scanned by script kiddies looking for various vulnerabilities, but as long as your server returns a 400 error for any w00tw00t requests, you shouldn’t have to worry. There are a few other variants of this scan as well, including one that makes a request for /w00tw00t.at.ISC.SANS.DFind:).

Further Reading

If you want to read more about the w00tw00t scan, here’s some extra resources for more information: