menu

Posts Tagged #Security

Posted on by Arnon Erba in News

Update 1/16/20: According to Namecheap, the issues with DNSSEC have been resolved as of 2:00am EST (11:00 PM PST).

Have a domain registered at Namecheap with DNSSEC turned on? Now might be a good time to check if it still resolves.

Since at least 11:21pm Eastern Standard Time (8:21pm Pacific Standard Time) today, DNSSEC for domain names on Basic/PremiumDNS has been broken. So far, the issue appears to be caused by an expired signing key, but according to the latest status update “there is no current timeline for resolution of this issue”. This happens to be a fairly serious issue as DNSSEC validation for affected domain names will fail and cause websites and services to become inaccessible to some users.

The full text of the status update is copied below. This post will be updated if the status of the incident changes.

We are currently experiencing temporary technical issues with DNSSEC for domain names on Basic/PremiumDNS. If your domain name has DNSSEC option enabled, it may cause DNS performance issues. Unfortunately, there is no current timeline for resolution of this issue. We will keep you updated on the progress. Meanwhile, please contact our Support Team for assistance and more details. Please accept our sincere apologies for the inconvenience. Thank you for your continued support and patience.

Oh well, maybe no one is using DNSSEC anyway.

Posted on by Arnon Erba in News

If you saw a headline earlier this week about a critical security flaw in VLC media player, you may not have gotten the whole story. In fact, the issue is not nearly as serious as it originally seemed.

About a month ago, a user opened a bug report for a crash in VLC caused by a specifically crafted mp4 file. With the cause of the crash still undetermined, MITRE assigned the bug a CVE identifier and gave it a “critical” score of 9.8.

With the bug’s true cause and impact still undetermined, Germany’s CERT-Bund issued an alert of their own warning of a critical flaw in VLC. Worse, because the now several-week-old VLC bug report did not list any significant progress by the VideoLAN team, CERT-Bund announced that no patch was available. The alert kicked off a flurry of other news articles that culminated in a misguided warning from Gizmodo to completely uninstall VLC.

Not a VLC Bug

The only problem was that there was never anything wrong with VLC in the first place. The crash described in the bug report was the result of a vulnerability in libEBML, a third-party library that VLC depends on. However, according to a thread on Twitter from the VideoLAN team, a patched version of libEBML has been shipped with VLC for over a year. It appears the bug report was generated from a Linux system with an older, vulnerable version of libEBML installed.

With that in mind, the CVE score was lowered to “medium” and the report in the VLC bug tracker was closed. Ubuntu released an update for libEBML, and Gizmodo withdrew their doomsday-level announcement. In the end, no patch for VLC is currently required, though some Linux distributions may need to make an updated version of libEBML available.

Read More

Posted on by Arnon Erba in News

This morning, Apple released iOS 12.1.4, an incremental update that fixes several security issues including the Group FaceTime eavesdropping bug from last month. The Group FaceTime service has also been re-enabled for devices running iOS 12.1.4 or higher.

The eavesdropping bug, discovered accidentally in January by a 14-year-old from Arizona, caused certain Group FaceTime calls to automatically connect even if the recipient did not answer the call. This flaw allowed macOS or iOS users to be eavesdropped on by any malicious FaceTime user. The bug was disclosed privately to Apple by the teen and his mother at least a week before it went public, but it appears that Apple did not clearly or immediately respond to the bug reports they filed.

Shortly after the bug went viral on January 28th, Apple took the Group FaceTime service offline as a temporary fix before a patch could be released. On February 1st, with Group FaceTime still offline, Apple announced that the bug had been fixed server-side and that a client-side software update to fully resolve the issue would be available the week of February 4th.

Read More

Posted on by Arnon Erba in Server Logs Explained

(Editor’s note: This post has been updated since publication. A new section has been added at the bottom of the post with additional information.)

In today’s world, the exhaustion of IPv4 addresses and the slow adoption of IPv6 means that publicly routable IPv4 addresses are in high demand. It also means that when you spin up a cloud-based virtual private server using a service like Digital Ocean, Linode, or Amazon Web Services (AWS), you’ll almost certainly get an IPv4 address that was previously in use by someone else. In the worst case, your new IP address might be on some blacklists, but the most likely situation is that you’ll get some extra “background noise” in your server logs.

The Logs

167.114.0.63 - - [06/May/2018:03:10:08 +0000] "GET /0f0qa0a/captive_portal.html HTTP/1.1" 404 152 "-" "Go-http-client/1.1"
37.187.139.66 - - [06/May/2018:03:10:41 +0000] "GET /0f0qa0a/captive_portal.html HTTP/1.1" 404 152 "-" "Go-http-client/1.1"

These Nginx logs were pulled from a fresh virtual private server that I created with a new-to-me IPv4 address. If you’re curious, my original Server Logs Explained post contains a breakdown of the log format I’m using, but I’ll cover what these log excerpts mean in this post as well.

Essentially, two completely different IP addresses performed an HTTP GET request for the same resource, /0f0qa0a/captive_portal.html. Unable to provide this mysterious file, my server responded with 404 Not Found. This pair of log entries became much more interesting when I noticed that my server kept getting the same two requests from the same two remote IP addresses every few seconds.

Some Detective Work

First of all, these log entries are completely harmless. Anyone can request any random page from a web server, and the server should return a 404 response if the page does not exist. At this point, curiosity is the only reason to continue exploring the source of the two requests.

A cursory WHOIS lookup on both IPs reveals that they are owned by OVH Hosting, a French company that provides cloud-based hosting services. It’s similar to the other cloud hosting companies I mentioned at the beginning of this post. It isn’t a big leap to assume that both IPs belong to virtual servers hosted by OVH, so with that assumption in mind, let’s move forward.

Next, a reverse DNS (rDNS) lookup on each IP address yields the following interesting results:

167.114.0.63 maps to prometheus-nodes-ca-ovh.afdevops.com
37.187.139.66 maps to prometheus-nodes-eu-ovh.afdevops.com

Right away, there’s “ovh” in the hostnames for the two servers, which seems to confirm that OVH Hosting was a good guess. There’s also something else interesting about these results: 167.114.0.63 is a Canadian IP address, and it has “ca” in its hostname. On top of that, 37.187.139.66 is a French IP address, and it has “eu” in its hostname. It’s beginning to look like these two servers are part of some company’s public-facing server infrastructure.

Let’s look at the far left section of the hostnames, prometheus-nodes. There’s no law that dictates how you should name your servers, but it is pretty common to give them logical names that correspond to the software that runs on them. With that in mind, what is “prometheus”?

Prometheus is a real piece of software. Its GitHub page describes it as “a systems and service monitoring system”, which would explain the persistent requests in my server logs. Monitoring solutions work by constantly checking a service, evaluating the responses they receive, and notifying administrators if something looks wrong. It seems reasonable to conclude that some important service was being hosted by the previous owner of my IP address, and someone forgot to reconfigure their monitoring solution after decommissioning the server.

There’s still more we can learn from the reverse hostnames. A simple Google search for one of the hostnames turns up this GitHub issue from February, where a bunch of information is listed that confirms that the server runs Prometheus. On top of that, the user who opened the issue claims to work for a company called “AnchorFree”.

AnchorFree — could that have anything to do with the afdevops.com portion of the hostnames? Even though there’s no public-facing website associated with that domain, Googling for “afdevops.com” turns up this user profile on Docker Hub. Guess what’s listed in the bio for that user? Anchorfree, Inc.

There’s something else, too. A WHOIS lookup on “afdevops.com” reveals that it is registered under the real name of one of the co-founders of AnchorFree. That was easy. Too bad GDPR is killing WHOIS at the end of this month. (Ed: This was accurate when this post was drafted in 2018.)

Now that we know that both servers are owned by AnchorFree, let’s figure out what AnchorFree actually is. Wikipedia and AnchorFree’s actual website both confirm something interesting — AnchorFree is the parent company behind Hotspot Shield, a fairly well-known “freemium” VPN app.

The Conclusion

Nothing in this post is particularly revealing, but it’s interesting to consider what happens when IP addresses owned by cloud service providers get reused. If you incorporate cloud service provider IP addresses into your server infrastructure, it’s important to remember that those addresses may be recycled for other customers in the future. If you set up important services or access control lists based on IP addresses you don’t own, it’s possible to introduce problems that won’t become apparent until later. In cases like this, it might be as simple as having your monitoring solution check the wrong IP address for a few days.

On the other hand, consider what might happen if you decommissioned a server but forgot to remove its forward DNS entry. If a malicious actor gained control of its old IP address, they could set up a phishing website that would appear to be hosted on your domain. Change control and detailed documentation are important when it comes to using public cloud services.

Update (1/9/20)

A reader contacted me in October 2019 to let me know he had been hit by similar requests from two additional IP addresses:

167.114.65.142, or prometheus-longterm-ca-ovh.afdevops.com
167.114.119.75, or prometheus-dev-ca-ovh.afdevops.com

I also happened across a completely unrelated blog post from 2018 that documents the same strange requests for /0f0qa0a/captive_portal.html that were the basis for this post. In that case, the author chose to block the repetitive requests with his firewall.

Finally, on a whim, I tried accessing one of the IP addresses in a web browser and was taken to a login page asking for AnchorFree SSO credentials. By itself, that isn’t particularly surprising, since we already know that AnchorFree runs those servers. In any case, it would be great if AnchorFree did a little auditing of their public cloud infrastructure.

Posted on by Arnon Erba in News

(Editor’s note: This post has been updated since publication.)

As announced in February this year, Google Chrome’s design is being evolved to more clearly indicate to users that websites using plain HTTP are not loaded securely and that HTTPS connections should be expected instead. Today, Chrome is pushing a change that affects all HTTP sites worldwide: starting in version 68, Chrome will display a “Not Secure” warning in the address bar for all sites loaded over HTTP.

This isn’t the first change Chrome has made to clearly indicate that HTTP is not secure. Chrome has been marking HTTP traffic as “Not Secure” in Incognito mode as far back as version 62. The “Not Secure” warning has also been appearing for HTTP sites in Chrome’s normal mode when a page contains a password field or when the user interacts with any input field.

Although Chrome has taken the lead, Mozilla Firefox is also on board with the effort to visually flag HTTP pages as insecure. Firefox currently displays an address bar warning for HTTP sites that contain login forms and displays a visible warning message next to login forms that are served insecurely.

What’s Next

The future will bring more changes for the way Chrome visually handles HTTP and HTTPS connections. As I covered back in May, Chrome is scheduled to remove the “Secure” text from HTTPS connections in September with the release of Chrome 69. One month later, in October 2018, Chrome will color the HTTP “Not Secure” warning red when users enter data into insecure sites in Chrome 70. Ed: A previous version of this post inaccurately reflected the circumstances in which the “Not Secure” warning will be colored red in Chrome 70. The color will only change when users enter data on HTTP pages.

Posted on by Arnon Erba in Server Logs Explained

(Editor’s note: This post has been updated since publication.)

Some log entries are particularly bizarre, like the one we’ll be looking at today:

169.229.3.91 - - [26/Jun/2016:00:35:26 -0700] "\xD5H\xC5p*\xB7:\x8F\x91\x8A\xE1\xAA\xE0p\xD9\xF2[;\xAE\xE7c\xF7\x9C\xAB~\x98\xCB\xAD\xCB\xBE\xCE\xED\xAF\xEC\x8B\x19\xC6\x08D\xEB\xA8\x91\x1De\x10\x18 u\x01zHj\x00\x8D|\x15\x8B;\x98\x08RaSH" 400 166 "-" "-"

My server responded with 400 Bad Request, but the most interesting part is the giant $request portion, which doesn’t include any of the normal components you would expect in an HTTP request:

\xD5H\xC5p*\xB7:\x8F\x91\x8A\xE1\xAA\xE0p\xD9\xF2[;\xAE\xE7c\xF7\x9C\xAB~\x98\xCB\xAD\xCB\xBE\xCE\xED\xAF\xEC\x8B\x19\xC6\x08D\xEB\xA8\x91\x1De\x10\x18 u\x01zHj\x00\x8D|\x15\x8B;\x98\x08RaSH

Note: See my first Server Logs Explained post for an example of how to interpret the entire log entry.

Wait, What?

If your first thought was that this looks like 64 bytes of garbage, then you’d be exactly right. As it turns out, I wasn’t the first person to see one of these bizarre log entries. According to this Information Security StackExchange question and answer, this server request is from an Internet-wide research scan led by the Electrical Engineering and Computer Sciences (EECS) department at the University of California at Berkeley.

A reverse DNS lookup of the IP address led to an illuminating hostname, researchscan1.EECS.Berkeley.EDU. It turns out there’s actually several machines related to the project:

  • 169.229.3.90, or researchscan0.EECS.Berkeley.EDU
  • 169.229.3.91, or researchscan1.EECS.Berkeley.EDU
  • 169.229.3.92, or researchscan2.EECS.Berkeley.EDU
  • 169.229.3.93, or researchscan3.EECS.Berkeley.EDU
  • 169.229.3.94, or researchscan4.EECS.Berkeley.EDU

If you access any of those IP addresses or hostnames in a web browser, you’ll see a brief description of the project. However, according to this answer on StackExchange, the text on those webpages has changed over time, so the most concise explanation comes in the form of a quote from the project leaders at Berkeley that’s reproduced in that answer:

We are performing a measurement study of a particular phenomenon on the Internet. To accurately asses the behavior we’re performing a daily scan of the IPv4 space by sending a single benign packet to every IP on port 80 consisting of 64 random bytes of data. […] No, we are not attempting to gain unauthorized access. […] It’s simply randomly generated data that conforms to a certain set of criteria.

I contacted the project team a few months ago as well, but have not heard anything back. Given that the StackExchange answer and my log entry both date back to 2016, it’s possible that the research project is already over and is now just Internet history. Either way, it’s interesting to finally know what it is.