As of last week, this site is being served to you by a shiny new SSD-backed VPS at DreamHost. I was hoping it would be running NginX as well, but try as I might, I couldn’t get WordPress in a subdirectory to play nice with NginX. Speed Force worked fine, but it’s at the top level of a site. Ramblings and Re-Reading Les Misérables aren’t.

Fortunately, the new virtual servers are faster and cheaper (newer hardware, after all), and with the rest of my sites running NginX I end up with about the same overall memory footprint for two VPSes so that I could put this back on Apache. I suppose that saved me time converting the zillions of .htaccess rules I’ve amassed over the years. And with the faster systems, they’re able to handle more complex/simultaneous actions without timing out or spiking memory.

It’s always annoying when someone figures out a way to exploit intentional behavior, especially when it’s a key part of the design.

Sucuri reports on a denial-of-service attack that used thousands of legit WordPress sites to distribute the attack by sending fake pingbacks “from” the target site to all of the reflectors. Those blogs would all contact the targeted site to confirm the pingback and retrieve a title and summary…all at once, overwhelming it and taking it offline.

The quick-and-dirty solution is to remove XML-RPC functionality, but that also breaks certain plugins (like Jetpack) and the ability to connect to your blog using the WordPress mobile apps.

A little background on why Pingbacks work this way:

Waaaay back in the early days of blogging, most bloggers would interact by way of comments. If you wrote a blog post, and I was inspired to write a response, I would then go over to your site and post a comment letting you know about my own post. Two systems were proposed in 2002 to automate this process: pingbacks and trackbacks.

  • Trackbacks sent a complete summary to the remote blog, including the title of your post, the link, and an excerpt (which you could manually craft, or let your software handle).
  • Pingbacks sent a notice — a “ping” — to the remote site with the URL of your post, and then the remote site would retrieve it and extract the title and a summary.

This was also around the time that blog comment spam and spammy blogs were getting to be a big problem. What would happen is a spamming site would send out trackbacks to as many sites as possible claiming that they’d responded to some post, thereby getting backlinks on a zillion blogs and increasing their page rank. Pingbacks had an advantage: Because you were calling back already, your server could check to see whether the other site really had linked to you. It took a long time, but eventually this escalated into spammy blogs creating a temporary post with real links to the pages they pinged, then replacing it with a spam page after a short amount of time.

The problem now is: How do you block abuse of an as-designed behavior? That’s happened before: Back in the early days of the internet, it was considered polite to run your mail server as an open relay and rude to lock it down, but after spammers started massively abusing them, an open relay became a sign of a sysadmin who didn’t know what he was doing.

The comments on the Sucuri article suggest that Akismet, as a collaborative comment-spam filter, might be able to mitigate this type of attack. Wordfence’s collaborative security filter seems like another system well-positioned to detect it. But if that approach fails, pingbacks might just go the way of open relays.

Update March 18: Akismet has released a new version of the anti-spam plugin that mitigates this problem in two ways:

  1. Spam checks on pingbacks are now done before the verification request is sent, so that once an attack is identified, Akismet will prevent blogs from participating.
  2. An X-Pingback-Forwarded-For header is added to the verification request identifying where the pingback actually came from, making WordPress+Akismet a less attractive choice as a reflector by removing the anonymity.

Item #2, IMO, belongs in WordPress itself, not in a plugin, but I imagine this was a way to roll out the feature more quickly, at least to those sites using Akismet.

Update April 8: The X-Pingback-Forwarded-For header has been added to WordPress 3.8.2 and the upcoming 3.9.

My job sent me to a class on scaling, optimizing and troubleshooting MySQL this week. I’ve been digging around a bit on some test databases at work, but of course as someone running a self-hosted WordPress blog, I had another MySQL server to practice on right here — one with real-world data and (admittedly low) load, but where I was only accountable to myself if I messed anything up.

Unfortunately, DreamHost’s MySQL VPS doesn’t give you much control over the server, and of course when you’re working with a third-party application, there’s only so much you can change the database without breaking compatibility. But I found some interesting surprises:

1: Everything was using the older MyISAM engine, because DreamHost is running an older version of MySQL that uses it as the default. Switching to the newer InnoDB (and back) is simple and safe enough that I figured it was worth a try.

2: There was a lot of junk left over from old plugins that I haven’t used in years. Continue reading

I’ve been seeing brute-force login attacks on another of my WordPress sites, but instead of targeting typical usernames like admin or extracting post authors, they’re random name and number combinations like Emanuel95A. What use could that possibly be? You’re not likely to hit on an existing user that way.

It turns out it’s not a dictionary attack after all. It’s not really a login attack either, at least not deliberately. It’s actually a bot trying to register new usernames (maybe for spam, maybe in preparation for a privilege escalation attack, who knows?), which explains the name and number combination: they’re actually trying to get a username that’s not already in use.

The bot hasn’t figured out that registration is turned off, so when WordPress redirects it to the login form, it keeps trying to register…in the login form…over and over until it gets locked out. (On a related note, if you don’t have something like the Limit Login Attempts plugin on your site, install one now.)

Because registration was off and repeated logins were blocked, it wasn’t currently a threat, but the alerts for all the lockouts were getting a bit annoying. I decided instead of nicely sending the “user” to the login page, I’d kick back a 403 error instead. Rather than hack WP or write a plugin, I just added a mod_rewrite rule:

# Broken register bots are repeatedly trying to log into the site.
RewriteCond %{QUERY_STRING} (registration=disabled|action=register) [NC,OR]
RewriteCond %{HTTP_REFERER} registration=disabled [NC]
RewriteRule ^wp-login.php - [F,L]

That leaves the form active under most circumstances, but stops everything if it’s been redirected from the registration page.

Update: Brion sums it up over at Google+:

Malware needs better exception handling 😀