N.B: Winds is turning off trackbacks as we look at our options. Here's why, and also why spam is becoming a cyber-security issue not just an annoyance.
...Hi, all. Some of you may have noticed that Winds has been a bit slow lately, or even offline some of the time when you attempt to read or post comments.
The problem is the level of spam we're getting - while comments spam is near zero here, trackback spam continues to multiply at an alarming rate. For example, one spamvertised URL I blocked today has generated over 1,500 trackback spams in under 3 hours. Yesterday we had two spamverised items blocked that day which went over the 1,500 mark (one broke 5,000). We're probably receiving even larger numbers of spams to other parts of our blacklist, esp. terms like "poker" et. al. - we'd have to look at the logs to see, but the number of spams coming in makes it impossible to display the log in HTML. The straight count of blocked items, however, suggests strongly that we're at 10,000-20,000 blocked spams per day over the last week or two. When the load climbs high enough, the result is an effective DOS (Denial Of Service) attack.
Nor do upgrades to Movable Type 3.2 or even the forthcoming version 3.3 look likely to solve the problem....
Any blocking system requires some computing power for each blocked item, esp. if the process also triggers database accesses etc. The result is that spam levels have effectively become a Denial-of-Service (DOS) attack, or a Distributed Denial-of-Service (DDOS) since many spammers use viruses and trojans as "repeaters". I've verified that this is the problem via our VPS dashboard.
Regarding the non-usefulness of a Movable Type upgrade, evariste had this to say:
"The new MT simply accepts all spam and holds it in sort of a "junk folder". The same number of perl CGIs will be running, one mt trackback script being spawned for each incoming spam trackback and one mt comment script being spawned for each incoming spam comment. The load on the database will get worse because instead of a lot of spam never making it into the database, MT will accept it and about 30 days later, delete it. So you will have a sort of 30-day "revolving balance" of spam trackbacks and comments loaded in the database at all times, being indexed and maintained in memory by MySQL at all times.
This is obviously an untenable proposition-if you're getting 1000 spam comment attempts and 5,000 spam trackback attempts a day, for instance, this will lead to something like 180,000 junk items in memory each and every day, increasing as the volume of spam increases. MySQL can't handle that-there's just no way in hell. Actually, I just looked at Winds and saw the entry you've published. 10,000-20,000 a day, eh? That's 300,000-600,000 junk items in memory in your "revolving account". MT 3.2 is simply not the way to go IMHO."
This is a steadily increasing problem for the blogosphere as a whole, since the computing power required to run phony trackback scripts is both more efficient and more likely to be scaled to high levels than any local defense system. In the long run, therefore, the spammers will win unless the playing field changes.
Which may happen, if enough of us work for it. As it climbs to DOS/DDOS levels, spam needs to become a security issue as well as an annoyance issue. With prosecution, prioritization, and punishments meted out accordingly.
Meanwhile, Winds is looking at our options. One thing we're forced to do immediately is turn off trackbacks (unfortunately, a fundamentally poorly-architected no-security system) while we do so, in order to preserve our readers' ability to use the site as intended.
Any opposed to the death penalty in this case?
Not I - unless there was a possibility of double-kneecapping by Black & Deckers, plus surgical removal of both eyes, as an alternative.
FYI, the spamvertised URL I blocked today is now over 2,500 hits in about 3 hours. That's just ONE item in our blacklist.
Joe:
You might consider a new blogging tool currently in development, called Minx.
Actually I should have been less mysterious about Pixy's little project. Read the comments, especially comment #2 and #4 to this post.
I feel your pain. Not that my lowly 1700 uniques a day is anything like WOC, but I turned off comments and trackbacks because I was tired of fighting the battle of spam.
Joe.
If I were you, I'd load a list of blocked IPs into a hardware firewall.
Then add the "no follow" as a plug-in to your MT. This scrambles the bots that try to pick up the trackbacks. That step has elminated all of my trackback spam, and I used to get a hundred a week or so.
I have MT 3.2, and the spam features it has are really not on scale with what your needs are. That's fine because most MT customers aren't WOC. I believe there are a lot of solutions, but first I'd stop those packets from bad IP addresses from getting anywhere near your server
As long as you've got your trackbacks turned off anyway - remove or rename your mt-tb (unless you have a fancy not-found page, in which case you might want to consider a htaccess redirect of some sort.)
Don't just turn them off via mt. The 404 uses a lot less bandwidth and cpu time.
I'm looking at out VPS dashboard, and the drop in load since we cut off Trackbacks is stunning - on the order of 60%-90% drop for consistent average load (never mind spikes).
Over a 6-hour period, the number of trackback spams promoting materials at itunisie was at 5,683 trackback spams when we cut the feature off and gave MT-Blacklist a trackback vacation. Reductil promoting spams were at 1,649, and heartcall-music spams were at 5,715. That's just the top 3 for the last 40 items we added to our blacklist over the last 2 days (40+ items added is also not unusual for a 2-day period).
Unfortunately, we do have a very fancy not-found page which you can see here - but so far, the server load is more than acceptable without Kathy's additional step (we have a somewhat more robust server setup than most). Many thanks for the suggestion, though, just in case things change... and others learning from our experience may find that to be a very useful tip.
How many spams per day before we can call it a civil war?
Until the other side fights back, alchemist, it never will be. Situations in which only one side is fighting and causing casualties are referred to as "cleansings," massacres, genocides, et. al.
Without sustained opposition that raises the profile and seriousness of the spam threat, and begins to draw blood among the core spammer network (actually, a fairly small group of problem children), the term "trackback & discussion cleansing" would be the best description of what's happening to the blogosphere.
Email use would be next - as anti-spam advocates predicted several years ago, to little effect as Congress was bought and paid for by the DMA et. al.
Kathy K;
I must disagree, based on my own empirical studies. The redirect is the best choice but it should go to a cheap page that returns success to the junker (something like this). In one experiment, on a weblog that had been active for a few years, I put this in place for the trackback script and then changed the script name. I went from 100-200 a day to 1-2 a week. My belief is that the junker scripts are still hitting the sand trap but don't realize it because they don't get any errors. I also run with non-numeric trackback URLs which helps as well, so the causal chain isn't completely clear.
Mr. Markham;
Or you could use AutoBan.
Annoying Old Guy (#11)
Autoban looks good. Thanks for the link. Maybe Joe should check it out instead of turning off features on the blog engine.
Ooh, burn
We're installing mod_perl tomorrow to see if that speeds up the server enough to make the spam DOS effect less potent.
Checking that comments are enabled.