Wednesday, November 22, 2006
The new whack a mole
Back in the good old days, spammers used stolen credit cards to buy dial-up service from retailers like Earthlink. They'd "bulk" for a day or two and Earthlink would kill the account, and Earthlink or Visa would take the loss. Even though Earthlink had full time staff identifying and killing the accounts, the volume of spam kept growing, because there was no way they could react fast enough. Reaction doesn't work. We called this game of disposable dial-up accounts "whack a mole."
This problem wasn't really solved. Major ISPs just blocked incoming email from the known dial-up network address ranges, and the delivery rates got so bad the spammers moved on to other illegal methods. Eventually the dial-up providers blocked the route from their customers to other people's email servers, but they were locking the door to an empty barn.
These days most spam comes from gangs of compromised computers organized into "bot-nets." Most of them are in consumers' homes, infected with malicious software (malware) that only afflicts Microsoft's operating system, and connected to cheap "broadband" (ADSL or cable TV) and left on all the time.
But a significant fraction of these spamming stations are low cost Web servers installed by the thousands in data centers like Everyone's Internet (EV1) and Schlund. You don't even have to corrupt their operating systems. They're running years-old copies of PHP-Nuke and Joomla and phpBB and Squirrel Mail. Web applications any fool can install by clicking a button on the retailer's "control panel." Unfortunately, the "control panel" doesn't have a button for "bring my PHP-Nuke up to the current version." And the guy who's renting time on one of these boxes has no idea how to install a security patch, and doesn't have the necessary access, and even if he did, the "control panel's" version of the application is just different enough from the original that you can't be sure a security patch for the original won't break it.
Over time, exploits become widely known for the old versions of these well known application programs. Simple programs that know how to install a spam sending form through a security hole that was fixed in a later version of the program. High school kids trade them like baseball cards.
Over time, a huge data center like Everyone's Internet is running tens or hundreds of thousands of instances of these exploitable Web applications. But they're renting those low-cost servers to "virtual" Internet service providers or "resellers." Sometimes there are layers of resellers and virtual ISPs. The data center, who owns the IP addresses, doesn't know who the actual customers are. They don't know what domains the actual customers have. And they don't want to know. It's that negligence that's bringing you most of those phish spams.
So what happens is people like me get the phish spams and report them to the designated owners of the network addresses, because it is usually really hard for us to find the virtual ISP. Everyone's Internet or Schlund forwards our complaints to the virtual ISPs, and they may or may not get passed down to someone who administers the exploited application or account. Then people like me block the spam source in our email servers, or rely on public list operators like Spamhaus.org to do it. Eventually the end user starts to notice there are places that won't take legitimate email from the same address. This sometimes leads to the compromised machine getting fixed or shut off and reloaded and rented to some other virtual ISP. Sometimes both paths break down, due to widespread apathy, negligence, and irresponsibility, and the compromised machine sends spam for years, until so many spammers are using it that its drive fills up and it crashes.
But most of the time the compromised system gets fixed within a few days or weeks. Now consider a place like Everyone's Internet with ten thousand servers hosting a million "web sites." If half of them are exploitable and 1% are spamming at any time, there are five thousand spam sources in their data center at any time. Just not the same ones from one day to the next. Spammers have gotten really good at spreading the sources around. This means Everyone's Internet, Schlund, Advanced Internet Technology, and a dozen more are huge, permanent sources of the worst kind of spam. They're undercounted in the statistics from Spamcop and Spamhaus and Brightmail because no IP address accounts for much. You can't say look, Leo Kuvayev is hosted at EV1 and they won't kill him, and give the IP address a Spamhaus advisory. And EV1 can brag about how they kill spamming accounts within a day or two of hearing about them.
It's the new whack a mole, but without the hassle of stealing credit cards.
What's wrong with this picture? Well, somehow, there are large hosting centers that don't have the problem. There must be a well known solution, and there is. Proactivity. You don't wait for volunteers to report spam from your IP addreses. You seed the Internet with a few hundred innocent-looking "spam trap" email addresses, and you automatically scan the resulting torrent of junk for spam from your own network. Even better, you search the Internet for domains hosted on your network, and search those domains for known exploitable applications, and get them fixed or removed BEFORE they start spamming. It's not rocket science. It's probably cheaper than waiting for spam reports and following through on them. But I'll bet it's not cheaper than just ignoring the problem and becoming a cesspool. At least in the short and medium term. Why isn't this happening? There's no real pressure on the big ISPs to force their customers to do work they don't want to do.
This problem wasn't really solved. Major ISPs just blocked incoming email from the known dial-up network address ranges, and the delivery rates got so bad the spammers moved on to other illegal methods. Eventually the dial-up providers blocked the route from their customers to other people's email servers, but they were locking the door to an empty barn.
These days most spam comes from gangs of compromised computers organized into "bot-nets." Most of them are in consumers' homes, infected with malicious software (malware) that only afflicts Microsoft's operating system, and connected to cheap "broadband" (ADSL or cable TV) and left on all the time.
But a significant fraction of these spamming stations are low cost Web servers installed by the thousands in data centers like Everyone's Internet (EV1) and Schlund. You don't even have to corrupt their operating systems. They're running years-old copies of PHP-Nuke and Joomla and phpBB and Squirrel Mail. Web applications any fool can install by clicking a button on the retailer's "control panel." Unfortunately, the "control panel" doesn't have a button for "bring my PHP-Nuke up to the current version." And the guy who's renting time on one of these boxes has no idea how to install a security patch, and doesn't have the necessary access, and even if he did, the "control panel's" version of the application is just different enough from the original that you can't be sure a security patch for the original won't break it.
Over time, exploits become widely known for the old versions of these well known application programs. Simple programs that know how to install a spam sending form through a security hole that was fixed in a later version of the program. High school kids trade them like baseball cards.
Over time, a huge data center like Everyone's Internet is running tens or hundreds of thousands of instances of these exploitable Web applications. But they're renting those low-cost servers to "virtual" Internet service providers or "resellers." Sometimes there are layers of resellers and virtual ISPs. The data center, who owns the IP addresses, doesn't know who the actual customers are. They don't know what domains the actual customers have. And they don't want to know. It's that negligence that's bringing you most of those phish spams.
So what happens is people like me get the phish spams and report them to the designated owners of the network addresses, because it is usually really hard for us to find the virtual ISP. Everyone's Internet or Schlund forwards our complaints to the virtual ISPs, and they may or may not get passed down to someone who administers the exploited application or account. Then people like me block the spam source in our email servers, or rely on public list operators like Spamhaus.org to do it. Eventually the end user starts to notice there are places that won't take legitimate email from the same address. This sometimes leads to the compromised machine getting fixed or shut off and reloaded and rented to some other virtual ISP. Sometimes both paths break down, due to widespread apathy, negligence, and irresponsibility, and the compromised machine sends spam for years, until so many spammers are using it that its drive fills up and it crashes.
But most of the time the compromised system gets fixed within a few days or weeks. Now consider a place like Everyone's Internet with ten thousand servers hosting a million "web sites." If half of them are exploitable and 1% are spamming at any time, there are five thousand spam sources in their data center at any time. Just not the same ones from one day to the next. Spammers have gotten really good at spreading the sources around. This means Everyone's Internet, Schlund, Advanced Internet Technology, and a dozen more are huge, permanent sources of the worst kind of spam. They're undercounted in the statistics from Spamcop and Spamhaus and Brightmail because no IP address accounts for much. You can't say look, Leo Kuvayev is hosted at EV1 and they won't kill him, and give the IP address a Spamhaus advisory. And EV1 can brag about how they kill spamming accounts within a day or two of hearing about them.
It's the new whack a mole, but without the hassle of stealing credit cards.
What's wrong with this picture? Well, somehow, there are large hosting centers that don't have the problem. There must be a well known solution, and there is. Proactivity. You don't wait for volunteers to report spam from your IP addreses. You seed the Internet with a few hundred innocent-looking "spam trap" email addresses, and you automatically scan the resulting torrent of junk for spam from your own network. Even better, you search the Internet for domains hosted on your network, and search those domains for known exploitable applications, and get them fixed or removed BEFORE they start spamming. It's not rocket science. It's probably cheaper than waiting for spam reports and following through on them. But I'll bet it's not cheaper than just ignoring the problem and becoming a cesspool. At least in the short and medium term. Why isn't this happening? There's no real pressure on the big ISPs to force their customers to do work they don't want to do.
Comments:
<< Home
I need help. I am receiving email from a spammer with my email address at the top! Convenient for me but big mistake for them I hope . ..can I use this slipup to thrash the problem?
Cheers, Sieffe
sieffe@hotmail.com
Cheers, Sieffe
sieffe@hotmail.com
I got a bunch of those this week too. It seems some spam operation uses the recipient address in his From line. Maybe he thinks it will fool the spam filters. Well guess what, it's going to backfire. If it says it's from me, but it's not from one of my IP addresses, it's spam! Duh.
It would seem to me that being pro-active isn't that particularly hard - run network analysis software, compare behavior of servers to a baseline, and any significant deviations from that baseline in terms of network traffic (i.e. email volume and variety of recipient addresses, source / recipient ports, variety of outgoing network connections, etc.) get tagged for evaluation by a human. After a few days, re-tune the parameters to filter out the known goods, and you'll probably have a 99% accuracy rate.
Heck, you can even automate this, by automatically throttling down network traffic (not cutting it off entirely) for hosts matching a profile... stage this downwards towards a complete choke after, say a week, and the very few false positives will start complaining about slow traffic real quick, and the rest of the traffic will go away.
Do this consistently, and the spammers will find other networks to attack. There's no great technological or policy innovation required.
Post a Comment
Heck, you can even automate this, by automatically throttling down network traffic (not cutting it off entirely) for hosts matching a profile... stage this downwards towards a complete choke after, say a week, and the very few false positives will start complaining about slow traffic real quick, and the rest of the traffic will go away.
Do this consistently, and the spammers will find other networks to attack. There's no great technological or policy innovation required.
<< Home