Spam Wars - Jan 27th, '04
If we needed to come up with a new email protocol to eliminate spam, and (1) all email clients needed to be updated to handle mail using that new protocol, and (2) all mail servers needed to be reprogrammed to deliver mail using the new protocol, and (3) everyone agreed that the cost benefits of zero spam would outweigh the cost of upgrading, then couldn't we stop spam? I think the answer is yes.
So why do we still have spam? Why don't we have new protocol proposals lined up around the block? There seems to be a huge market for trusted email, but no one is offering products to fill the need.
We know how to do secure and verifiable SSL transactions over http, why can't we have a similar system for email? I don't see any reason the two systems couldn't exist side by side during a transition. And the reduced load due to zero spam would more than make up for any overhead related to public key encryption handshakes. And while we're at it we could even build in anti-virus checking.
What am I missing here? Safe email with zero spam seems like a killer app.
17 Comments
Comments:
I have a nice spam filtering setup so I only get a couple a day in my inbox, compared to the hundreds filtered, so I dont have the need, but some people do.
Not even close. I get lots of confirmation email from automated ecommerce systems. I also sign up for mailing lists. I need those and the opt-in idea would block them. And I use PopFile to filter spam, so almost none of it shows up in my inox, but that doesn't solve the problem. The mail server is still getting hosed, and huge amounts of bandwidth are being wasted.
Spam filters make the problem worse because they just make spammers send more spam.
When you sign up for a list just add that sending address to the whitelist of the opt-in system.
Setup a "confirmation for ecommerce" email address and add that to the whitelist of your opt-in system. When the company sells your info go after them.
I myself dont have a real problem with spam. From my logs I get a couple hundred a day but only a couple ever reach my inbox. Spam in my inbox is the same as spam in my physical mailbox. I just filter out what I dont need.
The reason we dont get more in our mailbox is because it costs money to have something sent out via USPS. Email is free. Do we charge a nickle per message? No, that would ruin the advantages of email.
Setting up something like SSL would help the matter, but it would be just like telemarketers. They would be valid people with a valid pitch but you would still have to tell each and everyone of them to take you off their list.
Plus you would have the added cost of getting an SSL cert to prove who you are. Most people using email would not want the additional costs involved in that.
We could create a do-not-spam list but that would just as useless as the do-not-call list. There would be too many loop holes to make it worthwhile.
I look at my spam filtering box like I look at my firewall. I need both of them to keep out the garbage I dont want.
Is there a solution out there? I'm sure there is. Is it technically feasible in the next few years to implement it? Probably not. 90% of the email users can't set up PGP keys. I dont think they could handle setting up a SSL cert to validate who they are.
We could train them, but these are the same people that cannot deny the urge to open the doubleclickmetoscrewupyourcomputer.exe attachments because it came from someone pretending to be their long lost 3rd cousin and then send Nigerian politcal prisoners their bank account numbers in hopes of getting rich.
For geeks like us, a SSL or public/private key system would work fine. It's the regular users out there that could not handle it and it would kill the killer app.
Seems like email and spam have all that already.
and
"Setup a "confirmation for ecommerce" email address and add that to the whitelist"
Sure. It can be done. And it will work after a fashion. But this is patching a broken system, and it won't work for non-savvy users. A solution only works if the vast majority of users are using it. i'm not saying spam is a problem or me personally - popFile works great - I'm saying spam is a problem for the Internet.
"Do we charge a nickle per message? No, that would ruin the advantages of email."
No we just charge for the certification of mail servers. End users don't need to buy anything. Bu mail clients would only draw mail from certified servers. I don't ned to prove who I am unless i want to send mail. and we could have a sliding scale so that small sites can get a cheaper cert that would only send out a few emails a day.
The end user could use the same tool they have now, just upgraded to the new protocol. Eudora and Outlook can both be configured to draw from either pop or IMAP. Just add another option.
"They would be valid people with a valid pitch but you would still have to tell each and everyone of them to take you off their list."
The point would be that you would have a verified chain that allow you to do this. Right now asking a spammer to take you of the list doesn't wok because most can't be identified or have zero motivation to honor our request. If they were going to lose an expensive certification that is hard to replace they'd have a strong motivation for no pissing you off.
And taking yourself off their list would be as easy as clicking a link. And this is something that could be verified easily. If you have a confirmation that a domain isn't suppose to send to your email address, and your address gets an email from that domain, their cert goes bye-bye. That could even be automated.
Sure, the system could be gamed, hacked, etc, but at least we'd be operating in a closed system which would allow for self-repair.
Would servers only accept email from other certified servers or would this only affect the client pieces?
What about all the email that is sent without a mail server of any kind? How many millions of lines of Perl are there out there that handle SMTP transactions without the help of a server? You would break every app that relied on itself for SMTP.
"I don't ned to prove who I am unless i want to send mail."
So I would need to pay to send my Mom an email or would the owner of the server that I am using as my outbound SMTP server be charged for me sending an email to my Mom? If you implement a sliding scale that means you are paying a per message rate to the owner of the server.
I would assume that the end user would be paying for this in the end. If an ISP has to pay for a cert to send email, they wont eat those costs just to be nice, they will pass that down to the consumer.
I can then see a monthly statement sent out to users listing their monthly email traffic. Similar to a phone bill.
Then we would get long distance email and local email charges. How about toll free email?
"If you have a confirmation that a domain isn't suppose to send to your email address, and your address gets an email from that domain, their cert goes bye-bye. That could even be automated."
What would stop them from getting a new cert after theirs is "revoked"? I have signed up for several SSL certs using my personal information and several different companies information. Its not like I can only get one my entire life.
This would work about as well as people being able to randomly submit domain names to RBLs. How many people have been shut out of email for a couple days until they realized that the person submitting the domain name did it because they were pissed at some support tech or something.
The company I work for gets put on RBLs at least once a year because some guy signed up to receive emails from us (I have proof that they signed up) and instead of just clicking the link at the bottom, or sending us an email, to remove themselves they submit our domain name to an RBL.
I dont think certifying mail servers is the answer. I dont really know what the answer is. We cannot really enforce anything since the Internet is pretty much anonymous. I dont think we can take away the anonymity of the Internet because thats one of its greatest strengths.
The reason the Internet took off like it did is because there were no real requirements other than speaking the correct language (TCP/IP protocols) and you had a valid phone number (IP Address). If you start requiring certs to do things on the web you will start to break it down and it will only be for those than can afford those certs.
I know several people that provide ISP type services for their family and friends out of the kindness of their hearts. Kind of like the old BBS days. If you require them to get certs so their family and friends can send and receive email you could prevent that person, and every person they are providing services to, from being on the Internet.
We could force ISPs to keep logs of all incoming and outgoing traffic based on IP Addresses, but that would be implementing a big brother system and we dont want that either.
"Would servers only accept email from other certified servers or would this only affect the client pieces?"
Right. Certified POP Protocol (CPP) would by definition be a new protocol, so cpp servers would only talk to other cpp servers. So you have a verifiable chain. The client app would only get mail from that chain.
"You would break every app that relied on itself for SMTP."
Sort of. But not quite. SMTP wouldn't go away, it would just get deprecated as mail clients moved to cpp. The thought is that as more people switched to cpp clients, the old code would need to be rewritten to use the new cpp mail servers. And these servers would have to be certified. It could be as easy as mapping php's mail() function to a new server globally. Or having Sendmail default to cpp rather than smtp.
It would be a pain, but not a crisis.
I already pay a fee to have http traffic served from JonSullivan.com. It makes sense that I'd have to pay a fee to serve mail from cpp.jonsullivan.com as opposed to smtp.jonsullivan.com. The cert could be tied to a limit of emails per day so that I wouldn't have to the same fee that a large company would.
"So I would need to pay to send my Mom an email or would the owner of the server that I am using as my outbound SMTP server be charged for me sending an email to my Mom?"
Probably neither. The charge for the certification only needs to support the infrastructure, which could be automated and distributed. I think the cost would be small enough that mail servers wouldn't need to pass it on. Especially since they'd be saving big time in bandwidth and admin if spam goes away. I would recommend making the process of getting a certification the hard part. Make it hard enough that a company wouldn't want to risk losing the cert.
But at any rate, the only fees would be paid by mail servers only, not end users.
"they wont eat those costs just to be nice, they will pass that down to the consumer."
But it would be a net savings. Dealing with spam and worms eats up a huge amount of resources. Make that go away and the savings to both ISPs and big businesses would be huge.
"I can then see a monthly statement sent out to users listing their monthly email traffic. Similar to a phone bill."
I think it would still be free. The new protocol would cost much less in the end.
"What would stop them from getting a new cert after theirs is "revoked"?"
Let's say it takes a week to get a certification. And the servers can be automated so that they can identify spam or virii. So a spammer would only be able to get a few spams through before the cert got pulled. At that point it's not worth it to the spammer to try and get a new cert. Spam relies on sending millions of messages. If you can limit them to only a few thousand the profit motive suddenly dries up.
"instead of just clicking the link at the bottom, or sending us an email, to remove themselves they submit our domain name to an RBL."
You get around this by building the RBL into the server. The end user must click on the link, which allows the cpp mail server to verify things. Did the user ask to be taken off your list? Did you send them mail after that? If yes and yes, you screwed up, and your cert is in jeopardy. It's a closed system that can verify itself. You take the idiots out of the loop.
Obviously it's not as easy as that. But I think the idea has legs.
"I dont think we can take away the anonymity of the Internet because thats one of its greatest strengths."
You would still be able to send anonymously over a non-anonymous system. My idea sort of breaks down here. But I think you'd be able to get a free, anonymous email account and then count on the cpp server to close the account if it got used for spam.
"If you start requiring certs to do things on the web you will start to break it down and it will only be for those than can afford those certs."
This hasn't stopped SSL. Hasn't stopped domain registrations. And placing the burden on the mail servers, who will be one of the main beneficiaries when it comes to less admin and less bandwidth, and I think you'd have plenty of buy-in.
"If you require them to get certs so their family and friends can send and receive email you could prevent that person, and every person they are providing services to, from being on the Internet."
They already pay for bandwidth and domain registration. And they'd probably pay less if they didn't have to transport spam and worms to the people they're serving mail to, admin mail filters, etc. It's not like it's free now. And it's not like there isn't an admin burden. What I'm talking about should make that cost and burden less.
Look at it from my perpective. I host all my mail/ftp/www services out of my house on a residential DSL line. I have plenty of bandwidth to have the thousand or so messages and hundred MB or so of traffic a day. Essestially this is free to me since I am already going to be paying for the DSL line. The added services do not cost me anything extra, other than my time.
If I have to go out and get a cert every couple years to be able to send email, thats going to be a pain. I dont use SSL right now since the cost of the cert is beyond what I would get out of it.
Maybe, if the cost of a cert was only a few bucks, it might not be that bad. But I imagine the cost of the cert is going to be a few hundred bucks.
The reason we have so many domain registrations going on is because it's only $10 to get a domain name. If it was still around $150 to get a domain name there would be far less of them.
I would have to see a breakdown of mail servers and who runs them. If it turns out that the majority are by big companies then it would be more feasible, but in my area of the country most of the ISPs are Mom & Pop shops that are barely getting by. Adding extra certs might bring them over the edge. Plus you would have to make sure it wasnt a host name based cert. It would have to be an machine cert of some kind. A mail server can host thousands of domains and a cert for each one would be cost prohibitive.
I like the idea of making certs hard to get. That would deter them. The auto identifying spam would be a tricky one though. There isnt much difference between legitimate bulk email and unsolicited spam. False Positives could become a problem.
And I was figuring the cert cost would be close to $25 for mom & pop mail servers.
There will be false positives in any automated hueristic/bayesian/scoring system, its how they are handled that is the tricky part.
I'm sort of thinking that a great deal of the work could be done by the end user and the agency generating the email. And then the server could track the interactions and take automatic action.
So If I get something from House Of Blues which I consider spam, I can click a link in the email which will tell them never to send it to me again. That will send the opy-out request to House of Blues and also the certification authority where it can be logged. If they then send me another spam and I click on the opt-out a second time, the cert authority will have a record that House of Blues is spamming me. This is something they could (in theory) take action on without human intervention.
If that imaginary system can be built, then spam goes away very quickly.
This breaks down once you start throwing real world issues at it. For instance, what if I opt out of HOB emails, but then I buy a ticket which I want an email confirmation for? But I think even that can be integrated into the system as exception handling.
And keep in mind that spammers require massive numbers of messages to make a profit. This activity would be easy to spot when you looked at the opt-out logs. If the certification authority sees that a server is getting 10% opt-out rates over the course of an hour, then it's pretty obvious that mail server is spamming.
a) It's already illegal.
b) That hasn't slowed it down one tiny bit.
We need a new mail protocol that has a built-in process to stifle such activity. I think my proposal does that.
To add your comment, click here.
|