Email: Why Must Something So Useful Suck So Much

Yep. It’s time to dust off the old blog for a rant.

Wikipedia’s Email article claims that email was first used back in 1965. So here we are 42 years later and no one can guess how the hell that email you just sent will appear in other people’s email clients.

It is so bad I can’t even accurately guess how it will appear in my own sent folder viewed in the same email client I just composed it in. The software hate all email programs inflict on users is just brilliant. Here we have software that people need to interact with on a daily basis and this software is designed in a way where the software appears to be doing what you want it to do then behind your back it screws up the output.

I can only assume that some evil bastard developed algorithms that methodically inserts seemingly random line feeds, randomly decides how to wrap text, and otherwise decides to arbitrarily mangle carefully formatted text. All this in a conspiracy to make the user look like yet another computer illiterate moron idiot to his peers. Except all his peers are running the same software and the software is doing the same crap behind their backs. And every email program seems to have it’s own evil bastard algorithm!

Mailing lists are a great example of this. Internal mailing lists and Linux oriented mailing lists tend to avoid seeing this issue because everyone tends to be running the same or a small selection of the same email programs. Look at a public mailing list where this isn’t the case and you’ll see formatting issues on every other email. Some minor, some fairly major.

We all see this crap happen every day and we all accept it. Many of us have tried clinging to some rules to try to make the world a sane place. Some of us adopted these rules like a minor religion and unconsciously judged random strangers based on how well they adhered to them. Your badly outputted email was obviously the result of not following “The Rules”!

Some of these rules are as follows:

• Never send HTML email. Ever. Plain text is the way to go.
• MIME is evil. Don’t send it, defang it.
• Thou shalt wordwrap at 72 characters.
• Avoid attaching text as an attachment.
• Never top post! Only idiots top post!

I’m sorry to say that I’ve come to the conclusion that those rules are all useless. Today. In reality.

And here is why:

Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Plain text is no longer plain text. It has not for years and I’ve only come to realize it just now. Line feeds have meaning above and beyond this is a new line! In Thunderbird they act pretty much like a paragraph tag in HTML. Thunderbird makes this all painfully obvious. Thunderbird sucks.

Thunderbird commits countless sins when handling “plain text” emails. The configuration UI lets you set a max plain text line length or width. It offers a Rewrap menu option you can use to make sure text is wrapped at the appropriate width and line breaks are added where needed. When typing in a composition window it looks like the text width setting is being observed and you are composing a plain text email that will look as it appears in the window.

WRONG! Behind your back it is doing this flowed text crap and trying to treat line breaks as paragraph markers or some HTML inspired crap. Worse yet if you are trying to carefully format a mail and use the enter key quite a bit for formatting and add in a bit of quoting and use of the Rewrap menu option then your email is a wreck. A wreck you won’t realize happened until you hit send then go look at it in a coworkers client or even from the Sent folder of the same client you just sent it from.

So with this format=flowed garbage in use thanks to RFC 3676 we now have two pools of email clients. Those who send text out “flowed” and those who don’t. The ones that do implement flowed formatting seem to implement it differently. The ones that don’t handle “flowed” but receive it can render it differently.

Those who send flowed text to others with email clients that support flowed see paragraphs that format themselves much like they do in HTML. You can resize windows and it’ll reflow and “look nice”.

Those who send flowed text to non-flowed clients potentially send a mess of unreadable garbage. I’ve seen it before and the formatting of your carefully prepared email looks can very easily look like vomit.

Those who use non-flowed clients who send text to flowed clients can have issues in flowed clients. From what I’ve experienced non-flowed clients will not escape this issue. For one thing a hard wrap limit does not “flow”. This leads to trivial GUI display issues but that is really the minor thing. The problem is livable on desktops but really becomes much more obvious when dealing with devices like cellphones. Email support on cellphones blows in general so this isn’t too shocking.

The thing that annoys me the most about non-flowed mail clients is handling quotes. I’ve yet to use an email client that did not require manual massaging of quotes when nested quotes cause text to overflow the fixed width. The usual strategy for this is to avoid nesting whenever possible. Normally a good idea but nested quotes can be useful and getting rid of them is really a workaround for a limitation in the program.

So with that said…

I often need to send email that mixes normal paragraphs of text with precise output from programs. The output from programs often exceeds common line length settings and thus can’t be wrapped.

The only ways I’ve found to do this so it appears as expected in as many different email clients as possible is to do one of the following:

• Put the format sensitive text in a text file and attach it with the appropriate MIME type and have the user view the attachment either inline or in a seperate text editor.
• Put the text in a word document or PDF. This approach seems to have won out among those outside technology jobs long ago.
• Be a man, admit you were wrong all those years, and just use HTML email and move on with your life.

These are not solutions. They are workarounds. But I’ve given up. I’ve been beaten down and I accept it. Please let the brainwashing begin if it’ll make the pain stop. Give the choice of the above then I’ll pick… HTML.

HTML was designed for cases like this inside a web browser. It isn’t a logical leap to bring it to your email client. Basic HTML tag use gives you all the formatting tools you need with less of the overhead. No this doesn’t mean use CSS, inline images or Outlook stationary. And no it doesn’t mean ever imbed IE. The use of those is still in the realm of hate and at least the team who worked on Outlook 2007 realized that and fixed most of it.

HTML gives us <P> and the winner… <PRE> The PRE tag saves us, seriously. I can now tell my email client the logical format of my email and tell it which part in particular not to screw around with. And every one I’ve tried listens to me! I can use things like bullet lists without doing cute little ASCII art tricks that get butchered by every other email client. It is the closest thing out there to “It just works!”

Now obviously this choice doesn’t get anywhere close to fixing all the hate email programs inflict on us but for me it fixes some of the hate in my common day to day use clients.

Oh and for those who cling to text email I’m going to make a horrific suggestion… Evolution is the only program I’ve seen that approaches getting it right. Yes, that bloated slow thing that takes forever to read any IMAP folders with more than a handful of messages. It still has issues and doesn’t escape many of the problems above. But at least the UI works somewhat predictably. Will someone please port the composition window over to Thunderbird? Please?

Now some of you may be going to yourself “well I don’t have this problem in “. Yes, you do or will soon. You may very well be a victim of this mesh of slightly incompatible email clients sending and receiving from each other and not even realize it. We are not all going to standardize on one specific email client implementation anytime soon. Of course if you don’t care about it then that is your perfectly valid opinion. I obviously cared enough to rant about it.

Oh, and Top Posting is still annoying as hell for those of us who try to keep a summary of responses in a long email based conversation for their own sanity. It is still annoying all these years later.

Update: It was actually a LWN article with a complaint from a Linux kernel developer that actually helped me down this road…