UTF8 to ISO plugin V1.60 for Eudora (11/2006)

(Written in Delphi 4, sourcecode lost several years ago)
Eudora E-Mail client for Windows V4 or better required.

Freeware by Daniel Kuhn


Download from http://www.windharp.de/software/utf8iso.zip
This plugin will not be developed any further, since I stopped using Eudora. I suggest you try thunderbird instead of eudora.

What does it do?

Nowadays more and more e-mails are encoded using the unicode encoding UTF-8. Basically this is an advanced charset, including most symbols existing in the world. For example you can write Japanese and some similiar charsets with unicode.

This plugin checks all incoming messages (and can be invoked manually) for UTF-8 encoding and tries to decode as much as possible. It is naturally limited, because Eudora can only handle the characters in the ISO8859-1 charsets, so for example Japanese characters can't be decoded.

To check if a message is UTF-8 encoded, the plugin looks at the message header "CONTENT-TYPE:" and checks if it contains the strings "CHARSET" and "UTF-8". If those strings are not found, the message stays untouched in the automated mode. In manual mode, this check is turned off to give you the chance to decide yourself if a conversion is needed. Only message parts of the type "text/plain" and optionally "text/html" are processed, so binaries stay untouched. Multiline message headers are supported.

Why is this plugin needed?

The e-mail standard says that all e-mail programs modified 1999 or later must (read "must"!) understand UTF-8 encoded e-mails. The people at Qualcomm didn't manage to make Eudara UTF-8 compliant, though. So e-mails you receive that are in UTF-8 may contain funny characters. Change to a different client? No, Eudora is fine. It is among the best e-mail clients I have ever seen. Except for this bug, that is...

Installation

Very simple. Really. Just copy the UTF8ISO.DLL into your Eudora\Plugins directory. The only thing you need to do now is restart Eudora, the plugin is loaded automatically and starts working once new e-mails are received.

Configuration

Starting in V1.30 this plugin has a configuration menu. You will be able to disable all additional features I add to the plugin for compatibility reasons. You can reach the menu via the Special / Message plugins Settings... menu (see Image 1). Mark the UTF8->ISO entry and click settings (see Image 2).

Image 1: Setting Menu

Image 2: Choose Plugin

Image 3: Setup Screen

Currently the following options are available (see Image 3):

• Decode "=C1=A0" Encoding
  According to a report by Pekka Haapalainen the Nokia 9500 Communicator encodes extended character in the "=C9=A1=09" format. I can imagine that some people might not want those characters to be converted, so you can disable this feature if you like to.
  
  
• Process HTML-Messages
  As a result of several requests, I have chosen to add support for HTML messages. Because HTML can be difficult, and I am unable to fully test this feature, this feature is disabled by default. If you want UTF8ISO to convert HTML messages, you will have to activate this feature here.
  
  
• Decode MIME64 Encoding
  I still don't understand why some mail-clients encode text/plain messages in MIME64, but Peter Barneveld (Netherlands) showed me an example where this is the case (OWA). So I implemented a MIME64 decoder, that can be disabled here.

How to use UTF8ISO

Inbound filtering is fully automated. Therefore no interaction is needed after you installed the plugin (See the above section about configuration about possibilities to configure UTF8ISO's behaviour). You may convert old messages (received before the installation), too. Just open the message and execute Edit / Message-Plugins / UTF8 -> ISO. The result will be shown and you will be asked asked for confirmation when you close the window.

Tip: Due to a bug Eudora introduced in V6, a message converted in the preview pane can not be saved. Because of this, you need to doubleclick the message to open it, engage the editing mode (pencil icon), and start the manual conversion afterwards. Starting in V7.x you need to end the editmode by another click on the pencil icon before closing the mail.

Image4: Manual usage

Features and Limitations

• No configuration needed, working out-of-the-box.
• Tested and reported working for special characters of: German, French and Scandinavian languages
• The "UTF-8" bit of the message is not removed. If I would remove it, the "Send to browser" trick would not work to view messages that could not be fully decoded. Mixed messages (say Japanese and German) have the german part displayed correctly in Eudora and the Japanese part displayed correctly when sent to browser.
• All messages containing the searched string in their header are converted. I don't check if it really is UTF-8, I just read the label. That might cause misconversions, but none were reported so far.
• The "subject" line Eudora shows in its table is not converted. Due to limitations in the API Eudora provides, those headers can't be changed. If you look at the headers inside the mail you can see it converted there.
• Only characters in the ISO8859-1 charset are supported by Eudora. That is nothing I could change, that is a limitation of Eudora. So please donb't ask me if I can add support for languages that don't use ISO8859-1.

How to uninstall the plugin?

What? Uninstall it? This superb addon? No way...
Okay, it is as easy as the installation: Simply close Eudora and delete the file UTF8ISO.dll (and UTF8ISO.INI if it exists) from your Plugins-folder.

Version history

V1.00 - March 2004

V1.10 - September 2004

• Improved UTF8-detection: The old version looked for fixed strings, this one looks for necessary substrings. Most messages will be identified correctly now
• Bugfix: Some unprocessed messages had parts of their headers displayed inside the message. Well, I found out how to workaround a bug in Eudora now :-)
• Speedup: I figured out how to use another part of the EMAPI of Eudora, speeding the whole thing up a lot.

V1.20 - October 2004

• Manual Conversion: Fixed a bug that prevented the manual conversion from working.
• Documentation: Some people had difficulties to figure out how the manual conversion is started. Added some more information to the documentation.

V1.30 - February 2005

• Feature: Configuration menu added. Only one option yet, but this was necessary.
• Fix: Mails with character encoding (for example sent by Nokia 9500 Communicator) are now converted. This could lead to other problems, so this feature is optional. Thanks for reporting this bug goes to Pekka Haapalainen (Finland) who submitted a very detailed and helpful bug report.
• Fix: Mails that had the Content-Type split over several lines were not translated. This is now fixed. Thanks to Gorka Eguileor Gimeno for sending in a bug report.
• Documentation: Added German documentation, general description of UTF-8 encoding, more details about the limitations of this plugin, "How to Report a bug".

V1.40 - June 2005

• Feature: The plugin is now able to handle HTML mails. This feature has not been thoroughly testes, so it has to be manually activated in the configuration menu.
• Feature: MIME64 encoded text messages can now be handled. This feature can be disabled in the configuration menu.
• Fix: Mails that contained single "=XX" characters at the end of a line made the Plugin crash and the content of the message was removed.
• Fix: There was still an issue with mails that had their content type split over several lines.
• Fix: The Plugin now has the unique Plugin ID 5263 which was assigned to me by Qualcomm. This might solve problems when Eudora would not start with my plugin installed, but there might be other reasons.
• Unresolved Problem: Under Eudora V6.2 the manual conversion works, but the result can not be saved. Discussion with other tool developers shows that this is a bug in Eudora 6.2 and not related to my plugin.

V1.50 - March 2006

• Fix: Ebay now uses some weired kind of extra header which confused my plugin, so V1.4 did not convert all Ebay mails that needed conversion.

V1.60 - November 2006

• Fix: The new header-check introduced in V1.5 did not recognize multipart-messages automatically. Some of them (for example those sent as text+HTML) have not been converted in V1.5
• Fix: Eudora dislikes when the manual conversion changes the length of the email headers. headers remain untouched now.
• Fix: Eudora now converts everything to HTML before engaging a manual conversion. For example <br> instead of a line feed and &nbsp; instead of it's binary equivalent. Identification of headers and conversion of some characters (e.g. á) where affected. Should be fixed now.

Warranty

You are using this plugin at your own risk. I don't take any responsibility for problems that might occur, like for example data loss or corruption of your database. I don't give any warranty that this plugin will work on your system, too. Short: No warranty whatsoever.

How to report a bug?

Found a bug? I am sorry, but there is most likely nothing I can do about it. The source code is lost and I don't use Eudora anymore. Most problems have been fixable by checking the settings menu.

Cool thing that, may I ... ?

• donate something for your work? Sure. Payment is not required (this plugin is free for everybody), but when you use UTF8ISO in a commercial environment, it seems fair, doesn't it?
  Feel free to send money to my Paypal account for the email address given in the header of this file.
• have the source code? Sorry, the source code was lost several years ago.
• contact you with some questions? Sure. As long as they are related to this plugin feel free to send your enquiries via E-Mails (You may use UTF-8 encoding ;-) )

UTF8ISO (c) 2007 - Daniel Kuhn