Smart quotes in MimeMessage not displaying correctly in Outlook
Our application takes text from a web form and emails it to the appropriate user. However, when someone copies / pastes in the infamous "smart quotes" or other special characters from Word, things get hairy.
User enters
he said hello to me - isn't that nice?
But when the message appears in Outlook 2003, it looks like this:
he said hi to me, is it nice?
The code for this was:
Session session = Session.getInstance(props, new MailAuthenticator());
Message msg = new MimeMessage(session);
//removed setting to/from addresses to simplify
msg.setSubject(subject);
msg.setText(text);
msg.setHeader("X-Mailer", MailSender.class.getName());
msg.setSentDate(new Date());
Transport.send(msg);
After doing a little research, I figured out that this is probably a character encoding issue and trying to move things to UTF-8. So I updated the code this way:
Session session = Session.getInstance(props, new MailAuthenticator());
MimeMessage msg = new MimeMessage(session);
//removed setting to/from addresses to simplify
msg.setHeader("X-Mailer", MailSender.class.getName());
msg.addHeader("Content-Type", "text/plain");
msg.addHeader("charset", "UTF-8");
msg.setSentDate(new Date());
Transport.send(msg);
It brought me closer, but not cigars:
he said hello to me - don't you like it?
I can't imagine this is an unusual problem - what am I missing?
a source to share
Is the page with your form also using UTF-8 or some other encoding? If you don't specify the encoding of the webpage, the format of the data going into your script is guessing.
Edit: The encoding in the post should be set like this:
msg.addHeader("Content-Type", "text/plain; charset=UTF-8");
since charset is not a separate header, but the Content-type option
a source to share
I would make sure the data received from the browser is correct - dump the unicode codes and check them against charts
public static void printCodepoints(char[] s) {
for (int i = 0; i < s.length; i++) {
int codePoint = Character.isHighSurrogate(s[i]) ? Character
.toCodePoint(s[i], s[++i])
: s[i];
System.out.println(Integer.toHexString(codePoint));
}
}
For example, the symbol DOUBLE LEFT QUOTATION MARK ( “ ) is character U + 201C.
It's been a long time since I used the post API, but the MimeMessage.html.setText (text, charset) method might be worth looking at. The documentation for setText (String) says it uses the default character set (probably windows-1252 if you are using Windows English / Latin-1).
a source to share