At work our application reads email from a number of POP3 servers using JavaMail. If you trigger the POP3 server’s timeout the session is closed silently und JavaMail does not complain on session.close. In the result, the message is not deleted and will be processed again on the next turn of the background processing task. To fix this, you need to check the state of the POP3 folder after processing each message. Read on for more details on this issue.

Problem

Some of the mails our application is getting are not only „messages“ meant to be read by humans but contain structured information that is processed. This means DB operations. Processing takes not more than a few seconds – unless the Oracle DB optimizer chooses to take a bad SQL execution plan. At least this is what happened to us last week: the processing of a structured message took like 20 minutes.

The symptom then was that after the application was finished processing the email, it did not deleted the message from the server. It then started processing the mail in the next turn. As we found out the POP3 server (qpopper) has configured a timeout of five minutes. After that it answers with an „-ERR“ status code to the next client command. The code we used to process the mail was roughly this:

[sourcecode language=“java“]
try {
mail = pop3Folder.getMessage(mailIndex + 1);
// … do something with mail …
} finally {
mail.setFlag(Flags.Flag.DELETED, true);
}
folder.close();
[/sourcecode]

When setting the delete flag, JavaMail sends the DELE command. That was answered with „-ERR“ (from POP3 protocol exchange):

C: STAT
S: +OK 1 1866
C: RETR 1
S: ...
C: DELE 1
S: -ERR POP timeout from some.host
C: QUIT
S: +OK Pop server at some.host signing off.


and the session is gone. Using session.close did not complain. So no exception occured. In fact, if an exception had occured we already had an compensation mechanism that would have prevented processing the same mail again. Also it would have pointed us to the problem more directly.

Solution

The solution is to check the protocol state using folder.isOpen after processing a mail.

If that folder is not open, the mails you have so far processed have not been deleted because in POP3, mails are deleted when closing a folder. So open the folder again and delete all messages (messageRead) that you have processed before. Then carry on, i.e. close the folder.

[sourcecode language=“java“]
boolean isOpen = folder.isOpen();
if (!isOpen) {
folder.open(Folder.READ_WRITE);
final int mailCount = folder.getMessageCount();
for(int i = 1; i <= Math.min(messageRead, mailCount); i++) {
Message mail = folder.getMessage(i);
mail.setFlag(Flags.Flag.DELETED, true);
}
}
// folder.close in „normal“ code flow
[/sourcecode]

The folder.isOpen sends a NOOP command and evaluates the response, so the „-ERR“ server code is not gone unnoticed.

Doing it right

POP3 was designed to download mails rather quickly. A better solution is to fetch emails from POP3, store it somewhere and only then process them („offline“).