Update supported auth mechanisms for IMAP and SMTP for standards compliance

Ideas
1

Abstract

Update smtplib and imaplib to use PLAIN SASL as the primary authentication mechanism, deprecate the legacy LOGIN and CRAM-MD5 mechanisms, and expose a SASLprep helper consistent with current RFC standards. This proposal is targeting Python 3.16.

Motivation

The standards for authentication mechanisms relating to SMTP (RFC 2554) and IMAP (RFC 2060) have been updated since the original Python implementation to gain support for Simple Authentication and Security Layer (SASL) mechanisms.

LOGIN and CRAM-MD5 are now considered legacy. Both SMTP and IMAP clients now MUST support PLAIN SASL.

OAUTHBEARER SASL has also seen adoption in widely used SMTP and IMAP servers. This proposal does not consider updating poplib because it is called “obsolescent” in the Python documentation.

Prior Discussions

@barry asked me to move the discussion here for wider input, from my SMTP PR. Since then I’ve done a lot of digging through RFCs and experimentation to arrive at this proposal.

References

RFC 2554 for SMTP authentication is obsoleted by RFC 4954 (2007), which requires client support for PLAIN SASL.

RFC 2060 for IMAP is obsoleted by RFC 3501 (2003), which requires client support for PLAIN SASL, and states that “The LOGIN command SHOULD NOT be used except as a last resort (after attempting and failing to authenticate using the AUTHENTICATE command one or more times), and it is recommended that client implementations have a means to disable any automatic use of the LOGIN command.”

PLAIN SASL was first described in RFC 2595 and formally defined as a standalone mechanism in RFC 4616 (2006). RFC 2595 specifies: “Non-US-ASCII characters are permitted as long as they are represented in UTF-8. Use of non-visible characters or characters which a user may be unable to enter on some keyboards is discouraged.” RFC 4616 states that the “authcid and passwd productions are form-free and will be verified by the server using SASLprep” (RFC 4013). It further states that “the NUL (U+0000) character MUST NOT appear in authzid, authcid, or passwd productions”.

In the IANA SASL registry, LOGIN is listed as OBSOLETE and CRAM-MD5 is listed as LIMITED.

The updated LOGIN SASL draft (draft-murchison-sasl-login-00, 2003) states that “the LOGIN SASL mechanism SHOULD NOT be used by a client when other plaintext mechanisms are offered by a server.” It also states that “only US-ASCII printable characters SHOULD be used in the username and password to permit maximal interoperability.”

The updated CRAM-MD5 SASL draft (draft-ietf-sasl-crammd5-10, 2008) states “the CRAM-MD5 mechanism is intended to have limited use on the Internet. The mechanism offers inadequate protection against common attacks against application-level protocols” and “is prone to interoperability problems”. It states that "The client SHOULD prepare the user name and shared secret strings using the SASLprep [RFC4013] profile of the Stringprep [RFC3454] algorithm. The resulting values SHOULD be encoded as UTF-8 [RFC3629]strings.

XOAUTH2 is the only non-plaintext auth mechanism supported my Microsoft and Apple (confirmed using IMAP4_SSL), but it was never standardized and is listed as OBSOLETE in the registry. We could consider adding XOAUTH2 as an example in the docs.

Specification

  • Update SMTP and IMAP4 to use PLAIN SASL as the primary auth mechanism.

  • Deprecate usage of CRAM-MD5 and LOGIN mechanisms.

  • Update IMAP to use SASLprep for CRAM-MD5.

  • Document best practices, such as using TLS before using PLAIN authentication, or using a more secure mechanism if available.

  • Document how to use other standards-based auth mechanisms and give an example like OTP SASL RFC 2444.

Open Questions

  • Should we support OAUTHBEARER SASL? It is supported by Gmail, Yahoo, and AOL email servers (confirmed using IMAP4_SSL). Typically one would use a dedicated library to fetch and use the bearer token, but we could add methods to IMAP4 and SMTP for completeness. An alternative would be to add examples of using OAUTHBEAERER in the IMAP4 and SMTP docs, as opposed to the OTP recommendation above.

  • Should we expose the saslprep() function? We could put in an existing namespace like email.utils(), since it would only be used for email protocols in the standard library. Of registered SASL mechanisms on the IANA registry, only PLAIN, CRAM-MD5, and SCRAM-SHA-* mention the use of SASLprep. SASLprep itself has been obseleted by the PRECIS framework (RFC 7613 2015), so it is unlikely to be used by future protocols.

Implementation

SASLprep helper function

  • Add a Lib/_saslprep.py that has been donated by MongoDB (my employer) from pymongo. It is licensed under Apache 2.0.

  • Use the saslprep() function to prepare username and password for PLAIN SASL. While not strictly required by the client, it enforces the recommended client restrictions in the specification and ensures that servers that predate SASLprep will not be adversely affected.

  • Add tests that use Unicode credentials and saslprep() in the mock server implementations to ensure standards compliance.

SMTP updates

  • Update SMTP.auth_plain() to apply saslprep() and return bytes.

  • Update SMTP.authenticate() to only encode str objects that are returned by authobject() calls.

  • Deprecate the use of auth_login() and auth_cram_md5().

  • Since auth_cram_md5() only accepts ASCII, there is no need to use saslprep().

  • Update login() to only use CRAM-MD5 and/or LOGIN if PLAIN is not available.

IMAP4 updates

  • Add IMAP4.login_plain() that uses saslprep() and the AUTHENTICATE PLAIN command.

  • Deprecate login() and login_cram_md5().

  • Update login_cram_md5() to use saslprep() as recommended by the standards, since it supports Unicode credentials.

1 Like
2

There’s really nothing wrong with using LOGIN over TLS. AUTHENTICATE PLAIN doesn’t gain you more security compared to LOGIN. I don’t think it should be deprecated in Python’s stdlib libraries.

BTW: The quote you mention was taken from a paragraph in the RFC which is dealing with unencrypted connections:

Note: Use of the LOGIN command over an insecure network
(such as the Internet) is a security risk, because anyone
monitoring network traffic can obtain plaintext passwords.
The LOGIN command SHOULD NOT be used except as a last
resort, and it is recommended that client implementations
have a means to disable any automatic use of the LOGIN
command.

The relevant quote would have been this one, a bit further down, suggesting to use a TLS protected connection:

Server sites SHOULD NOT use any configuration which permits the LOGIN command without such a protection mechanism against password snooping.

Also note that the RFC does not deprecate LOGIN in any way for TLS protected connections (either via STARTTLS or direct TLS connect). Indeed, it is probably the most widely available login method.

That said, I’m definitely +1 on adding support for more login methods such as AUTHENTICATE PLAIN.

4 Likes
3

Hi @malemburg, thank you for the feedback! Part of the reason I advocated for not using LOGIN when PLAIN is available is that if we change only auth_plain() to handle Unicode, then the error behavior is awkward. If the user provides incorrect Unicode credentials and PLAIN fails, then login() will try a LOGIN and raise a UnicodeDecodeError because it would be the last error encountered. We could raise the last encountered server error, and then fall back to the last encountered error.

4

Agreed. If PLAIN is available, it’s probably a better option to use, but I don’t see a need to deprecate LOGIN.

FWIW: Unicode support for credentials is not widely supported and comes with its own set of challenges (e.g. whether or not to normalize, using which form, etc.). Not only for IMAP / SMTP, but in general. Many systems don’t even support non-ASCII code points.

5

Fair point. We should document that it is possible per the standards, but that you should verify first that your server supports Unicode credentials.