How to Authenticate and Encrypt E-Mail with Server Admin in Lion Server

By John Rizzo from Mac OS X Lion Server For Dummies

Lion Server lets you require clients to encrypt passwords when they sign in to the e-mail server; it can also encrypt e-mail messages with the Secure Sockets Layer (SSL) standard. Password authentication is useful if users connect to your e-mail server over the Internet from home or when traveling. Mac OS X Server offers different methods of authentication because not all mail clients support the same methods.

In Server Admin, you can find these on the Advanced tab of the Mail Settings window:

In Server Admin, click the triangle to the left of your server to expand the list of services.
Click Mail from the list; then click the Settings icon in the toolbar.
Click the Advanced tab in the upper right and then click the Security tab in the second row of tabs.

Here, you can set encryption for authentication (usernames and passwords) and for e-mail messages.

Secure mail authentication

Server Admin and the Configuration Assistant both give you options for password encryption for SMTP and IMAP/POP. Which you choose depends on what your e-mail clients support. You can choose multiple authentication methods to support multiple e-mail clients.

It’s a good idea to disable any authentication method that your clients aren’t using. If you need only one type for all your clients, use that. Using only one type of authentication requires clients to use it.

Here’s the lowdown on your choices:

Kerberos and CRAM-MD5 are the most secure authentication methods. To use Kerberos for mail, you need Kerberos authentication in Open Directory or on another server. Of the two, Apple recommends Kerberos.
APOP is an encryption type used only for POP clients.
Login, Clear, and Plain are unsecure authentication methods that send passwords unencrypted. If you choose these in addition to the more secure authentication methods, clients that don’t have the more secure methods set up are allowed to log in without encryption. If you deselect these unencrypted options, these clients can’t log in until they’re configured for encryption.

In Server Admin, you’ll find these settings in the Server Admin Settings window. In the Configuration Assistant, you can configure these settings in the Security window.

Secure e-mail messages with SSL

The preceding methods encrypt only passwords and usernames. You can also encrypt e-mail itself with the Secure Sockets Layer (SSL) section in the lower part of the window. For POP and IMAP, SSL encryption is between your server and your clients. For SMTP, SSL encryption is between your server and other e-mail servers.

To use SSL, click the pop-up menu for SMTP or the menu for IMAP and POP. If you choose Require, the mail service won’t connect if the client or other Mail server isn’t supporting SSL.

If you choose Use, SSL encryption is used if a POP or IMAP client asks for it. If a client isn’t set up to request an SSL connection, the Mail service can still deliver mail to that client. The Use setting works the same with SMTP and other Mail servers: For Mail servers that don’t request SSL, Lion Server’s mail service sends mail unencrypted.

In the pop-up menus to the right, you can choose to select a certificate or to not use a certificate. You can use the Server app to import a certificate from a certificate authority or create a certificate-signing request and a keychain. To get there in the Server app, select your server under Hardware in the sidebar and then click the Settings tab.