Sam’s Notebook - Software Install - MSMTP For Email Notices of Bash Job Completion on Emu (Ubuntu)

After I finally resolved the installation of PB Jelly on Emu (running Ubuntu 16.04), I’ve had a PB Jelly assembly running for the past two weeks! I’ve gotten tired of checking on its status (i.e. is it still running?) every day, so I dove in and figured out how to set up Emu to email me when the job is complete!

To get this going, I mainly followed this msmtp ArchWiki guide., but here are the specifics of how I set it up.

Step 1. Installed a mail server:

sudo apt-get install sendmail

Step 2. Installed `msmtp`:

sudo apt-get install msmtp

Step 3. Created the following file in my home directory (/home/sam/): `~/.msmtprc`

The original contents of the file for testing were:


       # Example for a user configuration file ~/.msmtprc
       #
       # This file focuses on TLS and authentication. Features not used here include
       # logging, timeouts, SOCKS proxies, TLS parameters, Delivery Status Notification
       # (DSN) settings, and more.

       # Set default values for all following accounts.
       defaults

       # Use the mail submission port 587 instead of the SMTP port 25.
       port 587

       # Always use STARTTLS.
       tls on
       tls_starttls on
       tls_certcheck off
       # A freemail service
       account uw

       # Host name of the SMTP server
       host smtp.washington.edu

       # Envelope-from address
       from emu@uw.edu

       # Authentication. The password is given using one of five methods, see below.
       auth on
       user samwhite

       # Password method 3: Store the password directly in this file. Usually it is not
       # a good idea to store passwords in plain text files. If you do it anyway, at
       # least make sure that this file can only be read by yourself.
       password myuwpassword

       account default : uw

This is a configuration to allow emails to get sent via the Univ. of Washington email servers. Yes, I currently had UW password saved in this file, but will be addressing this issue below.

Step 4. Changed permissions on `~/.msmtprc` to be readable/writable only by me (important, particularly if you’ve stored your password in this file!):

chmod 600 ~/.msmtprc

Step 5. Assigned `sendmail` to use `msmtp` with the `set` command (this sets the following command as a positional parameter by adding to the `/etc/mail.rc` file:

echo “set sendmail=/usr/bin/msmtp” | sudo tee -a /etc/mail.rc

This command pipers the output of echo to sudo and uses tee -a to append to our desired file (/etc/mail.rc).

Step 5. Send a test email:

echo “Job complete!” | msmtp myuwemail@uw.edu

That will send an email with no subject and the body of the email will contain “Job complete!”.

That’s the basic set up for this.

To use it in your workflow, you’d append that command to the end of any Bash command or in a separate Jupyter notebook cell that is queued to run after a previous cell completes it’s job.

Example:

echo “This counts as a command”; echo “Job complete!” | msmtp myuwemail@uw.edu

This will run the first echo command. When that finishes, then the email command will run. You can get fancy and have different emails in response to how the running program exits (i.e. fails or is successful) and send different email responses, but I’m not going to get into that.

Anyway, not bad! However, we want to make this a bit nicer and more secure.

Improve security:

Step 1. Generate a GPG Key:

Follow the instructions under the Creating an Encryption Key section at this link.

DO NOT CREATE A PASSWORD! JUST HIT ENTER WHEN AT THAT STEP.

Technically, this is does not follow proper security protocols, but this is better than having a plain text password, and setting it up this way is the only way the mail program will send without prompting the user for a password (which kills the automation we’re trying to achieve).

Step 2. Create an encrypted password file:

gpg –encrypt -o ~/.msmtp-password.gpg -r youremailaddress -

After entering that, type your UW email password(NOTE: You will not receive a new prompt, so just type it in), and then Enter. Then, press Ctrl-d.

Step 3. Add the following line to your `~/.msmtprc` file:

passwordeval “gpg –quiet –for-your-eyes-only –no-tty –decrypt ~/.msmtp-password.gpg”

Here’s what the file looks like now:


       # Example for a user configuration file ~/.msmtprc
       #
       # This file focuses on TLS and authentication. Features not used here include
       # logging, timeouts, SOCKS proxies, TLS parameters, Delivery Status Notification
       # (DSN) settings, and more.

       # Set default values for all following accounts.
       defaults

       # Use the mail submission port 587 instead of the SMTP port 25.
       port 587

       # Always use STARTTLS.
       tls on
       tls_starttls on
       tls_certcheck off

       # Email account nickname
       account uw

       # Host name of the SMTP server
       host smtp.washington.edu

       # Envelope-from address
       from emu@uw.edu

       # Authentication. The password is given using one of five methods, see below.
       auth on
       user samwhite


       # Password method 2: Store the password in an encrypted file, and tell msmtp
       # which command to use to decrypt it. This is usually used with GnuPG, as in
       # this example. Usually gpg-agent will ask once for the decryption password.
       passwordeval    "gpg --quiet --for-your-eyes-only --no-tty --decrypt ~/.msmtp-password.gpg"

       account default : uw

Step 4. Change permissions on `~/.msmtp-password.gpg` so it’s only readable/writable by you:

chmod 600 ~/.msmtp-password.gpg

Step 5. Send a test email like before:

echo “Job complete!” | msmtp myuwemail@uw.edu

That’s it for security.

Add a subject to the emails:

Step 1. Create `~/.default_subject.mail` and add the following lines to the file (substitute your own email address):


To: myuwemail@uw.edu
From: [EMU]
Subject: JOB COMPLETE!

Feel free to change the Subject and/or From info to whatever you’d like.

Step 2. Send message using `~/.default_subject.mail`:

cat ~/.default_subject.mail | msmtp myuwemail@uw.edu

To use this in your workflow, you’ll do just like before (but using the command immediately above) and append to the end of any Bash command.

Make it short & sweet

Appending those lines is going to be difficult to remember, is annoying to type out, and displays your email address (particularly if using a publicly hosted Jupyter notebook like most of us in lab do). Here’s a nice way to remedy that.

Step 1. Add email address as variable in `~/.bashrc`:

Add the following lines to the end of your ~/.bashrc file:


# Email address
export EMAIL=myuwemail@uw.edu

Your email address is now saved in the variable $EMAIL. You will need to use the following command to load that information:

source ~/.bashrc

Verify that it worked:

echo “$EMAIL”

That should spit out your email address and is ready to be used!

Step 2. Add alias for full mail command to `~/.bash_aliases` file:

echo “alias emailme=‘cat ~/.default_subject.mail | msmtp “$EMAIL”’” >> ~/.bash_aliases

Verify that it worked:

source ~/.bash_aliases

emailme

So, from now on, all you have to do is append the command emailme to the end of any Bash commands and you’ll get email when the job is finished!!! You can edit Steps 1 & 2 to use a variable other than “EMAIL” and an alias other than “emailme” - use whatever you’d like.

Step 1. Installed a mail server:

Step 2. Installed msmtp:

Step 3. Created the following file in my home directory (/home/sam/): ~/.msmtprc

Step 4. Changed permissions on ~/.msmtprc to be readable/writable only by me (important, particularly if you’ve stored your password in this file!):

Step 5. Assigned sendmail to use msmtp with the set command (this sets the following command as a positional parameter by adding to the /etc/mail.rc file: