mu4e v0.9.9 is out!

I few months ago, I introduced mu4e, a modest little emacs mail client that I wrote. It seems many people picked it up, great!

what's new?

I have just released mu/mu4e version 0.9.9. There are quite a few changes, both user-visible and 'under-the-hood'. I've also spent some time on improving the manual (pdf), and I'm quite happy about it.

Some of the more visible new things in mu4e are:

• Support for crypto (decrypting messages, signing them)
• Support for refiling (like mutt, Wanderlust)
• Dynamic folders (instead of hard-coding the sent/draft/trash/refile folder, they can be functions that return different folders based on the message we're dealing with, see the example below)
• Same for the folder to save attachments
• A lot of smaller and bigger UI improvements

Also, the core mu program has seen a lot of improvements (many of which directly improve mu4e as well)

• Better support for non-ascii locales / character sets, such as ISO-2022-JP
• Improved on-line help ('mu help ...')
• Performance improvements for threaded display (~ 25% for 23K messages)

For a more complete list, see NEWS.

dynamic folders

As mentioned, mu4e now supports dynamic folders. Before, you'd set your trash folder to some static string:

(setq mu4e-trash-folder "/trash")

In some cases, you may want to have a bit more flexibility – for example, have a separate trash-folder (or sent-folder, drafts-folder, refile-folder) for private mail and work mail. You can now do something like:

(setq mu4e-trash-folder
  (lambda (msg)
    (if (and msg ;; msg may be nil
          (mu4e-message-contact-field-matches msg :to "me@work.com"))
      "/trash-work"
      "/trash")))

refiling

After I have dealt with some e-mail, I either delete it or move it to some archive folder – refiling. For this, there is now the r keybinding, and mu4e-refile-folder; and a place where dynamic folders really shine:

(setq mu4e-refile-folder
  (lambda (msg)
    (cond
      ;; messages to the mu mailing list go to the /mu folder
      ((mu4e-message-contact-field-matches msg :to
         "mu-discuss@googlegroups.com")
        "/mu")
      ;; messages sent directly to me go to /archive
      ;; also `mu4e-user-mail-address-regexp' can be used
      ((mu4e-message-contact-field-matches msg :to "me@example.com")
        "/private")
      ;; messages with football or soccer in the subject go to /football
      ((string-match "football\\|soccer" (or (mu4e-message-field msg :subject) ""))
        "/football")
      ;; everything else goes to /archive
      ;; important to have a catch-all at the end!
      (t "/archive"))))

How cool is that? After reading my inbox folder, I select all messages (C-x h), press r, and they're all moved to the right refiling folder.

crypto support

mu4e already supported signing/encrypting messages, but now it supports decryption and verifying signatures as well. This was one of the most requested new features. I think it is still a bit rough, but it has been working very well for me.

so…

I think version 0.9.9 is a great new step for mu4e. It already goes far beyond I ever planned to do. I received a lot of suggestions for new features, which is great! I'm not planning to implement all of those, but I will try to make mu4e even more programmable – it should be easy to augment mu4e with your own little elisp-snippets – the Barbapapa principle of software, already so clearly present in emacs itself.

introducing mu4e, an e-mail client for emacs

background

I've haven't written too much here lately. This is because I have spent most of my emacs hacking time on mu4e, an emacs e-mail client. It's only slightly over six months old since its first release, but it has progressed rather quickly! So, I think it is time to give it an introduction to a wider audience. In future posts, I'll go into more detail.

E-mail is a very important means of communication for me; and for purposes of integration, it prefer to do my e-mail with emacs. For a long time, I used mutt with emacs as the editor; a few years back I switched to Wanderlust, and wrote a few articles about it.

There's a lot to like about Wanderlust: it's very featureful, and allows for a lot of customization. Still, there were a few things I was not fully happy with; the most important one was the speed of some operations; another annoyance was the fact that I got rather frequent cache-corruptions when using newer emacs versions.

So, what was I to do? Well, a few years back, I had written an e-mail indexer/searcher, called mu. It takes the messages in a maildir-folder, and allows you to query them. I wondered if I could use mu as the backend for an emacs-based e-mail client. So, I fired up emacs, started writing some code and some later I had a first version of… mu4e.

how does it work?

mu4e is an emacs front-end for mu; when you start it, mu4e connects to a mu server process (started on-demand). It accepts simple commands, and responds with emacs s-expressions, in asynchronous fashion. In practice, all the heavy work is done in this server process, and emacs shows the results when it is ready (usually almost instantly). Emacs does not need to wait for the backend, and things stay snappy, even with tens of thousands (or more) e-mail messages. The only operation that may still require some waiting is sending mail, since that uses the emacs built-in smtpmail, which blocks during its operation.

mu expects your e-mail messages to be stored in a maildir (a one-file-per-message system); you can get these mails there through tools like offlineimap or fetchmail. mu periodically indexes the messages, and stores the results in a Xapian database. A lot of what mu4e does is querying this database – so for example, what you see as 'my inbox folder' in the mu4e-frontend, is in fact just the result of a query of all message that happen to live in that folder. You can query for many things – message sender, subject, date, words in the body and more.

mu4e does not just query messages; it can also move messages between folders, delete them, modify their flags and so on (and afterwards update the database). All 'state' is in the file system, and the database is just a quick way to get to it; if you delete the database and then re-index, nothing is lost, and changes made with other tools (e.g. mutt) are picked up by mu as well.

trying mu4e

Some Linux distributions ship mu4e, but not necessarily a very recent version; and because mu4e is developing quickly, you may want to use the latest official release, or (if you are more adventurous) check out the git repository. You'll need GNU/Emacs 23 or 24, GLib, GMime >= 2.4 and Xapian.

After installing mu4e, you can check the mu4e manual, which should be fairly complete. In fact, one of the goals of mu4e is to make it easy to set up; for that reason, the documentation includes complete examples of how to set things up. The Getting started chapter should get you up to speed quickly.

So, give it a try if you're interested; the mu=/=mu4e project is very open to suggestions (and even bugs reports!), so feel free to put them here or subscribe to the mu mailing list.

a small tour

I'll end this with a number of screenshots, to give you a rough idea of how things look.

The main screen

When you start mu4e (M-x mu4e), you're taken to the main screen, which list the things you can do. The entries should speak for themselves. Note that you can define you're own bookmarks. In general, mu4e tries to make things as easy as possible, and provide auto-completions where possible.

The headers / view split-screen

The results of your queries are shown as a list of headers; if you click one of the headers, mu4e splits the screen horizontally (optionally, you can split the screen vertically, or only show the message view). You can customize the headers shown, re-order them and so on.

The message view supports html message, inline images, and has a mechanism to associate user-functions with messages and attachments (such as 'view in browser' or 'pipe through command'); see the manual for some examples.

The message composition view

Obviously, you can reply to messages, forward them, or compose a new message. For writing and sending messages, mu4e reuses emacs' built-in message-mode. There's support for auto-completing addresses (based on the e-mails you've sent/received before).

There's even (experimental) support for writing you're messages using org-mode, and then sending them as rich-text (html) messages.

parting thoughts

So, this was a short introduction to mu4e, barely scratching the surface, but hopefully conveying the main ideas. The manual discusses things in much more detail, and I'll write more about it in the future.

mu4e is still a young project, but, for what's it worth, I've been using it full-time for over six months, and a growing number of people are doing the same. So, if you're interested, give it try and let us know you think!

Have fun!

update

Regular emacs-fu programming will resume shortly, but for now I'll provide a brief update on some of my projects.

• Recently, I discussed sauron, and event-tracking tool for emacs. I added some new features, fixed some bugs, and got some contributions (yay!)
  • tweak/improve priority handling
  • add a backend for the emacs-24 notification system
  • add settings to make the sauron frame 'sticky', and to hide the mode-line
  • enable dbus-message source outside your session; useful for cron/procmail etc.
  • display events in a tabular fashion
  • add support for John Wiegley's event.el
  • much improved documentation
  • other things…

  Sauron version 0.2 is available through Marmalade (see Package Management Revisited)

• I released version 0.9.8 of mu (e-mail searcher/indexer, previously discussed in Searching e-mails with Wanderlust and mu). Now, for emacs-users I've added mu4e (manual), a new & experimental e-mail client. It won't be big and professional like gnus, but it's fun to hack on, and I've been using it for a few months.

showing pop-ups

Updated: yes, it's %s, not %d Sometimes, it's nice when emacs can warn you when something is happening or should happen. For example, when a new e-mail has arrived, or when there's a meeting in 15 minutes you should attend.

As always, there are different way to do this, but here's what I've been using for while. Various versions of this have been circulating around mailing lists, so I don't know whom to credit with the original idea – anyway, this is the (modified) version that I'm using.

(defun djcb-popup (title msg &optional icon sound)
  "Show a popup if we're on X, or echo it otherwise; TITLE is the title
of the message, MSG is the context. Optionally, you can provide an ICON and
a sound to be played"

  (interactive)
  (when sound (shell-command
                (concat "mplayer -really-quiet " sound " 2> /dev/null")))
  (if (eq window-system 'x)
    (shell-command (concat "notify-send "

                     (if icon (concat "-i " icon) "")
                     " '" title "' '" msg "'"))
    ;; text only version

    (message (concat title ": " msg))))

A couple of notes:

• I'm using notify-send for sending notifications; this assumes you are using that system (it's part of the libnotify-bin package in Debian/Ubuntu). You can of course replace it with whatever is available on your system. Alternatives are zenity or kdialog or xmessage (for old-timers) and their equivalents (?) on Windows, MacOS.
• I'm now using mplayer for playing sounds. This is a bit heavy, but at least plays all kinds of audio files. If you only care about .wav-files, you could replace it with e.g. aplay;
• as always, please ignore my ego-centric function names :-)

Now, we can use this function by evaluation e.g.

(djcb-popup "Warning" "The end is near"
   "/usr/share/icons/test.png" "/usr/share/sounds/beep.ogg")

showing pop-ups from org-mode appointments

The above popup function is most useful when it's does its work based on some event. To be notified of appointments and the like, there is the emacs appt facility. Here, we set up this appt, and then hook it up with org-mode, so appt can warn us when there's something happening soon…

;; the appointment notification facility
(setq
  appt-message-warning-time 15 ;; warn 15 min in advance

  appt-display-mode-line t     ;; show in the modeline
  appt-display-format 'window) ;; use our func
(appt-activate 1)              ;; active appt (appointment notification)
(display-time)                 ;; time display is required for this...

 ;; update appt each time agenda opened

(add-hook 'org-finalize-agenda-hook 'org-agenda-to-appt)

;; our little façade-function for djcb-popup
 (defun djcb-appt-display (min-to-app new-time msg)
    (djcb-popup (format "Appointment in %s minute(s)" min-to-app) msg 
      "/usr/share/icons/gnome/32x32/status/appointment-soon.png"

      "/usr/share/sounds/ubuntu/stereo/phone-incoming-call.ogg"))
  (setq appt-disp-window-function (function djcb-appt-display))

Of course, you can freely choose a icon / sound to your liking.

showing pop-ups for new mail

Another event you might want to be warned about is new mail. There is something to be set for not letting yourself be disturbed for new mail, but if you sufficiently filter your mails before they enter your inbox, it can be a good way to periodically bring you back from your deep sl ^H^H thinking. For Wanderlust, I use something like this:

(add-hook 'wl-biff-notify-hook
    (lambda()
      (djcb-popup "Wanderlust" "You have new mail!"
        "/usr/share/icons/gnome/32x32/status/mail-unread.png"
        "/usr/share/sounds/ubuntu/stereo/phone-incoming-call.ogg")))

Exercise for the reader: adapt this for your chosen mail client.

wanderlust tips and tricks

Earlier, I spoke of the wonderful Wanderlust e-mail client. After years of using mutt, I am a quite happy Wanderlust-user now. Now, it's few months since my conversion, time to discuss some of the customizations I did. Not all the defaults are so well-chosen (in my opinion), but fortunately, the package is very configurable.

If you are interested in Wanderlust, this entry might save you some time in figuring out such customizations and some other tricks. If you haven't done so before, I'd recommend you to read the older entry first. Also, the entry about BBDB may be useful.

Before going into the customizations, let me first answer a question I got asked a couple of times: why I am using Wanderlust and not, say, VM, gnus, Mew or even mutt or some other client?

To start with the last part, an emacs-based client fits in very well with my workflow, which is (duh) revolves around emacs. Doing my email there as well makes a lot of sense - a little return-on-investment for the time spent taming emacs and its bag of tricks.

The reason I particularly like Wanderlust, is that it works very well with mail stored in maildirs - as you may know, maildir is a one-file-per-message way of storing your mail on disk. That's great for backing up things, and sync'ing different machines.

Unlike VM and gnus, Wanderlust keeps the mail in the maildir as-is, and does not use a separate spoolfile – thus, all changes are reflected in the maildir itself, making it possible to use different clients (ie., use mutt when needed). Even more important, the wonderful tool offlineimap does two-way synchronization with IMAP-servers, and downloads everything into a maildir. So, I can download all the mail on my laptop machine, go offline and work on the messages (delete, move, reply etc.) during a flight, and when I'm back online, I can synchronize things. All this 'cloud'-stuff is nice, but I like to have my mails on my side of the intertubes.

Ok, now let's take a look at some of the customizations and tricks. All of these are little snippets to add to your ~/.wl-file.

Forwarded mails should use 'Fwd:', not 'Forward:'

I wonder why this is not the default.

(setq
  wl-forward-subject-prefix "Fwd: " )    ;; use "Fwd: " not "Forward: "

Reply-to-all should not be the default

By default, Wanderlust uses Reply-to-All; that is usually not what we (well, I) want. The code below makes Reply-to-Sender the default, with Reply-to-All behind C-u; ie. A or a will reply to sender, C-u A and C-u a reply to all.

(Note, the uppercase A is for replying with quoting the original message, while the lowercase version starts the reply with an empty message)

;; from a WL-mailinglist post by David Bremner

;; Invert behaviour of with and without argument replies.
;; just the author
(setq wl-draft-reply-without-argument-list
  '(("Reply-To" ("Reply-To") nil nil)
     ("Mail-Reply-To" ("Mail-Reply-To") nil nil)
     ("From" ("From") nil nil)))


;; bombard the world
(setq wl-draft-reply-with-argument-list
  '(("Followup-To" nil nil ("Followup-To"))
     ("Mail-Followup-To" ("Mail-Followup-To") nil ("Newsgroups"))
     ("Reply-To" ("Reply-To") ("To" "Cc" "From") ("Newsgroups"))
     ("From" ("From") ("To" "Cc") ("Newsgroups"))))

Setting up spam-handling

If you're using spamassassin for spamfiltering, you can quite easily integrate it with Wanderlust:

(require 'wl-spam)
(wl-spam-setup)
(setq elmo-spam-scheme 'sa)   ;; sa for spamassassin, see the elmo-spam-scheme
                              ;; docs for alternatives
(setq wl-spam-folder ".spam") ;; maildir to store spam

After this, you quite easily handle spam in the 'Summary' with some keybindings:

• k C : check whether spamassassin considers this message 'spam'
• k m : mark message(s) as spam (move to spam folder)
• k n : learn this message is 'ham'
• k s : learn this message is 'spam'

Note, there are some hooks for other spamfiltering solutions as well.

How to easily refile messages

I receive all my messages in only two mailboxes: one for personal mail, and one for mailing lists. If, after reading, I want to keep the message, I'll refile it to some other folder (after all, it's good to empty your mailboxes quite often. Wanderlust makes this refiling quite easy; the first way is to do it semi-automatic, i.e., let Wanderlust 'guess' the folder for you, based on the contents of the message. Then, when pressing 'o' in the summary, it will suggest this folder, and you can refile (move) the message. You can set up this 'guessing' something like this:

(setq
  ;; refile rules determine the default where mails are put

  ;; when you mark them for refiling ('o'); cfg. save-hooks in mutt
   wl-refile-rule-alist  
  '(
     ("Subject" ;; put more specific rules before more general ones.
       ("emacs"   . ".emacs")   ;; emacs-related mail

       ("running" . ".running") ;; running-related mail
       )
     
     (("To" "Cc" "Delivered-To") 
       ("myself@company.com"      . ".workmail") 
       ("myself@home.com"         . ".privatemail"))
     
     (("Precedence" "Priority")
       ("bulk\|1\|2\|list"       . ".bulkmail"))))

Explicit refiling

Semi-automatic refiling works fairly well, but you might also want to have some explicit shortcuts to move messages to specific folders. For example, to move message from your inbox to your Project X-folder, or your Project Y-folder.

(defun djcb-wl-summary-refile (&optional folder)
  "refile the current message to FOLDER; if FOLDER is nil, use the default"
  (interactive)
  (wl-summary-refile (wl-summary-message-number) folder)
  (wl-summary-next)
  (message (concat "refiled to " folder)))

(define-key wl-summary-mode-map (kbd "b x") ;; => Project X

  '(lambda()(interactive)(djcb-wl-summary-refile ".project-x"))) 
(define-key wl-summary-mode-map (kbd "b y") ;; => Project Y
  '(lambda()(interactive)(djcb-wl-summary-refile ".project-y")))

Assuming you have (Maildir) folders project-x and project-y.

Check outgoing mail

It's not uncommon to forget to add a subject or an attachment when you send a mail (or at least, when I send a mail…). However, using wl-mail-send-pre-hook we can let Wanderlust warn us when something like that happens.

;; suggested by Masaru Nomiya on the WL mailing list

(defun djcb-wl-draft-subject-check ()
  "check whether the message has a subject before sending"
  (if (and (< (length (std11-field-body "Subject")) 1)
        (null (y-or-n-p "No subject! Send current draft?")))
      (error "Abort.")))


;; note, this check could cause some false positives; anyway, better
;; safe than sorry...
(defun djcb-wl-draft-attachment-check ()
  "if attachment is mention but none included, warn the the user"
  (save-excursion
    (goto-char 0)
    (unless ;; don't we have an attachment?

      (re-search-forward "^Content-Disposition: attachment" nil t) 
     (when ;; no attachment; did we mention an attachment?
        (re-search-forward "attach" nil t)
        (unless (y-or-n-p "Possibly missing an attachment. Send current draft?")
          (error "Abort."))))))

(add-hook 'wl-mail-send-pre-hook 'djcb-wl-draft-subject-check)
(add-hook 'wl-mail-send-pre-hook 'djcb-wl-draft-attachment-check)

Ok, that's all for now… I'll get back to Wanderlust in the future; of course, feel free to add your own tricks in the comments-section.

e-mail with wanderlust

e-mail with wanderlust

(This is part I of entry on the wanderlust e-mail client; part II will appear soon).

Earlier, I have written about how I am using mutt as my e-mailclient. I discussed running mutt inside emacs as well. Of course, mutt is an external program, which puts some limits on its integration with emacs. I did try various emacs-based clients, such as VM (ViewMail) and GNUS, but they always left me a bit unsatisfied.

To start with, it was rather hard to set these programs up – and I am an Emacs-user, I like tweaking things… Still, it was hard to get even simple things working. Maybe I have uncommon wishes, but my desired setup already sent me to the edges of the googleable universe.

But now I have found an emacs-based client that seems to work really well for me. It's called Wanderlust, and it's a fine piece of Japanese engineering. It can be a little bit intimidating to set up at first, even though I found it still much easier than the other emacs-based clients I tried. Anyway, I am sharing a very basic setup here, enough to get you going.

What about my setup? Well: I use maildirs – that is, I download my mail into a local ~/Maildir directory-tree, either with e.g. fetchmail or with offlineimap. It's a particulary nice setup for offline-usage: whenever there's a network connection, I suck up all the mails and have them available offline. I work like this since the days when there was only expensive dial-up access to the net, and later I found it very convenient when I was traveling with a laptop and had only occasional net-access.

So, Maildir access is pretty important for me, and I'll describe my setup for using Wanderlust with it here. If you're using IMAP instead of Maildirs, you might be interested in the Emacs with Wanderlust and GMail-article.

Getting started

So, how to get started with Wanderlust?

In short: install Wanderlust and put some stuff in two files (~/.wl and ~/.folders).

A little bit longer:

• get wanderlist - I am using the wl-beta packages from Ubuntu/Debian, which makes this a painless process, but you can also use source packages;
• put your Wanderlust-configuration in a special file: ~/.wl;
• put a list of the mail folders you're interested in, in a file called

~/.folders. (yes, you can customize all this)

For the rest of the discussion, let's assume we have a Maildir which contains some folders:

• inbox for incoming e-mail
• bulk for incoming Mailing List mail
• drafts for drafts
• sent for sent e-mail
• trash for junk email

All incoming mail is going to either inbox or bulk. I'm not going to discuss how to get the mails there – I assume you're already have these thing set up; otherwise, you can take a look at tools like fetchmail, retchmail, procmail and friends. Note that much of the discussion here applies as well if you're using Wanderlust with POP or IMAP.

What to put in ~/.wl?

So, how to setup Wanderlust to use this? Well, our configuration goes into a file called ~/.wl. There's a million more things you can set up here , but let's stick to the basics here. I'll discuss more tricks and extensions later.

;; mode:-*-emacs-lisp-*-
;; wanderlust 
(setq 
  elmo-maildir-folder-path "~/Maildir"          ;; where i store my mail

  wl-stay-folder-window t                       ;; show the folder pane (left)
  wl-folder-window-width 25                     ;; toggle on/off with 'i'
  
  wl-smtp-posting-server "localhost"            ;; put the smtp server here
  wl-local-domain "myhost.example.com"          ;; put something here...
  wl-message-id-domain "myhost.example.com"     ;; ...

  wl-from "Me <me@example.com>"                  ;; my From:

  ;; note: all below are dirs (Maildirs) under elmo-maildir-folder-path 
  ;; the '.'-prefix is for marking them as maildirs
  wl-fcc ".sent"                       ;; sent msgs go to the "sent"-folder
  wl-fcc-force-as-read t               ;; mark sent messages as read 
  wl-default-folder ".inbox"           ;; my main inbox 
  wl-draft-folder ".drafts"            ;; store drafts in 'postponed'
  wl-trash-folder ".trash"             ;; put trash in 'trash'
  wl-spam-folder ".trash"              ;; ...spam as well
  wl-queue-folder ".queue"             ;; we don't use this

  ;; check this folder periodically, and update modeline
  wl-biff-check-folder-list '(".todo") ;; check every 180 seconds
                                       ;; (default: wl-biff-check-interval)

  ;; hide many fields from message buffers
  wl-message-ignored-field-list '("^.*:")
  wl-message-visible-field-list
  '("^\\(To\\|Cc\\):"
    "^Subject:"
    "^\\(From\\|Reply-To\\):"
    "^Organization:"
    "^Message-Id:"
    "^\\(Posted\\|Date\\):"
    )
  wl-message-sort-field-list
  '("^From"
    "^Organization:"
    "^X-Attribution:"
     "^Subject"
     "^Date"
     "^To"
     "^Cc"))

What to put in ~/.folders?

So, that was the basic setup. Now we need to tell wanderlust about the folders we'd like to see in the user-interface: ~/.folders. Wanderlust does not automatically use all the folder in your ~/Maildir. The folder names in ~/.folders can refer to maildirs, newsgroups, POP-account, IMAP-folders and a couple of other ones. In particular, you can combine folders to show as aggregate folders in Wanderlust.

Wanderlust distinguishes the kind of folder something is by looking at a special prefix character. For maildirs, this is a dot ('.'), IMAP-folders get a '+', newsgroups get a '-' and so on. After the mailbox name, you can have a 'friendly name' in quotes, which is what will show up in the user-interface.

Something like this:

# WL folders (put the '#' always at the beginning of the line)
.inbox     "Inbox"
.bulk      "Mailinglists"
.drafts    "Drafts"
.sent      "Sent Mail"
.trash     "Junk"

Note, the little format has some more tricks; e.g., if you had some folders, you cold have something like:

# WL folders (put the '#' always at the beginning of the line)

# 'Lists' will be a tree node with three elements
Lists {
        .mailinglist1 "A mailinglist"
        .mailinglist2 "Another Mailinglist" 
        .mailinglist3 "Yet one more mailinglist"
}

# you make virtual folders from combined lists with the '*'
AllMyMailingLists  *.mailinglist1,.mailinglist2,.mailinglist3

Anyway, there's plenty of room for experimentation here…

Now, we're ready! Start Wanderlust with M-x wl. You will be greeted, and things are relatively easy from there on. You might want to use the menu at first at least, as the default keybindings are somewhat surprising (like a for reply, and w to compose a new message).

What's next?

As I said, there are 10E6 things to configure and to customize, but this should give you a reasonable setup to start with; you might want to change some details (like the folder names) to match your situation. Good luck!

As a true emacs-mailer, Wanderlust nicely integrates with e.g. BBDB (the addressbook) and org-mode. You can also use GPG (through mailcrypt), spamfiltering, and so on. I will discuss some of those things in the second part of the discussion of Wanderlust.