searching e-mails with wanderlust and mu

I have discussed the Wanderlust e-mail client a couple of times already. I'm still using it, so I keep on learning new tricks. Even though there has been quite a bit of action in the competing gnus e-mail client, for my particular use-case, Wanderlust is still the best option.

'My particular use-case' consists of storing my mail in Maildirs, which I fill with either offlineimap (which has fortunately found a new maintainer) or fetchmail.

mu

When dealing with e-mail, one particularly important feature for me is the ability to search my messages. In fact, it's so important for me that I wrote some software to do this for me; the software is called mu; it indexes the messages in my Maildirs, and then allows for searching them using queries, based on message contents, headers, or other message properties.

mu works through a command-line interface, although there is an experimental GUI available as well. The command-line interface makes it possible to hook mu up with various mail-clients, such as mutt, or Wanderlust. Some Linux distributions ship mu, but since the versions they ship are often a bit outdated, I recommend building it yourself from the sources linked on the mu website. The process is fairly straightforward; and there is plenty of documentation in the form of man pages.

mu and wanderlust

I've been combining mu and wanderlust for a while (see mu and wanderlust - the old way, below), but this week Sam B. on the mu mailing list showed a way to do so in a much more elegant way - using virtual or query folders.

How does this work? Well, after installing mu, add the following to your Wanderlust setup file (~/.wl or it's moral equivalent – see the older Wanderlust posts for the details):

(require 'elmo-search)
(elmo-search-register-engine
    'mu 'local-file
    :prog "/usr/local/bin/mu" ;; or wherever you've installed it
    :args '("find" pattern "--fields" "l") :charset 'utf-8)

(setq elmo-search-default-engine 'mu)
;; for when you type "g" in folder or summary.
(setq wl-default-spec "[")

So, to start with the last part, whenever you type g in folder or summary, in the mode-line you will get something like Folder name (.inbox): [. Now simply type your mu search expression and press Enter, and wanderlust opens a (temporary) folder with the search results. Brilliant!

Next, to add virtual folders for searches you do often, simply add some folder specifications like the following to your .folders file (again, check the older Wanderlust posts if you're not familiar with folders-file):

VFolders {
# message I received today
  [date:today..now]!mu  "Today"

# messages bigger than 1Mb  
  [size:1m..100m]!mu    "Big"

# signed messages i got in 2010 related to emacs
  [date:2010..2011 flag:signed emacs]!mu "Signed-Emacs2010"

# unread messages
  [not flag:seen]!mu    "Unread"
# or (for mu  >= 0.9.4):
# [flag:unread]! mu      "Unread"
}

After this, restart Wanderlust, and there you go! Wanderlust will display your brand new virtual folders with an icon that looks like a little whale.

You can put arbitrary mu search expressions between the [], matching whatever is useful in a certain case. Check the mu documentation to see how to do this.

Note, the messages you get in these virtual folders are links to the original messages. In practice, this means that changes you make to the links do no affect the originals – if you delete a link you're not deleting the message.

mu and wanderlust - the old way

This discussion would not complete without a description of the old way I used search. This method may still be useful for integrating mu with other clients such as mutt.

What I've been using for a while is a (in retrospect) rather clumsy way to integrate message searches with Wanderlust: based on the results of a query, I would create some special Maildir and fill it with symbolic links to the matched messages, and the visit this special Maildir with Wanderlust. I'll include the code here to contrast it with the more elegant solution that we saw before, but also because the approach taken might be easily adapted for other mail-clients.

;; search using mutt
(defvar mu-wl-mu-program     "/usr/local/bin/mu")
(defvar mu-wl-search-folder  "search")

(defun mu-wl-search ()
  "search for messages with `mu', and jump to the results"
   (let* ((muexpr (read-string "Find messages matching: "))
          (sfldr  (concat elmo-maildir-folder-path "/"
                    mu-wl-search-folder))
          (cmdline (concat mu-wl-mu-program " find "
                      "--clearlinks --format=links --linksdir='" sfldr "' "
                     muexpr))    
          (rv (shell-command cmdline)))
    (cond
      ((= rv 0)  (message "Query succeeded"))
      ((= rv 2)  (message "No matches found"))
      (t (message "Error running query")))
  (= rv 0)))

(defun mu-wl-search-and-goto ()
  "search and jump to the folder with the results"
  (interactive)
  (when (mu-wl-search)
    (wl-summary-goto-folder-subr
      (concat "." mu-wl-search-folder)
      'force-update nil nil t)
    (wl-summary-sort-by-date)))

;; search by pressing 'Q'
(define-key wl-summary-mode-map (kbd "Q") ;; => query
  '(lambda()(interactive)(mu-wl-search-and-goto))) 
(define-key wl-folder-mode-map (kbd "Q") ;; => query
  '(lambda()(interactive)(mu-wl-search-and-goto))) 

After installing mu and putting the above in your wanderlust startup file, you should be able to search by pressing Q. The mu documentation has an example for mutt as well.

conclusion

It's straightforward to integrate advanced searching capabilities to Wanderlust using mu, and thanks to Sam B., it's gotten a lot easier! The second (old) approach may be useful as 'inspiration' for use in other e-mail clients as well, if they do not provide the kind of hooks that the first solution needs.

IELM: a REPL for emacs

Emacs-lisp (elisp) is a nice language to play around with code and try things as you develop them – explorative programming. I often use the *scratch* buffer for that, but sometimes it's nice to use a so-called 'REPL' ( Read-Eval-Print-Loop) instead. A REPL is a sort-of command-line interface where your expressions are evaluated as soon as they are considered 'complete' and you press Enter.

So, enter Emacs's built-in repl: IELM. You can activate it with M-x ielm, and the interaction looks something like the following:

*** Welcome to IELM ***  Type (describe-mode) for help.
ELISP> 123 
123
ELISP> (+ 1 2)
3
ELISP> ;; comment
ELISP> (defun fac (n)
         (if (= 0 n)
           1
           (* n (fac (- n 1)))))
fac
ELISP> (fac 5)
120
ELISP> 

By default, IELM evaluates complete expressions automatically as soon you as you press Enter. So one thing to remember is that if you want to have multi-line expression (like above), you must make sure that after each line the expression is not complete (i.e., the brackets are not balanced) -- otherwise the expression will be evaluated too early. That makes modes like autopair or paredit a bit inconvenient for this.

If you don't like that behavior, you can do:

(setq ielm-dynamic-return nil)

which will allow you to Enter as much as you want and only evaluate things when you press C-j. But then you might as well use *scratch* I suppose. Personally, I use IELM mostly as a calculator.