2009-06-08

e-mail with wanderlust

e-mail with wanderlust

(This is part I of entry on the wanderlust e-mail client; part II will appear soon).

Earlier, I have written about how I am using mutt as my e-mailclient. I discussed running mutt inside emacs as well. Of course, mutt is an external program, which puts some limits on its integration with emacs. I did try various emacs-based clients, such as VM (ViewMail) and GNUS, but they always left me a bit unsatisfied.

To start with, it was rather hard to set these programs up – and I am an Emacs-user, I like tweaking things… Still, it was hard to get even simple things working. Maybe I have uncommon wishes, but my desired setup already sent me to the edges of the googleable universe.

But now I have found an emacs-based client that seems to work really well for me. It's called Wanderlust, and it's a fine piece of Japanese engineering. It can be a little bit intimidating to set up at first, even though I found it still much easier than the other emacs-based clients I tried. Anyway, I am sharing a very basic setup here, enough to get you going.

What about my setup? Well: I use maildirs – that is, I download my mail into a local ~/Maildir directory-tree, either with e.g. fetchmail or with offlineimap. It's a particulary nice setup for offline-usage: whenever there's a network connection, I suck up all the mails and have them available offline. I work like this since the days when there was only expensive dial-up access to the net, and later I found it very convenient when I was traveling with a laptop and had only occasional net-access.

So, Maildir access is pretty important for me, and I'll describe my setup for using Wanderlust with it here. If you're using IMAP instead of Maildirs, you might be interested in the Emacs with Wanderlust and GMail-article.

Getting started

So, how to get started with Wanderlust?

In short: install Wanderlust and put some stuff in two files (~/.wl and ~/.folders).

A little bit longer:

  • get wanderlist - I am using the wl-beta packages from Ubuntu/Debian, which makes this a painless process, but you can also use source packages;
  • put your Wanderlust-configuration in a special file: ~/.wl;
  • put a list of the mail folders you're interested in, in a file called

~/.folders. (yes, you can customize all this)

For the rest of the discussion, let's assume we have a Maildir which contains some folders:

  • inbox for incoming e-mail
  • bulk for incoming Mailing List mail
  • drafts for drafts
  • sent for sent e-mail
  • trash for junk email

All incoming mail is going to either inbox or bulk. I'm not going to discuss how to get the mails there – I assume you're already have these thing set up; otherwise, you can take a look at tools like fetchmail, retchmail, procmail and friends. Note that much of the discussion here applies as well if you're using Wanderlust with POP or IMAP.

What to put in ~/.wl?

So, how to setup Wanderlust to use this? Well, our configuration goes into a file called ~/.wl. There's a million more things you can set up here , but let's stick to the basics here. I'll discuss more tricks and extensions later.

;; mode:-*-emacs-lisp-*-
;; wanderlust 
(setq 
  elmo-maildir-folder-path "~/Maildir"          ;; where i store my mail

  wl-stay-folder-window t                       ;; show the folder pane (left)
  wl-folder-window-width 25                     ;; toggle on/off with 'i'
  
  wl-smtp-posting-server "localhost"            ;; put the smtp server here
  wl-local-domain "myhost.example.com"          ;; put something here...
  wl-message-id-domain "myhost.example.com"     ;; ...

  wl-from "Me <me@example.com>"                  ;; my From:

  ;; note: all below are dirs (Maildirs) under elmo-maildir-folder-path 
  ;; the '.'-prefix is for marking them as maildirs
  wl-fcc ".sent"                       ;; sent msgs go to the "sent"-folder
  wl-fcc-force-as-read t               ;; mark sent messages as read 
  wl-default-folder ".inbox"           ;; my main inbox 
  wl-draft-folder ".drafts"            ;; store drafts in 'postponed'
  wl-trash-folder ".trash"             ;; put trash in 'trash'
  wl-spam-folder ".trash"              ;; ...spam as well
  wl-queue-folder ".queue"             ;; we don't use this

  ;; check this folder periodically, and update modeline
  wl-biff-check-folder-list '(".todo") ;; check every 180 seconds
                                       ;; (default: wl-biff-check-interval)

  ;; hide many fields from message buffers
  wl-message-ignored-field-list '("^.*:")
  wl-message-visible-field-list
  '("^\\(To\\|Cc\\):"
    "^Subject:"
    "^\\(From\\|Reply-To\\):"
    "^Organization:"
    "^Message-Id:"
    "^\\(Posted\\|Date\\):"
    )
  wl-message-sort-field-list
  '("^From"
    "^Organization:"
    "^X-Attribution:"
     "^Subject"
     "^Date"
     "^To"
     "^Cc"))

What to put in ~/.folders?

So, that was the basic setup. Now we need to tell wanderlust about the folders we'd like to see in the user-interface: ~/.folders. Wanderlust does not automatically use all the folder in your ~/Maildir. The folder names in ~/.folders can refer to maildirs, newsgroups, POP-account, IMAP-folders and a couple of other ones. In particular, you can combine folders to show as aggregate folders in Wanderlust.

Wanderlust distinguishes the kind of folder something is by looking at a special prefix character. For maildirs, this is a dot ('.'), IMAP-folders get a '+', newsgroups get a '-' and so on. After the mailbox name, you can have a 'friendly name' in quotes, which is what will show up in the user-interface.

Something like this:

# WL folders (put the '#' always at the beginning of the line)
.inbox     "Inbox"
.bulk      "Mailinglists"
.drafts    "Drafts"
.sent      "Sent Mail"
.trash     "Junk"

Note, the little format has some more tricks; e.g., if you had some folders, you cold have something like:

# WL folders (put the '#' always at the beginning of the line)

# 'Lists' will be a tree node with three elements
Lists {
        .mailinglist1 "A mailinglist"
        .mailinglist2 "Another Mailinglist" 
        .mailinglist3 "Yet one more mailinglist"
}

# you make virtual folders from combined lists with the '*'
AllMyMailingLists  *.mailinglist1,.mailinglist2,.mailinglist3

Anyway, there's plenty of room for experimentation here…

Now, we're ready! Start Wanderlust with M-x wl. You will be greeted, and things are relatively easy from there on. You might want to use the menu at first at least, as the default keybindings are somewhat surprising (like a for reply, and w to compose a new message).

What's next?

As I said, there are 10E6 things to configure and to customize, but this should give you a reasonable setup to start with; you might want to change some details (like the folder names) to match your situation. Good luck!

As a true emacs-mailer, Wanderlust nicely integrates with e.g. BBDB (the addressbook) and org-mode. You can also use GPG (through mailcrypt), spamfiltering, and so on. I will discuss some of those things in the second part of the discussion of Wanderlust.

36 comments:

Petteri said...

I have also tried wanderlust, but the big problem with it was that when I switch back and fort between windows (emacs windows), the layout gets messed up. Maybe with somekind of hook that would put the email, folder and index view back to their places when switching to wanderlust could help, but at the time that seemed just too much work.

djcb said...

@Petteri: there are various ways to save your setup and restore when you go back; see e.g. http://www.emacswiki.org/emacs/CategoryWindows

The easiest way might be just to have a separate frame for email...

okflo said...

Did you ever try mew?

http://www.mew.org/ my current favorite mail-client for emacs. Exspecially as it works quite fine with imap.

djcb said...

@okflo: I have looked at it, but it does not seem to support Maildirs...

Rasmus Pank Roulund said...

Hello,
I am getting an error when trying to run wl. "Cannot open load file ssl". I have installed a snapshot version from may 2009. Of course, I have installed all depencies.

(I have also received the error "byte-code: Opening output file: no such file or directory, /home/rasmus/.emacs.d/url/exec_ssl_quietly")

It looks like a really nice mali-mode btw.

Does anybody know what I migth be missing?

I am running Emacs CVS (23.0.93.1) on Arch.

Anonymous said...

What are the reason(s) you are not satisified with VM? no left sidebar?

djcb said...

@Anonymous: about VM: I used VM for a while years ago, and it's a fine program.

But for me the lack of Maildir-support was a showstopper.

I don't care about the folder-list left sidebar, it's in the screenshot but I have it turned off.

Anonymous said...

djcb: thanks for the answer!

Fair enough.

maildir support is in the wishlist:
https://bugs.launchpad.net/viewmail/+bug/107092

but this does not seem too high in the roadmap:
http://bazaar.launchpad.net/~hack-robf/viewmail/trunk/annotate/head:/TODO

Granted viewmail is not really fast on big mboxes (compared to e.g. mutt).

djcb said...

@Anonymous: one nice thing about wanderlust is that it uses the Maildirs as-is, ie., the mailer doesn't use it's own parallel system (exc. caching). So, I seamlessly switch between mutt, wl and others. instead, e.g. gnus seems to insist on its own system instead, even when using maildirs.

Anonymous said...

@Rasmus Pank Roulund

Either copy utils/ssl.el to the site-lisp/wl directory or uncomment the line ;(setq wl-install-utils t) in WL-CFG and rebuild the wanderlust snapshot.

Anonymous said...

Thanks for the post about wonderlust, I especially appreciate the configurations you provided to get started! I didn't know about it and am giving it a try. It seems pretty nice so far.

I'm having trouble getting it to check for new emails, even with middle clicking on the envelope icon, it doesn't seem to work. It does say, "Checking new emails..." but then doesn't seem to show anything. If I go over to my INBOX folder and open it again, they show up...

In any case, I'm looking forwards to your next installment of the wanderlust post!

Anonymous said...

Hi,

I am using wanderlust since around 5 years, switched from mew when that did not have decent imap support yet. Just lately discovered the maildir stuff in it - perfect for accessing imap backups created with offlineimap.

Thanks

Juergen

til said...

Great post, can't wait for the sequel! I'm checking out wanderlust right now as well, after many years of using mutt, mainly because I want an email client that can integrate with org-mode, so that I can quickly link to emails and mark them as TODO, WAITING, tag them etc.

A minor typo: to specify an IMAP folder, wl wants the prefix '%', not '+'

13-49 said...

Wanderlust rocks and is definitely the best IMAP client ever existed. I did some wl hacking, especially for more superior IMAP support, you can find my patches here: http://repo.or.cz/w/more-wl.git

djcb said...

@13-49: great work! I hope WL development will pick up a bit, and a new release with your patches...

Robert said...

Re VM, I gave that up because it didn't have real IMAP support --- VM really needed you to have all your email on the mail client machine.

If you want to use multiple machines and keep your email on an IMAP server, then VM is not suitable (or at least wasn't, when I gave up on it a couple of years ago). VM was effectively treating IMAP as if it was a dumb server.

Dan said...

I looked at the home page but am concerned that it doesn't appear to be maintained. Half the links are broken. The download link requires a userid/password that I don't know. The online manual hasn't been updated since 2005.

Is this software actively maintained? It looks like abandonware.

djcb said...

@Dan: good question.

All I can say is that it's still being supported (see the mailing list); also, there are some efforts for updating wanderlust; see http://repo.or.cz/w/more-wl.git for the work that Vitaly Mayatskikh is doing on that.

The community could be a bit more vibrant, but it's definitely not 'abandonware'.

Dominic said...

Thanks for the two really helpful posts about wanderlust. I have now got getmail + maildirs set up on my machine fine and wanderlust sees them fine, interacts with them and so on.

BUT - I have a big problem. I would like to manage multiple accounts from within wl. Is this possible? Only a single customisable variable for smtp servers seems to imply not :(

I'm looking for something similar to Thunderbird's multiple accounts.

djcb said...

@Dominic: well, this is emacs so everything is possible with a bit of elisp :)

you can have multiple accounts for incoming mail (i have), and simply deliver those to different mailboxes (for example using procmail or something like that)

for outgoing mail, you can use something like the wl-mail-send-pre-hook
mentioned in http://emacs-fu.blogspot.com/2009/09/wanderlust-tips-and-tricks.html, and add a function that asks you which smtp-server you want to use.

Anonymous said...

I have made progress and thanks for your suggestions - now understand draft-config-alist and where it needs to be set. Sending from multiple accounts is working fine - just need to figure out how to get the correct From: field into the draft buffer. Otherwise I'm almost ready to switch across to wl entirely!

djcb said...

@Anonymous: you could use wl-template-alist:
===================
(setq wl-template-alist
'(("default"
("From" . wl-from)
(body . "\nBest wishes,\nMickey Mouse.\n"))
("mailinglist"
("From" . "anotheraddress@example.com"))))
===================

and then maybe

============================
;; template mode
;; go through templates with arrow keys
(define-key wl-template-mode-map (kbd "<right>") 'wl-template-next)
(define-key wl-template-mode-map (kbd "<left>") 'wl-template-prev)
===============================

to select templates for your different account, and then set the smtp server based on the From:. Good luck!

ramestica said...

Hi,

sorry if this comment comes out of topic. I giving wanderlust a try and what I seem not able to 'fix' is the structure of my imap folders. For year I have worked in a server that does not allow to have messages in a folder that contains more folders, which has been no real limitation for the way I like to surf my emails. Only the leaves contain messages. At the same time, you are probably guessing that I have a lot of folders within other folders and so on.

Is there a was in wanderlust to properly match my current imap folder structure with something that will look similar to the graphical representation I get in thunderbird, for example. Access groups and petnames I have tried, but I do not get to good results.

djcb said...

@ramestica:

well, you need to put the folder you'd like to see in your ~/.folders; if you want to see the hierarchy, you can put
'%/' (it sorts them alphabetically...)

now, nested folders are represented by dots, so INBOX/foo/bar would be put as '%INBOX.foo.bar'

good luck!

Anonymous said...

wanderlust depends on some strange mime packages. you can find them here: http://www.kanji.zinbun.kyoto-u.ac.jp/~tomo/lemi/dist/

derele said...

Hi,

I love this blog!
I am currently choosing smtp servers based on my ip-address, the reason for this is, that the university I am working at does not allow external smtp-servers to be used:
I am getting the ip based on this post

I have th following in my .emacs
;;;;;;;;;;;;;;;;;;;;;;;;;;,,,;; get the actual ip;;;;;;;;;;;;;;;;;;;;;;
(defun get-ip-address (&optional dev)
"get the IP-address for device DEV (default: eth0)"
(let ((dev (if dev dev "eth0")))
(format-network-address (car (network-interface-info dev)) t)))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; mail-setup: wanderlust ;;;;;;;;
(autoload 'wl "wl" "Wanderlust" t)
(autoload 'wl-other-frame "wl" "Wanderlust on new frame." t)
(autoload 'wl-draft "wl-draft" "Write draft with Wanderlust." t)

;; SMTP
(let
((my-ip (if (get-ip-address)(get-ip-address)(get-ip-address "wlan0")))
) ;; if we find wired take it otherwise go for wifi
(if my-ip ;; there is a network
(if (string-match "129.13.174." my-ip) ;;inside uni
(setq
wl-smtp-connection-type 'starttls
wl-smtp-posting-port 587
wl-smtp-posting-user "my_uni_name"
wl-smtp-posting-server "smtp.stud.uni-karlsruhe.de"
wl-local-domain "uni-karlsruhe.de")
(setq ; else outside use gmail
wl-smtp-connection-type 'starttls
wl-smtp-posting-port 587
wl-smtp-posting-user "my_name@gmail.com"
wl-smtp-posting-server "smtp.gmail.com"
wl-local-domain "gmail.com")
)
;; else we are offline
(print "No Network connections found on eth0 and wlan0")
))

It works fine as long as the network connection is established before my emacs-daemon is stared on startup...

djcb said...

@derele: you could probably update the smtp-settings in wl-draft-send-hook; you could probably even set it WL's status to 'unplugged' if you're not connected, so mail will be queued instead.

derele said...

Cool! Thanks! This is the most advanced thing I've done in elisp so far...
After your advice I have the following in my .wl:

(add-hook 'wl-draft-send-hook
;; SMTP
'(lambda()
(let
((my-ip (if (get-ip-address)(get-ip-address)(get-ip-address "wlan0")))
) ;; if we find wired take it otherwise go for wifi
(if my-ip ;; there is a network
(if (string-match "129.13.174." my-ip) ;;inside uni
(setq
wl-smtp-connection-type 'starttls
wl-smtp-posting-port 587
wl-smtp-posting-user "dc134"
wl-smtp-posting-server "smtp.stud.uni-karlsruhe.de"
wl-local-domain "uni-karlsruhe.de")
(setq ; else outside use gmail
wl-smtp-connection-type 'starttls
wl-smtp-posting-port 587
wl-smtp-posting-user "emanuelheitlinger@gmail.com"
wl-smtp-posting-server "smtp.gmail.com"
wl-local-domain "gmail.com")
)
;; else we are offline
(print "No Network connections found on eth0 and wlan0")
))
)
)

It is great because it keeps my .emacs tidy and it is loaded at the time I need it.
I could not find the right wl-status variable (there are twelve variables involving "wl" and "status") to set at the point of the print statement.

derele said...

Missed a line needed for gmail...

(add-hook 'wl-draft-send-hook
;; SMTP
'(lambda()
(let
((my-ip (if (get-ip-address)(get-ip-address)(get-ip-address "wlan0")))
) ;; if we find wired take it otherwise go for wifi
(if my-ip ;; there is a network
(if (string-match "129.13.174." my-ip) ;;inside uni
(setq
wl-smtp-connection-type 'starttls
wl-smtp-posting-port 587
wl-smtp-posting-user "dc134"
wl-smtp-posting-server "smtp.stud.uni-karlsruhe.de"
wl-local-domain "uni-karlsruhe.de")
(setq ; else outside use gmail
wl-smtp-connection-type 'starttls
wl-smtp-posting-port 587
wl-smtp-authenticate-type "plain" ; <- gmail needs this
wl-smtp-posting-user "emanuelheitlinger@gmail.com"
wl-smtp-posting-server "smtp.gmail.com"
wl-local-domain "gmail.com")
)
;; else we are offline
(print "No Network connections found on eth0 and wlan0")
))
)
)

41 said...

I just tried wanderlust and it's very promising.
But I can't get my smtp server to work.
Is it possible to use external smtp clients like msmtp with wanderlust? I could not figure out how.

And I'd like to see my Inbox folder by default when I start wl, is this possible as well?

Luiz√£o said...

Could you also share the fetchmail/Maildir configuration you use? I'd like to try working that way but I don't know how to set that up.

I've already searched the web and everything, maybe I'm missing something or not understanding well the manual. Thanks in advance

The SA Survivalist said...

Hey dj - thanks for all the tips. Wl is now working very well for me. But what is that color theme you have there?

djcb said...

@SA Survivalist: I think it's a version of the theme shown in http://emacs-fu.blogspot.com/2009/03/color-theming.html

J said...

Rasmus Pank Roulund:I am getting an error when trying to run wl. "Cannot open load file ssl". I have installed a snapshot version from may 2009. Of course, I have installed all depencies.


You need to uncomment the line

(setq wl-install-utils t)

in the file WL-CFG in the wanderlust project directory. Then utils will be built when you run make, which will include ssl support.

Anonymous said...

In summary view, how can you sort threads by youngest message in each thread?

In mutt speak:

set sort=threads

set sort_aux=last-date-received

Anonymous said...

Yes, I also would like to know that. How can you sort in summary the threads by the youngest message in them.

Sort of like in gmail.