emacs-fu

editing with root-privileges -- once more

2013-03-17T20:13:00.002+02:00

We've already discussed editing root-owned files here a few times; it's one of those tasks where in a reflex I still open a terminal and use vi to do the job… the only way to overcome that seems to be to make it really easy to do the same from within my running emacs:

  (defun djcb-find-file-as-root ()
  "Like `ido-find-file, but automatically edit the file with
root-privileges (using tramp/sudo), if the file is not writable by
user."
  (interactive)
  (let ((file (ido-read-file-name "Edit as root: ")))
    (unless (file-writable-p file)
      (setq file (concat "/sudo:root@localhost:" file)))
    (find-file file)))
;; or some other keybinding...
(global-set-key (kbd "C-x F") 'djcb-find-file-as-root)

We could take it one step further still – overload the normal (ido-)find-file with a version that checks the permissions first, and if needed, use the above function to open it. But maybe that is too easy; we should be careful with root-owned files after all.

mu4e v0.9.9 is out!

2012-10-15T11:46:00.000+03:00

I few months ago, I introduced mu4e, a modest little emacs mail client that I wrote. It seems many people picked it up, great!

what's new?

I have just released mu/mu4e version 0.9.9. There are quite a few changes, both user-visible and 'under-the-hood'. I've also spent some time on improving the manual (pdf), and I'm quite happy about it.

Some of the more visible new things in mu4e are:

Support for crypto (decrypting messages, signing them)
Support for refiling (like mutt, Wanderlust)
Dynamic folders (instead of hard-coding the sent/draft/trash/refile folder, they can be functions that return different folders based on the message we're dealing with, see the example below)
Same for the folder to save attachments
A lot of smaller and bigger UI improvements

Also, the core mu program has seen a lot of improvements (many of which directly improve mu4e as well)

Better support for non-ascii locales / character sets, such as ISO-2022-JP
Improved on-line help ('mu help ...')
Performance improvements for threaded display (~ 25% for 23K messages)

For a more complete list, see NEWS.

dynamic folders

As mentioned, mu4e now supports dynamic folders. Before, you'd set your trash folder to some static string:

(setq mu4e-trash-folder "/trash")

In some cases, you may want to have a bit more flexibility – for example, have a separate trash-folder (or sent-folder, drafts-folder, refile-folder) for private mail and work mail. You can now do something like:

(setq mu4e-trash-folder
  (lambda (msg)
    (if (and msg ;; msg may be nil
          (mu4e-message-contact-field-matches msg :to "me@work.com"))
      "/trash-work"
      "/trash")))

refiling

After I have dealt with some e-mail, I either delete it or move it to some archive folder – refiling. For this, there is now the r keybinding, and mu4e-refile-folder; and a place where dynamic folders really shine:

(setq mu4e-refile-folder
  (lambda (msg)
    (cond
      ;; messages to the mu mailing list go to the /mu folder
      ((mu4e-message-contact-field-matches msg :to
         "mu-discuss@googlegroups.com")
        "/mu")
      ;; messages sent directly to me go to /archive
      ;; also `mu4e-user-mail-address-regexp' can be used
      ((mu4e-message-contact-field-matches msg :to "me@example.com")
        "/private")
      ;; messages with football or soccer in the subject go to /football
      ((string-match "football\\|soccer" (or (mu4e-message-field msg :subject) ""))
        "/football")
      ;; everything else goes to /archive
      ;; important to have a catch-all at the end!
      (t "/archive"))))

How cool is that? After reading my inbox folder, I select all messages (C-x h), press r, and they're all moved to the right refiling folder.

crypto support

mu4e already supported signing/encrypting messages, but now it supports decryption and verifying signatures as well. This was one of the most requested new features. I think it is still a bit rough, but it has been working very well for me.

so…

I think version 0.9.9 is a great new step for mu4e. It already goes far beyond I ever planned to do. I received a lot of suggestions for new features, which is great! I'm not planning to implement all of those, but I will try to make mu4e even more programmable – it should be easy to augment mu4e with your own little elisp-snippets – the Barbapapa principle of software, already so clearly present in emacs itself.

introducing mu4e, an e-mail client for emacs

2012-08-20T22:31:00.000+03:00

background

I've haven't written too much here lately. This is because I have spent most of my emacs hacking time on mu4e, an emacs e-mail client. It's only slightly over six months old since its first release, but it has progressed rather quickly! So, I think it is time to give it an introduction to a wider audience. In future posts, I'll go into more detail.

E-mail is a very important means of communication for me; and for purposes of integration, it prefer to do my e-mail with emacs. For a long time, I used mutt with emacs as the editor; a few years back I switched to Wanderlust, and wrote a few articles about it.

There's a lot to like about Wanderlust: it's very featureful, and allows for a lot of customization. Still, there were a few things I was not fully happy with; the most important one was the speed of some operations; another annoyance was the fact that I got rather frequent cache-corruptions when using newer emacs versions.

So, what was I to do? Well, a few years back, I had written an e-mail indexer/searcher, called mu. It takes the messages in a maildir-folder, and allows you to query them. I wondered if I could use mu as the backend for an emacs-based e-mail client. So, I fired up emacs, started writing some code and some later I had a first version of… mu4e.

how does it work?

mu4e is an emacs front-end for mu; when you start it, mu4e connects to a mu server process (started on-demand). It accepts simple commands, and responds with emacs s-expressions, in asynchronous fashion. In practice, all the heavy work is done in this server process, and emacs shows the results when it is ready (usually almost instantly). Emacs does not need to wait for the backend, and things stay snappy, even with tens of thousands (or more) e-mail messages. The only operation that may still require some waiting is sending mail, since that uses the emacs built-in smtpmail, which blocks during its operation.

mu expects your e-mail messages to be stored in a maildir (a one-file-per-message system); you can get these mails there through tools like offlineimap or fetchmail. mu periodically indexes the messages, and stores the results in a Xapian database. A lot of what mu4e does is querying this database – so for example, what you see as 'my inbox folder' in the mu4e-frontend, is in fact just the result of a query of all message that happen to live in that folder. You can query for many things – message sender, subject, date, words in the body and more.

mu4e does not just query messages; it can also move messages between folders, delete them, modify their flags and so on (and afterwards update the database). All 'state' is in the file system, and the database is just a quick way to get to it; if you delete the database and then re-index, nothing is lost, and changes made with other tools (e.g. mutt) are picked up by mu as well.

trying mu4e

Some Linux distributions ship mu4e, but not necessarily a very recent version; and because mu4e is developing quickly, you may want to use the latest official release, or (if you are more adventurous) check out the git repository. You'll need GNU/Emacs 23 or 24, GLib, GMime >= 2.4 and Xapian.

After installing mu4e, you can check the mu4e manual, which should be fairly complete. In fact, one of the goals of mu4e is to make it easy to set up; for that reason, the documentation includes complete examples of how to set things up. The Getting started chapter should get you up to speed quickly.

So, give it a try if you're interested; the mu=/=mu4e project is very open to suggestions (and even bugs reports!), so feel free to put them here or subscribe to the mu mailing list.

a small tour

I'll end this with a number of screenshots, to give you a rough idea of how things look.

The main screen

When you start mu4e (M-x mu4e), you're taken to the main screen, which list the things you can do. The entries should speak for themselves. Note that you can define you're own bookmarks. In general, mu4e tries to make things as easy as possible, and provide auto-completions where possible.

The headers / view split-screen

The results of your queries are shown as a list of headers; if you click one of the headers, mu4e splits the screen horizontally (optionally, you can split the screen vertically, or only show the message view). You can customize the headers shown, re-order them and so on.

The message view supports html message, inline images, and has a mechanism to associate user-functions with messages and attachments (such as 'view in browser' or 'pipe through command'); see the manual for some examples.

The message composition view

Obviously, you can reply to messages, forward them, or compose a new message. For writing and sending messages, mu4e reuses emacs' built-in message-mode. There's support for auto-completing addresses (based on the e-mails you've sent/received before).

There's even (experimental) support for writing you're messages using org-mode, and then sending them as rich-text (html) messages.

parting thoughts

So, this was a short introduction to mu4e, barely scratching the surface, but hopefully conveying the main ideas. The manual discusses things in much more detail, and I'll write more about it in the future.

mu4e is still a young project, but, for what's it worth, I've been using it full-time for over six months, and a growing number of people are doing the same. So, if you're interested, give it try and let us know you think!

Have fun!

file management with sunrise-commander

2012-08-10T18:03:00.000+03:00

I tend to do a lot of file management from the shell (zsh, in my case); this is very flexible / powerful and so on. But, for some things a bit more graphical approach is nicer.

There are fully graphical file-managers like Nautilus or Thunar, but for a bit more keyboard-friendly file-management, there are so-called orthodox file managers - the archetypical Norton Commander and its descendants, like midnight-commander (mc) and gnome commander.

Not surprisingly, emacs has its own incarnation - it is called the sunrise commander and happily it's obtainable through ELPA. It's become an important of my workflow. When you're used to Midnight Commander, you'll feel right at home.

If you want to use emacs for yet another of your computer-based activities, give it a try.

replace-regexp and numbering lines

2012-07-09T11:36:00.001+03:00

I saw the Got Emacs? posting showing off the new emacs-24 rectangle-number-lines command, to number a bunch of lines in buffer, i.e..:

foo
bar
cuux

becomes:

1 foo
2 bar
3 cuux

Very cool! An alternative is to use cua-mode, mark the column for the numbers with cua-set-rectangle-mark (C-RET), and then use M-x cua-sequence-rectangle (which takes you throught the steps, and has a lot of flexibility.

But let's look at yet another way: using replace-regexp. If we select (mark) the list once more, we can do M-x replace-regexp RET ^ RET \#. RET Note that the # is a special meta-character that represents the number of replacements already made. This has the somewhat clumsy side-effect that your list be numbered, starting at 0 rather than 1, so you should add a dummy-element at the beginning. Clearly, replace-regexp is inferior for simply adding some line numbers – however, it has the flexibility to do some smarter things.

Smarter things? Yes! replace-regexp allows you to use arbitrary Lisp-expressions in the replace strings. So, let's suppose that we want to use letters instead of numbers for our lines. Easy – again, select (mark) your lines, M-x replace-regexp RET ^ RET \,(format "%c. " (+ ?a \#)) RET and we get:

a. foo
b. bar
c. cuux

Admittedly, not the most world-shattering thing, but it does show the powers hidden in something as common as replace-regexp.

who holds this value?

2012-06-20T21:34:00.001+03:00

Something from the category of useful things hiding in emacs… Suppose you are looking for the variable that holds a certain value. How to find it?

Easy: M-x apropos-value

So, for example, finding all variables that hold your e-mail address:

M-x apropos-value RET me@example.com RET

and you'll get all the matches in the *Apropos*-buffer. HT: Stephen Eglen.

Also check the various other M-x apropos-... commands, they all help you find useful information if you can remember a word. Except for… M-x apropos-zippy… eh?

euro 2012 games in your org-mode agenda

2012-06-06T07:42:00.001+03:00

Things have been rather quiet at emacs-fu - reason for this is that most of my emacs hacking time has been spent on mu4e, the emacs e-mail client I wrote. It's been shaping up pretty nicely, I should probably write some emacs-fu posts about it :)

Another interesting pastime (esp. in Europe) is football/soccer, in particular the Euro2012 games; long-time readers will remember the schedule for world cup games; I made a new one for Euro2012: https://github.com/djcb/org-euro2012.

In order to have the games show up in your agenda, make sure the file is in your org-agenda-files. If needed, you could add it with something like this in your org-mode settings (change the directory path to wherever you have put euro2012.org):

(add-to-list 'org-agenda-files "~/org/euro2012.org")

One small issue with the schedule is that it uses the central-european summer time (UTC+2), and there is no automatic way to adjust times for the local time zone. As a work-around, Juan Pechiar provided the following function which makes it easy to update all org-timestamps in a file:

(defun update-org-hours (n)
  "Change all org-mode timestamps in the current buffer by N hours."
  (interactive "nAdd hours: ")
  (save-excursion
    (goto-char (point-min))
    (while (re-search-forward "[[<]" nil t)
      (when (org-at-timestamp-p t)
        (org-timestamp-change n 'hour)))))

Evaluate this function. After that, you can go to the file with the schedule, and give an M-x update-org-hours, provide the offset for your timezone, compared to UTC+2.

Let the games begin!

social networking with bitlbee and erc

2012-03-16T18:44:00.001+02:00

Instant messaging (IM) is one of the great time sinks of our age. Emacs-users, social butterflies as we are, of course want to take part in that -- preferrably from the comfort of our own environment.

There are various ways to use services like MS Live Messenger, Facebook Chat, GTalk, Yahoo etc. from within emacs – the one I use is called BitlBee. BitlBee is a program that presents all the various forms of IM as IRC-channels. In other words, while BitlBee knows all the details about communicating with these networks, all you need is an IRC-client to connect to BitlBee. This IRC-client does not have to be Emacs-based - any client can be used - but I am using ERC. Note, the below assumes you are somewhat familiar with it.

So, let's see how we can set up BitlBee on our system; there are public BitlBee-servers available online, but in the case of IM, I wouldn't necessarily want to trust them with my account data… so I'm using my own.

Setting up Bitlbee

So, how can we set this up? First, install BitlBee – many distributions have ready-made packages, otherwise you can build from source. In the examples below, I am assuming you are using Bitlbee version 3 or higher; if you use a different version, the details will vary a bit.

You can either run Bitlbee as a system-wide daemon, or just use it on your own. I am doing the former (for the latter, bitlbee.el is useful).

To connect ERC to the Bitlbee daemon, you can use something like the following:

(defun i-wanna-be-social ()
  "Connect to IM networks using bitlbee."
  (interactive)
  (erc :server "localhost" :port 6667 :nick "user"))

I'm sure you can come up with a better nick than user… Anyhow, with this function in your ~/.emacs, we can connect to bitlbee with:

M-x i-wanna-be-social RET

This should connect us to BitlBee; when all goes well, this will look something like this:

*** You have joined channel &bitlbee
*** mindcrime has changed mode for &bitlbee to +t
*** Users on &bitlbee: @user @root
*** Topic for &bitlbee: Welcome to the control channel. Type help for help
    information.
<root> Welcome to the BitlBee gateway!
<root>
<root> If you've never used BitlBee before, please do read the help
  information using the help command. Lots of FAQs are answered there.
<root> If you already have an account on this server, just use the identify
  command to identify yourself.
<root> The nick is (probably) not registered
*** &bitlbee modes: +t
<ERC>

Now, this first time, you will need to register yourself (this is only needed once); use the same nick (user in the example) that you used before:

<user> register user secretpassword
<root> Account successfully created

We're registered! This means, that bitlbee knows about you, and will save your settings.

Re-entering bitlbee

Just to complete the bitlbee-connecting part: the next time you want to use bitlbee, use i-wanna-be-social as before. However, now you need to identify yourself (rather than register):

<user> identify user secretpassword
<root> Password accepted, settings and accounts loaded

This can be automated by adding something like the following to your config:

(defun bitlbee-identify ()
  (when (and (string= "localhost" erc-session-server)
          (string= "&bitlbee" (buffer-name)))
    (erc-message "PRIVMSG" (format "%s identify user secretpassword"
                             (erc-default-target)
                             djcb-bitlbee-password))))

(add-hook 'erc-join-hook 'bitlbee-identify)

Modify user and secretpassword as desired. If you don't want write out your passwords in your emacs config files, take a look at keeping your secrets secret.

Adding accounts

Now, let's add some IM-accounts (just some examples here; also see Bitlbee Quickstart. Note, add any point during this, you can see your accounts with the command:

<user> account list

and using the numbers (or the accounts 'tag') you can switch an account on (and off):

<user> account 0 on

There are many other commands – use help command the consult the built-in documentation.

Connecting to IM networks

To connect to IM networks, you have to add the accounts to BitlBee. It will save them, so you only need to do that once. Let's do it - in each case, replace user and password with whatever you use for those.

You can see your combined logged-in buddy list using M-x erc-channel-names (or C-c C-n). Or use the blist command (see help blist).

After adding an account, it's a good idea to issue the save command, to ensure that bitlbee saves it.

Jabber

<user> account add jabber foobar@jabber.org mypassword
<root> Account successfully added with tag jabber
<user> account jabber on
<root> jabber - Logging in: Connecting
<root> jabber - Logging in: Connected to server, logging in
<root> jabber - Logging in: Converting stream to TLS
<root> jabber - Logging in: Connected to server, logging in
<root> jabber - Logging in: Authentication finished
<root> jabber - Logging in: Authenticated, requesting buddy list
<root> jabber - Logging in: Logged in

(and of course, you can use other servers beside jabber.org)

GoogleTalk

GoogleTalk (gtalk) is implemented using XMPP (jabber), and it's recommended to use oauth for authentication. Note, the foobar below is just a placeholder (for a password); bitlbee wants us to put something there, but since we'll be using oauth, it's not actually used.

<user> account add jabber myaccount@gmail.com foobar
<root> Account successfully added with tag gtalk
<user> account gtalk set oauth on
<root> oauth = `on'
<user> account gtalk set nick_source full_name
<root> nick_source = `full_name'
<user> account gtalk on
<root> gtalk - Logging in: Starting OAuth authentication

Now, a second window will open with a URL:

<jabber_oauth> Open this URL in your browser to authenticate:
https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.\
com/auth/googletalk&response_type=code&redirect_uri=urn:ietf:wg:oauth:2.0:oob&client_i\
d=78312399893489.apps.googleusercontent.com

<jabber_oauth> Respond to this message with the returned authorization
token.

Follow this URL in your browser, and it will take you to some Google page for authentication. When that is completed, you will receive some string cookie, which you paste back into the newly opened window.

<user> 4/sIns904fdlkP5nudjCF4mBHF7Go_-E0g8
*** jabber_oauth is AWAY: Offline

Et voilà! We're connected to Gtalk (don't worry about the Offline-warning).

Facebook

Apart from being a social website, Facebook can also be used for IM. You can do this through the website, or you can use its jabber interface. It works very similar to gtalk; only important thing is that you get yourself a Facebook username:

<user> account add jabber myusername0@chat.facebook.com
<root> Account successfully added with tag fb
<root> You can now use the /OPER command to enter the password
<root> Alternatively, enable OAuth if the account supports it: account
  fb set oauth on
<user> account fb set oauth on
<root> oauth = `on'
<user> account gtalk set nick_source full_name
<root> nick_source = `full_name'
<user> account fb on
<root> fb - Logging in: Starting OAuth authentication

Then, go through the oath-authentication steps (see the discussion about adding Gtalk accounts above).

Once authenticated, you'll get something like this:

<root> fb - Logging in: Requesting OAuth access token
<root> fb - Logging in: Connecting
<root> fb - Logging in: Connected to server, logging in
<root> fb - Logging in: Converting stream to TLS
<root> fb - Logging in: Connected to server, logging in
<root> fb - Logging in: Authentication finished
<root> fb - Logging in: Server claims your JID is
  `-748234518@chat.facebook.com' instead of
  `myusername0@chat.facebook.com'. This mismatch may cause problems with
  groupchats and possibly other things.
<root> fb - Logging in: Authenticated, requesting buddy list
<root> fb - Logging in: Logged in

It's to now act upon the warning, so, we log out, change the user name and long back in:

<user> account fb off
<root> fb - Signing off..
<user> account fb set username -748234518@chat.facebook.com
<root> username = `-748234518@chat.facebook.com'
<user> account fb on

MSN / Live Messenger

MSN uses its own protocol (although apparently they're also supporting XMPP ("jabber") now). Suppose you have an account there, user partygirl89@hotmail.com with password iamcute:

<user> account add msn partygirl89@hotmail.com iamcute
<root> Account successfully added
<user> account msn on
<root> msn - Logging in: Connecting
<root> msn - Logging in: Connected to server, waiting for reply
<root> msn - Logging in: Transferring to other server
<root> msn - Logging in: Connected to server, waiting for reply
<root> msn - Logging in: Authenticated, getting buddy list
<root> msn - Login error: Error reported by MSN server: Invalid
       (non-existent) handle                                      [12:17]
<root> msn - Logging in: Logged in

The 'Error reported' does not seem to matter.

Other accounts

It's similarly easy to setup Twitter-accounts and Identi.ca-accounts; I've stopped using those though, as it turned out to be a little too easy for some typing in the wrong window to end op as a tweet… The risk is less with twittering-mode and identica-mode.

For ICQ/Yahoo/AIM see below – replace the username/password with your own.

ICQ

<user> account add oscar ICQ-ID PASSWORD login.icq.com
<root> ...

AIM

<user> account add oscar AIM-NICK PASSWORD login.aol.oscar.com
<root> ...

Yahoo!
```
<user> account add yahoo YAHOO-NICK PASSWORD
<root> ...
```
And I'm not even talking about combining bitlbee and Skype – yes, that is possible, too.

Chatting

Now, chatting is easy, following the normal ERC conventions (and settings). When people talk to you, a window opens (or a frame – see erc-auto-query. And you can initiate conversations with people by using /msg nick, with nick of course being the nickname of the person you want to talk to.

ERC/Bitlbee also work together nicely with Sauron, the emacs event tracker.

Have fun! I have only scratched the surface here - you now have the full arsenal of Elisp and ERC power available for your chatting.

special characters

2012-02-10T21:57:00.003+02:00

Special characters

Living in a post-ASCII world offers great opportunities, but brings some problems, too. After all, it's nice to be able to write Ångström or Καλημέρα or ☺☎☯, but it's not necessarily easy to enter those characters.

input methods

So - what to do? First, you can set the input-method, as explained in the emacs manual. This is the best solution if you're writing a non-Latin language – Russian, Thai, Japanese, …

If you only occasionally need some accented character, input methods like latin-postfix (e" -> ë), latin-prefix ("e -> ë) or TeX (\"e -> ë) are useful. They also tend to annoy me a bit, as they often assume I need an accented character, when all I want is to put a word in quotation marks…

compose key

Another method is to use a special compose key; for example, under Gnome 3 it's in the the Region and Language applet, under Options..., in Gnome 2 it's in the Keyboard applet in Layouts/Options…. This works for all programs, not just emacs (see this Ubuntu help page for some details). I've set my Compose-key to Right-Alt, so Right-Alt "e -> ë.

Using the compose key works pretty well for me; setting the input method may be more convenient when you need to write a lot of accented characters.

Now, his may be good and well for the accented characters and other variants of Latin characters, such as the German Ess-Zet ligature "ß" (note, you can get that character with latin-prefix "s -> ß, latin-postfix s" -> ß or <compose-key> ss -> ß). But what about Greek characters? Mathematical symbols? Smileys?

ucs-insert

One way to add characters like α, ∞ or ☺ is to use ucs-insert, by default bound to C-x 8 RET. If you know the official Unicode name for a character, you can find it there; note that there's auto-completion and you can use * wild-cards. For the mentioned characters, that would be GREEK SMALL LETTER ALPHA, INFINITY and WHITE SMILING FACE.

You can also use the Unicode code points; so C-x 8 RET 03b1 RET will insert α as well, since its code point is U+03B1. In case you don't know the code points of Unicode characters, a tool like the Character Map (gucharmap) in Gnome may be useful.

abbrev

Since ucs-insert may not be convenient in all cases, you may want to add shortcuts for oft-used special characters to you abbrev table. See the entry on Abbrevs in the emacs manual. I usually edit the entries by hand with M-x edit-abbrevs, and I have entries like:

(text-mode-abbrev-table)
"Delta0"       0    "Δ"
"^2"           0    "²"
"^3"           0    "³"
"almost0"      0    "≈"
"alpha0"       0    "α"
"any0"         0    "∀"
"beta0"        0    "β"
"c0"           0    "©"
"deg0"         0    "℃"
"delta0"       0    "δ"
"elm0"         0    "∈"
"epsilon0"     0    "ε"
"eta0"         0    "η"
"heart0"       0    "♥"
"inf0"         0    "∞"
"int0"         0    "∫"
"notis0"       0    "≠"

Now, alpha0 will be auto-replaced with α. I'm using the 0 suffix for most entries so I can easily remember them, without making it hard to use alpha as a normal word. Note, abbrevs are a bit picky when it comes to the characters in the shortcut – for example, setting != -> ≠ won't work.

inheriting abbrevs from other modes

If you have set up a nice set of abbreviations for text-mode, you may want to use them in other modes as well; you can accomplish this by including the text-mode abbreviations into the table for the current one, for example in your ERC setup:

;; inherit abbrevs from text-mode
(abbrev-table-put erc-mode-abbrev-table :parents (list text-mode-abbrev-table))

update

2012-02-01T20:24:00.000+02:00

Regular emacs-fu programming will resume shortly, but for now I'll provide a brief update on some of my projects.

Recently, I discussed sauron, and event-tracking tool for emacs. I added some new features, fixed some bugs, and got some contributions (yay!)
- tweak/improve priority handling
- add a backend for the emacs-24 notification system
- add settings to make the sauron frame 'sticky', and to hide the mode-line
- enable dbus-message source outside your session; useful for cron/procmail etc.
- display events in a tabular fashion
- add support for John Wiegley's event.el
- much improved documentation
- other things…
Sauron version 0.2 is available through Marmalade (see Package Management Revisited)
I released version 0.9.8 of mu (e-mail searcher/indexer, previously discussed in Searching e-mails with Wanderlust and mu). Now, for emacs-users I've added mu4e (manual), a new & experimental e-mail client. It won't be big and professional like gnus, but it's fun to hack on, and I've been using it for a few months.

sauron: keeping an eye on what's going on

2011-12-22T00:01:00.000+02:00

I'm a fairly busy person, and need to keep track of a lot of things. That includes following a bunch of internal IRC channels, attending meetings, meeting deadlines and so on. But I don't want to stare at my org-mode calendar, or flip through ERC buffers all the time.

Instead, I'd like to have one little emacs frame (window) that gathers these ('events'), and transfers me to wherever the event came from when I click it - some IRC-channel in ERC, my org-calendar etc. and other inputs. Note, using Bitlbee, you can include Facebook-contacts, GoogleTalk-buddies and Twitter-tweets, … in ERC - so, you can track just about anything.

In addition, with so many inputs, I'd also like the possibility to filter out unwanted events, and generate various light/sound effects and fireworks, proportional to the priority of the event.

For all this, I wrote a little emacs-tool called Sauron that does just that. M-x sauron-start pops up a frame that receives events, and M-x sauron-stop hides it and stops listening. It works with ERC, org, and listens for D-Bus messages; so it's pretty easy to get events from all over the place.

It's a bit of a balancing act to get all the important information while not being swamped in noise, but Sauron allows you to fine-tune it to whatever works the best for you. I've tried to have sane defaults though, so things should mostly work without too much configuration - but if you need the power, it's there. I also added some convenience functions to make it easy to get sounds and other special effects.

So - it's brand new, it is of seems-to-work-for-me-quality, and I'd like to invite others to try it out, hack it, give feedback, add new back-ends and so on – what better Christmas present to ask for!

There's documentation, examples etc. to be found in Sauron's github repository.

system administration with emacs

2011-12-10T14:33:00.001+02:00

When performing system administration tasks, one often needs to edit files owned by root.

For both security and safety reasons, it's a good idea to do as little as possible as root (or with root privileges). For that reason, you probably don't want to run Emacs as 'root', because it's simply too powerful. I often see people use vi (usually, vim) instead – but since it allows you to do just about anything as well (like running shell commands), that's not much of an improvement.

Now, a while back we discussed editing files owned by root with tramp – you can use emacs with your normal user-account, and use sudo to write the root-owned file. That makes it much harder to screw things up.

Another reason people use vi for little editing jobs is because its startup time is significantly shorter than the startup time for a new emacs instance. For that, however, we have emacs daemon.

Combining tramp and emacs daemon and a shell function:

# edit file with root privs
function E() {
         emacsclient -c -a emacs "/sudo:root@localhost:$1"
}

Now, we can very quickly edit any file owned by root using 'E' --

$ E /etc/hosts

So, if you prefer emacs, there's little reason to use vi, even for editing system files – although I have to admit that it takes time to evict 'sudo vi <system file>' from my muscle memory.

Update reader Yi Wang mentions that, in fact, we can make this a bit more general using sudoedit; so, instead of using Tramp, we can use:

# edit file with root privs
alias E="SUDO_EDITOR=\"emacsclient -c -a emacs\" sudoedit"

This works also without absolute paths.

package management revisited

2011-11-21T22:22:00.001+02:00

Ages ago we discussed ELPA, the emacs packaging system. It allows you to
browse through an online repository of emacs packages and install, uninstall
and upgrade them, all from within the emacs comfort zone (menu:
Options/Manage Emacs Packages).

Emacs 24.1 (a pretest is available) will include a version of ELPA --
package.el; the version included with emacs 24 supports multiple
archives. The official archive is the one provided by GNU, then there is
the current one for ELPA, but there's now a third one available: Marmalade.

Marmelade ('Spreadable Elisp') offers a very easy way for developers to
make their packages available to emacs users, and an even easier way for
users to test out interesting packages.

To use all available package repositories (GNU, ELPA and Marmalade), add
the following snippet to your .emacs:

(setq package-archives '(("ELPA" . "http://tromey.com/elpa/") 
                          ("gnu" . "http://elpa.gnu.org/packages/")
                          ("marmalade" . "http://marmalade-repo.org/packages/")))

This works out-of-the-box for Emacs 24; for Emacs 23, you'll need the newer
package.el, which you can get from the emacs 24 repo (if necessary, remove
the old package.el first).

Currently, there are 870 packages listed in the three repositories – a lot of
toys to play with! In fact, it is already getting quite hard to find what
you're looking for; a bit more structured way (maybe using categories) would
be useful. Also, a bit more information than the one-line description would
be very useful.

Nevertheless, it's a great addition to emacs, which will make it much easier
to play with packages – without even having to read the installation
instructions.

quick note-taking with deft and org-mode

2011-09-15T21:49:00.001+03:00

Emacs must be gathering a lot of enthusiasts lately; there's hardly a week where I don't discover some new gem. Recently, I discovered deft. And apparently, I wasn't the only one.

So what is it deft good for? Well, often I want to jot down some quick thing during a meeting or a telephone-call. Of course, I don't want to think about file names or anything else distracting me from my task, just get me that note already! In addition, at some later time, I want to be able to quickly search through the notes I made.

For MacOS, there's a program called Notational Velocity which does this. But really - it sounds like a typical task for emacs - wouldn't it be nice to have an emacs package that does roughly the same?

And that is what deft does - enable you to quickly write notes, and retrieving them later. The author has an excellent introduction on his website, so my job is very easy :) deft is not part of org-mode, but they can work together seamlessly. Here's my set-up:

;; http://jblevins.org/projects/deft/
(when (require 'deft nil 'noerror) 
   (setq
      deft-extension "org"
      deft-directory "~/Org/deft/"
      deft-text-mode 'org-mode)
   (global-set-key (kbd "<f9>") 'deft))

This blob goes in my .emacs. Note, the first line ensures that emacs starts without errors, even when I run on a system without deft. Apart from that, I make deft use org files for note taking, which makes it all very familiar.

All notes are saved ~/Org/deft - you can set it to something else of course. A Dropbox-folder seems to be a popular choice for synchronizing between machines.

Finally, the last line binds F9 to deft-mode. So, when I need a quick note, I can type F9 C-c C-n and start writing.

finding just about anything

2011-09-08T23:22:00.000+03:00

Buffer-switching is one of those things I do all the time in Emacs. And it should be quick and efficient, or it will break my 'flow'. There are many ways to customize the buffer switching experience, and we already discussed quite a few of those here: using Ctrl-Tab, iswitchb/ido, ibuffer, the lusty explorer and others.

Some of those packages - ido and lusty-explorer - do not solely focus on buffer-switching - they also let you open files using the same user interface. But why stop at files and buffers - wouldn't it be nice if we could quickly find just about anything quickly?

Indeed that would be nice - and there's a way to do just that - using the aptly named anything package. I was always a bit put off by this package due to the screenshots (see link), but once I got over that, I've become a very happy user.

Anyhow, what can anything find? I mentioned buffers and files, but it can also find bookmarks, recently used files, any file on your system (using locate), man-pages, info-pages, emacs-function/variables, FIXME-comments, org-headlines, bbdb-contacts, google-suggests… and a million other things. It can probably find your car keys, too.

Installation is pretty straightforward, using the git-repository:

git clone git://repo.or.cz/anything-config.git

Then, go into the just-cloned directory and execute make. After that, add to your .emacs (or ~/.emacs.d/init.el):

;; path-to-anything is the path which has the 'anything' we just cloned
(add-to-list 'load-path "path-to-anything")
(require 'anything-config)

This will add a menu with various anything-commands, and a bunch of key-bindings, starting with F5. Play around a bit with it, it's nice. The results are shown in a little temporary buffer, and pressing Tab will let you do (search-type dependent) actions on the matches - for example ediff two buffers.

Of course, the real fun starts when we super-charge some of the normal emacs functions with anything-based ones. Let's look at buffer switching. Let's create an anything-based version, and assign it to C-x b, the good-old switch-to-buffer.

The thing to consider is that while anything can find just about anything, you (well, I) usually only want to search for a certain set of things; when I want to switch to another buffer, I don't want to match man-pages. Luckily, anything allows for making nifty function which use certain subsets of the search sources. So for buffer switching, I have:

(global-set-key (kbd "C-x b")
  (lambda() (interactive)
    (anything
     :prompt "Switch to: "
     :candidate-number-limit 10                 ;; up to 10 of each 
     :sources
     '( anything-c-source-buffers               ;; buffers 
        anything-c-source-recentf               ;; recent files 
        anything-c-source-bookmarks             ;; bookmarks
        anything-c-source-files-in-current-dir+ ;; current dir
        anything-c-source-locate))))            ;; use 'locate'

This will search in buffers, then in my recent-files, then in my bookmarks, the files in the current directory, and finally check with the locate tool. That last one is pretty neat, and finally gives me something back for the countless times I've wondered why the hard disk is grinding – indeed, it was locate, updating its database.

Then, I've defined another binding for searching general documentation on my system; I've put it under C-c I. This looks look something like the following:

(global-set-key (kbd "C-c I")  ;; i -> info
  (lambda () (interactive)
    (anything
      :prompt "Info about: "
      :candidate-number-limit 3
      :sources
      '( anything-c-source-info-libc             ;; glibc docs
         anything-c-source-man-pages             ;; man pages
         anything-c-source-info-emacs))))        ;; emacs

These are sources I query quite regularly; there are many more to be found - for most packages with info-pages there's a corresponding anything-c-source-info-...; there's a list in anything-config.el.

Now, those are my general documentation sources; in specific modes, I have specialized information sources; for example, for elisp-mode:

(add-hook 'emacs-lisp-mode-hook
  (lambda()
  ;; other stuff...
  ;; ...
  ;; put useful info under C-c i
    (local-set-key (kbd "C-c i")
      (lambda() (interactive)
        (anything
          :prompt "Info about: "
          :candidate-number-limit 5
          :sources
          '( anything-c-source-emacs-functions
             anything-c-source-emacs-variables
             anything-c-source-info-elisp
             anything-c-source-emacs-commands
             anything-c-source-emacs-source-defun
             anything-c-source-emacs-lisp-expectations
             anything-c-source-emacs-lisp-toplevels
             anything-c-source-emacs-functions-with-abbrevs
             anything-c-source-info-emacs))))

This post just scratches the surface of what is possible – so go and experiment :) One interesting thing is to add your own sources; I played a bit with that already,

There are many things in anything I did not cover yet. First, there are many more sources to search - and it's pretty easy to write your own – see the EmacsWiki-page.

customizing the mode-line

2011-08-25T23:48:00.001+03:00

The mode-line is the emacs 'status bar', the bar just above the minibuffer that shows various pieces of information, such as the buffer name, the major mode, maybe the current line number, some indicators for active minor modes, and so on. As I'm looking at it, it starts with 1<U:**- (which is: input-method: latin-1-alt-postfix, buffer-coding-system: utf8-unix, line-ending: unix-style, buffer is writable and buffer is modified – the tooltips help).
As with just about anything in emacs, the mode-line can be customized just the way you like. I give some example below, not because I think it is necessarily the best way, but just to give you a bit of an example to start with when making your own best-mode-line-ever.
I'm not going through all the details of the example, but let me highlight a few things that make it a bit easier to understand.
First of all, the mode-line can be customized by setting the variable mode-line-format; this variable becomes buffer-local automatically when changed, so if you want to set it for all buffers, you'll need to use setq-default in your .emacs (or equivalent). The format is quite similar to the one for frame-title-format, which we discussed in setting the frame title a while back.
mode-line-format is a list of items which are evaluated, and put together as a string which then ends up as the mode-line contents. These properties can be any string. The following types of items can be used:

First, normal strings are just shown as-is;
Then, there are some special format parameters which will be replaced with their value in the mode-line, from the Emacs-documentation:

  %b -- print buffer name.      %f -- print visited file name.
  %F -- print frame name.
  %* -- print %, * or hyphen.   %+ -- print *, % or hyphen.
        %& is like %*, but ignore read-only-ness.
        % means buffer is read-only and * means it is modified.
        For a modified read-only buffer, %* gives % and %+ gives *.
  %s -- print process status.   %l -- print the current line number.
  %c -- print the current column number (this makes editing slower).
        To make the column number update correctly in all cases,
        `column-number-mode' must be non-nil.
  %i -- print the size of the buffer.
  %I -- like %i, but use k, M, G, etc., to abbreviate.
  %p -- print percent of buffer above top of window, or Top, Bot or All.
  %P -- print percent of buffer above bottom of window, perhaps plus Top,
        or print Bottom or All.
  %n -- print Narrow if appropriate.
  %t -- visited file is text or binary (if OS supports this distinction).
  %z -- print mnemonics of keyboard, terminal, and buffer coding systems.
  %Z -- like %z, but including the end-of-line format.
  %e -- print error message about full memory.
  %@ -- print @ or hyphen.  @ means that default-directory is on a
        remote machine.
  %[ -- print one [ for each recursive editing level.  %] similar.
  %% -- print %.   %- -- print infinitely many dashes.
Decimal digits after the % specify field width to which to pad.

Forms of the type (:eval ...) are evaluated each time the mode-line is drawn (just like the '%'-parameters) ; so, if you have a value that changes of the course your emacs session, you should use (:eval ...). For example, for your emacs-uptime you could use (:eval (emacs-uptime "%hh")); while the emacs-PID does not change, so simply you could simply use (format "PID:%d").
The format parameter mentioned above are of evaluated each time as well. Note that you have to be a bit careful with evaluations - don't do too heavy operations there, and be careful the updates don't recurse.
There are many others which I won't go into now - please check the Elisp reference. It's a rather baroque format…

Now, let's put this all together in an example (tested with emacs 23 and 24). As I said, this is for demonstration purposes only; but hopefully it gives you some inspiration. A lot of the 'magic' (colors, tooltips, faces) happens with the propertize function; again, the Elisp documentation can tell you a lot more about that. I'm (ab)using the various font-lock-faces to have colors that blend in nicely with your current theme.
And it has a limitation still, namely that it does not react to mouse clicks; how to that, I will discuss in some future article.

;; use setq-default to set it for /all/ modes
(setq mode-line-format
  (list
    ;; the buffer name; the file name as a tool tip
    '(:eval (propertize "%b " 'face 'font-lock-keyword-face
        'help-echo (buffer-file-name)))

    ;; line and column
    "(" ;; '%02' to set to 2 chars at least; prevents flickering
      (propertize "%02l" 'face 'font-lock-type-face) ","
      (propertize "%02c" 'face 'font-lock-type-face) 
    ") "

    ;; relative position, size of file
    "["
    (propertize "%p" 'face 'font-lock-constant-face) ;; % above top
    "/"
    (propertize "%I" 'face 'font-lock-constant-face) ;; size
    "] "

    ;; the current major mode for the buffer.
    "["

    '(:eval (propertize "%m" 'face 'font-lock-string-face
              'help-echo buffer-file-coding-system))
    "] "


    "[" ;; insert vs overwrite mode, input-method in a tooltip
    '(:eval (propertize (if overwrite-mode "Ovr" "Ins")
              'face 'font-lock-preprocessor-face
              'help-echo (concat "Buffer is in "
                           (if overwrite-mode "overwrite" "insert") " mode")))

    ;; was this buffer modified since the last save?
    '(:eval (when (buffer-modified-p)
              (concat ","  (propertize "Mod"
                             'face 'font-lock-warning-face
                             'help-echo "Buffer has been modified"))))

    ;; is this buffer read-only?
    '(:eval (when buffer-read-only
              (concat ","  (propertize "RO"
                             'face 'font-lock-type-face
                             'help-echo "Buffer is read-only"))))  
    "] "

    ;; add the time, with the date and the emacs uptime in the tooltip
    '(:eval (propertize (format-time-string "%H:%M")
              'help-echo
              (concat (format-time-string "%c; ")
                      (emacs-uptime "Uptime:%hh"))))
    " --"
    ;; i don't want to see minor-modes; but if you want, uncomment this:
    ;; minor-mode-alist  ;; list of minor modes
    "%-" ;; fill with '-'
    ))

Have fun playing with this!

toward balanced and colorful delimiters

2011-05-16T22:59:00.001+03:00

Emacs has the very useful feature of blinking the corresponding left "(" when you type its partner ")" (and the same for other delimiters like "[]", "{}" etc., depending on the mode). This is very useful for programming, especially for languages like Lisp and Scheme and the like.

Further help can come from tools like autopair or paredit – although the latter is a bit too much bondage & discipline for me, many people love it.
Anyway, recently I discovered a new helper in the quest for balance in delimited universe: rainbow-delimiters. With this package, the delimiters all get different colors based on their nesting level. It works wonderfully well.
Installation is straightforward:

download rainbow-delimiters and put it in your load-path
add something like the following in your .emacs:

(when (require 'rainbow-delimiters nil 'noerror) 
  (add-hook 'scheme-mode-hook 'rainbow-delimiters-mode))

That's it – of course you'll need to do the equivalent for modes where you'd like to enable it. See the screenshot below – maybe a bit too colorful for some people, but I like it and I have found it actually useful to see the corresponding delimiters without having to move the cursor over them.

I already added some nice Zenburn colors for this to the Zenburn theme for Emacs-24, as can be seen in the screenshot.

porting the zenburn theme to emacs 24

2011-05-08T12:05:00.000+03:00

Just a small note: I already mentioned the zenburn color theme a couple of times. I have now ported the full (big) set of customizations to the emacs-24 format – find it at github; so if you run emacs 24 (the development version) and enjoy zenburn like me, you can give it try. Suggestions, additions are welcome.

Note, there seems to be a limitation with the :inherit-attribute for themes – in practice it means that some aspects of the theme may not work if a package sets the face before the theme does. Therefore, best to load the theme early from your .emacs or init.el; or alternatively visit the theme file (zenburn-theme.el in this case) and evaluate the buffer.

nice-looking pdfs with org-mode and xetex

2011-04-12T20:57:00.000+03:00

I've discussed the wonderful org-mode here a number of times already. It has become a pretty important part of my overall workflow. One thing I am using org-mode for, is to produce all kinds of PDF-documents that I can share with other people.

org-mode & LaTeX

In the past, I often used straight LaTeX for such things; I wrote my thesis with it, but also many other documents. There are many things I like about LaTeX, one of them being that I can use emacs for writing. Still, there are also a few things I do not particularly like. First, I think LaTeX is quite heavy with formatting directives, which hinder my writing flow (e.g., when I want to include an image, a table or a source code snippet). Another thing is that I find the default LaTeX styles a bit boring. Nothing wrong with it, but there just too many documents with the exact same lay-out.

Now, back to org-mode. One way to use org-mode is as a friendly way to generate LaTeX (and, consequently, PDFs). This is a big improvement! Much more than LaTeX itself, org-mode allows to focus on the contents of the document, rather than instructing LaTeX what to do. This comes at the price of small bit of flexibility, but, if needed org-mode allows you include straight LaTeX when needed – so while keeping easy things easy, hard things are still possible. The latter does require a bit of experience with LaTeX a though.

setting up XeTeX

Now, for the second issue, the way documents look, there are other solutions, and they live on the LaTeX side of things. I'm sure many have seen The Beauty of LaTeX. Using the XeTeX implementation of LaTeX and the fontspec package, you can create LaTeX documents with a bit 'refreshed' look.

So, the steps to get this working with org-mode:

install the texlive-xetex packages on Ubuntu and Debian (this installs a huge set of packages)
install the SIL fonts (I'm using ttf-sil-gentium and ttf-sil-charis, but there are more)
I'm also using DejaVu Mono (ttf-dejavu)

teaching org-mode about the new XeTeX stuff

We now need to define some LaTeX document class for org-mode that uses XeTeX and some of these new fonts. Let's call the document class djcb-org-article (as I often use the djcb- prefix for my own stuff), it could be something like the following (add to your org-setup – e.g., in your .emacs, make sure there is a (require 'org) before this:

;; 'djcb-org-article' for export org documents to the LaTex 'article', using
;; XeTeX and some fancy fonts; requires XeTeX (see org-latex-to-pdf-process)
(add-to-list 'org-export-latex-classes
  '("djcb-org-article"
"\\documentclass[11pt,a4paper]{article}
\\usepackage[T1]{fontenc}
\\usepackage{fontspec}
\\usepackage{graphicx} 
\\defaultfontfeatures{Mapping=tex-text}
\\setromanfont{Gentium}
\\setromanfont [BoldFont={Gentium Basic Bold},
                ItalicFont={Gentium Basic Italic}]{Gentium Basic}
\\setsansfont{Charis SIL}
\\setmonofont[Scale=0.8]{DejaVu Sans Mono}
\\usepackage{geometry}
\\geometry{a4paper, textwidth=6.5in, textheight=10in,
            marginparsep=7pt, marginparwidth=.6in}
\\pagestyle{empty}
\\title{}
      [NO-DEFAULT-PACKAGES]
      [NO-PACKAGES]"
     ("\\section{%s}" . "\\section*{%s}")
     ("\\subsection{%s}" . "\\subsection*{%s}")
     ("\\subsubsection{%s}" . "\\subsubsection*{%s}")
     ("\\paragraph{%s}" . "\\paragraph*{%s}")
     ("\\subparagraph{%s}" . "\\subparagraph*{%s}")))

Of course, this can be customized to your own preference; e.g., North-Americans may not be using A4-paper.

org-mode takes care of the export from its own format to LaTeX, but we need to tell it to use xelatex to process the LaTeX to PDF:

(setq org-latex-to-pdf-process 
  '("xelatex -interaction nonstopmode %f"
     "xelatex -interaction nonstopmode %f")) ;; for multiple passes

That's all that's needed on the setup-side.

creating a document

Now, let's create a little test document, test.org, to show how it works:

#+LaTeX_CLASS: djcb-org-article
#+TITLE: My little document

* Introduction
  
  This is my document. There are many like it, but this is mine. It's easy to
  write without *too* _many_ /distractions/.
  
** Normal distribution

   Probability density of the normal distribution, using familiar TeX notation
   for formulae:
 
   $$\frac{1}{\sqrt{2\pi\sigma^2}}e^{ -\frac{(x-\mu)^2}{2\sigma^2} }$$

** Some table

| *Greek God* | *Roman God* | *Element*      |
|-------------+-------------+----------------|
| Zeus        | Jupiter     | Sky and clouds |
| Hera        | Juno        | Family         |
| Poseidon    | Neptune     | Sea            |
| Hades       | Pluto       | Underworld     |

We can export this to a PDF using C-c C-e p (or C-c C-e d to automatically open the PDF in a PDF-viewer). This should all work nicely; if it doesn't, note that when exporting, say, test.org, org-mode will create a file called test.tex, and visit in a buffer. There's also a buffer with the output from various commands, but sometimes it can be useful to run LaTeX (xelatex in this case) on the file by hand, to find any problems. The wonderful org-documentation about exporting to LaTeX has more information.

I think the result is pretty nice – it stays true to the class LaTeX article class, but freshens it up a bit with some news font. If you can make something better – which is not unlikely – you are of course invited to contribute your own!

Concluding

org-mode is a pretty convenient way to write nice-looking PDFs. Combined with xelatex, they don't have to look too plain :). However, I'm aware of my limitations when it comes to the coolness/aesthetic aspects, but I hope others can show the way here.

Maybe org-mode could ship with a number of ready-made templates to make it easy to make nice-looking documents, resumes, reference cards, reports, meeting notes and so on.

searching e-mails with wanderlust and mu

2011-03-31T21:10:00.002+03:00

I have discussed the Wanderlust e-mail client a couple of times already. I'm still using it, so I keep on learning new tricks. Even though there has been quite a bit of action in the competing gnus e-mail client, for my particular use-case, Wanderlust is still the best option.

'My particular use-case' consists of storing my mail in Maildirs, which I fill with either offlineimap (which has fortunately found a new maintainer) or fetchmail.

mu

When dealing with e-mail, one particularly important feature for me is the ability to search my messages. In fact, it's so important for me that I wrote some software to do this for me; the software is called mu; it indexes the messages in my Maildirs, and then allows for searching them using queries, based on message contents, headers, or other message properties.

mu works through a command-line interface, although there is an experimental GUI available as well. The command-line interface makes it possible to hook mu up with various mail-clients, such as mutt, or Wanderlust. Some Linux distributions ship mu, but since the versions they ship are often a bit outdated, I recommend building it yourself from the sources linked on the mu website. The process is fairly straightforward; and there is plenty of documentation in the form of man pages.

mu and wanderlust

I've been combining mu and wanderlust for a while (see mu and wanderlust - the old way, below), but this week Sam B. on the mu mailing list showed a way to do so in a much more elegant way - using virtual or query folders.

How does this work? Well, after installing mu, add the following to your Wanderlust setup file (~/.wl or it's moral equivalent – see the older Wanderlust posts for the details):

(require 'elmo-search)
(elmo-search-register-engine
    'mu 'local-file
    :prog "/usr/local/bin/mu" ;; or wherever you've installed it
    :args '("find" pattern "--fields" "l") :charset 'utf-8)

(setq elmo-search-default-engine 'mu)
;; for when you type "g" in folder or summary.
(setq wl-default-spec "[")

So, to start with the last part, whenever you type g in folder or summary, in the mode-line you will get something like Folder name (.inbox): [. Now simply type your mu search expression and press Enter, and wanderlust opens a (temporary) folder with the search results. Brilliant!

Next, to add virtual folders for searches you do often, simply add some folder specifications like the following to your .folders file (again, check the older Wanderlust posts if you're not familiar with folders-file):

VFolders {
# message I received today
  [date:today..now]!mu  "Today"

# messages bigger than 1Mb  
  [size:1m..100m]!mu    "Big"

# signed messages i got in 2010 related to emacs
  [date:2010..2011 flag:signed emacs]!mu "Signed-Emacs2010"

# unread messages
  [not flag:seen]!mu    "Unread"
# or (for mu  >= 0.9.4):
# [flag:unread]! mu      "Unread"
}

After this, restart Wanderlust, and there you go! Wanderlust will display your brand new virtual folders with an icon that looks like a little whale.

You can put arbitrary mu search expressions between the [], matching whatever is useful in a certain case. Check the mu documentation to see how to do this.

Note, the messages you get in these virtual folders are links to the original messages. In practice, this means that changes you make to the links do no affect the originals – if you delete a link you're not deleting the message.

mu and wanderlust - the old way

This discussion would not complete without a description of the old way I used search. This method may still be useful for integrating mu with other clients such as mutt.

What I've been using for a while is a (in retrospect) rather clumsy way to integrate message searches with Wanderlust: based on the results of a query, I would create some special Maildir and fill it with symbolic links to the matched messages, and the visit this special Maildir with Wanderlust. I'll include the code here to contrast it with the more elegant solution that we saw before, but also because the approach taken might be easily adapted for other mail-clients.

;; search using mutt
(defvar mu-wl-mu-program     "/usr/local/bin/mu")
(defvar mu-wl-search-folder  "search")

(defun mu-wl-search ()
  "search for messages with `mu', and jump to the results"
   (let* ((muexpr (read-string "Find messages matching: "))
          (sfldr  (concat elmo-maildir-folder-path "/"
                    mu-wl-search-folder))
          (cmdline (concat mu-wl-mu-program " find "
                      "--clearlinks --format=links --linksdir='" sfldr "' "
                     muexpr))    
          (rv (shell-command cmdline)))
    (cond
      ((= rv 0)  (message "Query succeeded"))
      ((= rv 2)  (message "No matches found"))
      (t (message "Error running query")))
  (= rv 0)))

(defun mu-wl-search-and-goto ()
  "search and jump to the folder with the results"
  (interactive)
  (when (mu-wl-search)
    (wl-summary-goto-folder-subr
      (concat "." mu-wl-search-folder)
      'force-update nil nil t)
    (wl-summary-sort-by-date)))

;; search by pressing 'Q'
(define-key wl-summary-mode-map (kbd "Q") ;; => query
  '(lambda()(interactive)(mu-wl-search-and-goto))) 
(define-key wl-folder-mode-map (kbd "Q") ;; => query
  '(lambda()(interactive)(mu-wl-search-and-goto)))

After installing mu and putting the above in your wanderlust startup file, you should be able to search by pressing Q. The mu documentation has an example for mutt as well.

conclusion

It's straightforward to integrate advanced searching capabilities to Wanderlust using mu, and thanks to Sam B., it's gotten a lot easier! The second (old) approach may be useful as 'inspiration' for use in other e-mail clients as well, if they do not provide the kind of hooks that the first solution needs.

IELM: a REPL for emacs

2011-03-26T14:18:00.000+02:00

Emacs-lisp (elisp) is a nice language to play around with code and try things as you develop them – explorative programming. I often use the *scratch* buffer for that, but sometimes it's nice to use a so-called 'REPL' ( Read-Eval-Print-Loop) instead. A REPL is a sort-of command-line interface where your expressions are evaluated as soon as they are considered 'complete' and you press Enter.

So, enter Emacs's built-in repl: IELM. You can activate it with M-x ielm, and the interaction looks something like the following:

*** Welcome to IELM ***  Type (describe-mode) for help.
ELISP> 123 
123
ELISP> (+ 1 2)
3
ELISP> ;; comment
ELISP> (defun fac (n)
         (if (= 0 n)
           1
           (* n (fac (- n 1)))))
fac
ELISP> (fac 5)
120
ELISP>

By default, IELM evaluates complete expressions automatically as soon you as you press Enter. So one thing to remember is that if you want to have multi-line expression (like above), you must make sure that after each line the expression is not complete (i.e., the brackets are not balanced) -- otherwise the expression will be evaluated too early. That makes modes like autopair or paredit a bit inconvenient for this.

If you don't like that behavior, you can do:

(setq ielm-dynamic-return nil)

which will allow you to Enter as much as you want and only evaluate things when you press C-j. But then you might as well use *scratch* I suppose. Personally, I use IELM mostly as a calculator.

executable source code blocks with org-babel

2011-02-27T14:32:00.003+02:00

org-babel is the mechanism that org-mode offers for evaluating (executing) blocks of source code embedded in your org-mode-documents. This is useful for so-called reproducible research, i.e., where you allow your readers to go through the steps that led to your results.

Here, I'm just scratching the surface with some simple examples.

source code blocks

Let's first look at how it all began: source code blocks. I guess most org-mode-users will be familiar with those: the ability to include syntax-highlighted ('font-locked') snippets of source code in org-documents. We discussed source blocks before, they look like this:

#+begin_src perl
sub my_func {
        print "Hello, world!\n";
}
#+end_src

And note that code blocks can be edited in their 'native mode' using C-c'= (or, =org-edit-src-code).

When code block like this are exported to, say, HTML, they'll look like the following fully highlighted snippet (assuming you're reading Emacs-fu in it's full-color version):

sub my_func {
        print "Hello, world!\n";
}

evaluating source code

org-babel takes this a few steps further: instead of just looking at source code, we can actually evaluate (execute) it, using the org-mode-based system called org-babel.

If you have a recent version of org-mode (7.x or later), add the following to your .emacs (or equivalent):

(org-babel-do-load-languages
 'org-babel-load-languages
  '( (perl . t)         
     (ruby . t)
     (sh . t)
     (python . t)
     (emacs-lisp . t)   
   ))

This enables org-babel for the mentioned languages; there are many other languages available as well.

Now, suppose we have a snippet of python in an org-mode-buffer:

#+begin_src python
def hello(str):
        return "Hello, " + str + "!"
return hello ("dude")
#+end_src

You can move the cursor (point) inside the src-block and press C-c C-c (or, org-confirm-babel-evaluate). This causes the block of code to be evaluated (executed), after asking you for confirmation. The result will inserted below the block, like:

#+results:
: Hello, dude!

Note, in the hello example, the result of the block is the value of the evaluation - that is, the value of the last expression evaluated. This is the also the default, so we don't need to (but could) write:

#+begin_src python :results value

The alternative is to use the (standard) output of the function, which is activated with :results output, e.g.:

#+begin_src sh :results output
   echo "Hello $USER! Today is `date`"
#+end_src

Moving to this block and pressing C-c C=c would get you something like the following – probably with a different username and time:

˜#+results:
: Hello djcb! Today is Sun Feb 27 13:51:50 EET 2011

almost like functions

org-babel also allows you to refer to the code blocks from elsewhere in your document, by labeling your code-blocks with srcname. Let's say we have some Ruby code to revert a string:

#+begin_src ruby
def revert(s)
  if s == "" then 
    return ""
  else
    return s[-1].chr + revert(s.slice(0, s.length()-1))
  end
end
revert(str)
#+end_src

We can now 'call' this block; note that we get the result of evaluating the block. So if you want to use the result of a function in the block, you also need to add the call to that function (see the last line).

Now, we can use:

#+call: revert(str="VeryCoolStuff")

And we get:

: ffutSlooCyreV

Note, due to some limitation/bug in my version of org-babel, the strings should not contain spaces or other special characters, so the following will give result in an error note:

˜#+call: revert(str="Very Cool Stuff")

Whenever you try to evaluate a code block, emacs will ask for confirmation -- this is important, because of the obvious security implications of executing unknown code. Anyway, if you do trust the code, you can use the following to skip the confirmation:

(setq org-confirm-babel-evaluate nil)

These simple examples do not really capture the power that org-babel brings, but it's a start. There is quite a bit of documentation for org-babel to help you further. Finally, if you are already using org-babel, feel free to share your experiences in the comments!

keeping your secrets secret

2011-02-10T22:42:00.002+02:00

If you want to keep your secrets secret, it is a good idea to encrypt your data; I usually do that for files with passwords for various services, banking data, and so on. Since version 23, Emacs includes a package called EasyPG (an interface to GnuPG) which makes this seamless – just make sure that you have GnuPG installed.

It's easy to use EasyPG – the only thing you need to do is adding the .gpg -extension to your files, and EasyPG will automatically encrypt/decrypt them when writing/reading. So, for example, to create an encrypted org-mode-file, simply visit (C-x C-f) a file with a name like myfile.org.gpg; emacs opens this in Org-Mode (just like any .org-file). When you want to save the file, emacs will ask you for a password, and with this same password, you can open it again. Don't forget that password!

account data

You can store any kind of secret data in your encrypted files. One type I find particularly useful is to store account data (user names, passwords) for various services there. For example, I use the emacs identi.ca-mode client, which gets its account data through variables identica-username and identica-password.

I do not want to put this information in my main .emacs file for safety reasons, but instead, put it an encrypted file, together with the account data for other services (mail, twitter etc.). Emacs' require does not understand encrypted files, but load-library does. To deal with that, I have two files, secrets.el and secrets.el.gpg (in my load-path):

;; secrets.el
(load-library "secrets.el.gpg")
(provide 'secrets)

and

;; secrets.el.gpg
(setq identica-username "djcb"
      identica-password "$ekr3t")
;; ... other passwords ...

Now, in my .emacs I have a function for various services, like:

(defun start-identica ()
  (interactive)
  (require 'secrets)
  (identica-friends-timeline))

This will prompt me for the password, but only if I use anything that requires the secret data, and only once per session.

Update: as Richard notes in the comments, you can also use require by explicitly specifying the filename (parameter two). That might actually be easier -- thanks Richard!

using public keys

By default, EasyPG performs symmetric encryption; if you want to use public key encryption instead (useful when you want to share the encrypted files with others), you can use:

;; 'silent to use symmetric encryption
;; nil to ask for users unless specified
;; t to always ask for a user
(setq epa-file-select-keys t)

The 'users' in this snippet are the people in your GnuPG-keyring – EasyPG lists your keyring inhabitants, allowing for easy selection. You can also specify the people who can decrypt your file by putting something like the following at the beginning of the file you want to encrypt.

# -*- epa-file-encrypt-to: ("foo@bar.org") -*-

so

EasyPG brings more functionality for encryption, decryption, signing, managing your keyring and so on, but I haven't used that much yet. Anyhow, the automatic support for reading/writing file is really nice.

extending ERC with your own commands

2011-02-02T20:40:00.000+02:00

ERC is the leading Emacs-based IRC-client; I already discussed ERC before. I have been using ERC a lot in recent times, as it's an essential way to communicate at work with team members in remote locations. There are other IRC-clients – most people around me seem to use either irssi or xchat, but these don't integrate so well with my emacs-based workflow, the easy with which it can be extended to do exactly what I want; in this although they have their own strenghts. One of the great strengths of ERC is article I give some examples.

Apart from chatting, you can send commands (long list) to the IRC-server, for example to request information about other users, change your 'nick', leave the channel, and so on. As in most IRC-clients, you can send these commands with ERC by prefixing them with /, so you'd type:

/nick ninjaturtle

to change your nickname.

The nice thing about ERC is how easy it to add your own commands to this. In your .emacs (after loading ERC), you can add something like:

(defun erc-cmd-MYSYSTEM ()
  "show some information about my system"
  (let ((str (shell-command-to-string "uname -a")))
    (when str (erc-send-message str))))

Or, add add a function called erc-cmd-XXXX (with the XXXX being the command name in capitals, will add command XXXX, which you can invoke with /XXXX or /xxxx). So, with the above function, I can now do something like:

ERC> /mysystem
<djcb> Linux cthulhu 2.6.35-25-generic #44 SMP Fri Jan 21 17:40:48 UTC
       2011 i686 GNU/Linux

Let's look at some other (somewhat) useful command: /calc; again, just a small example, I'm sure something can come up with something a bit more elegant - perhaps using emacs' built-in calc.

(defun erc-cmd-CALC (&rest args)
  "calculate value of some expression using bc"
  (let ((expr (mapconcat 'identity args " ")))
    (when (length expr)
      (let ((result (shell-command-to-string (concat "echo '" expr "' | bc "))))
        (when result (erc-send-message (concat expr " = " result)))))))

ERC> /calc 2 * (3 + 4) / 7
<djcb> 2 * (3 + 4) / 7  = 2

Now, a small warning, just because it's easy to dump the output of, say, cowsay in an IRC-channel using your own /cowsay command, does not mean it is a good idea – in fact, using something like that is guaranteed to get you kicked out fairly quickly from many channels.

That being said, I'm sure many people have come up with much more clever things than the examples here; feel free to share your inventions in the comments!

setting the frame title

2011-01-06T17:47:00.002+02:00

The 'frame title' (window title) that emacs uses in graphical environments defaults to something like emacs@hostname.

Of course emacs lets us customize this, by changing the value of frame-title-format. Emacs accepts many different things there, (see the documentation for frame-title-format and mode-line-format for that), but let's look at an example.

Instead of the default emacs@hostname, I find it more useful to include the name of the file I'm working on instead, or, in case of non-file buffers, the buffer name. To do this, I have something like the following in my .emacs:

(setq frame-title-format
  '("" invocation-name ": "(:eval (if (buffer-file-name)
                (abbreviate-file-name (buffer-file-name))
                  "%b"))))

As you see, frame-title-format is a template for the items that are present in the title bar; i.e.. emacs concatenates the items in the list, and it supports various %-constructs, which are replaced with actual values; see below.

In addition to the %-constructs, you can use :eval to make emacs evaluate the expression whenever it wants to update the title bar.

invocation-name is the name of the emacs binary.

abbreviate-file-name replaces the home directory part in file names with ~; for very deep paths it might be nice to do some abbreviation as well as some shells do; this is left as an exercise to the reader :)

You can experiment with some other things to put in frame-title-format; use the :eval construct as above to use emacs-lisp functions, and the various %-specifiers which are replaced by certain values; the emacs documentation lists the following:

%b -- print buffer name.      %f -- print visited file name.
%F -- print frame name.
%* -- print %, * or hyphen.   %+ -- print *, % or hyphen.
      %& is like %*, but ignore read-only-ness.
      % means buffer is read-only and * means it is modified.
      For a modified read-only buffer, %* gives % and %+ gives *.
%s -- print process status.
%i -- print the size of the buffer.
%I -- like %i, but use k, M, G, etc., to abbreviate.
%p -- print percent of buffer above top of window, or Top, Bot or All.
%P -- print percent of buffer above bottom of window, perhaps plus Top,
      or print Bottom or All.
%n -- print Narrow if appropriate.
%t -- visited file is text or binary (if OS supports this distinction).
%z -- print mnemonics of keyboard, terminal, and buffer coding systems.
%Z -- like %z, but including the end-of-line format.
%e -- print error message about full memory.
%@ -- print @ or hyphen.  @ means that default-directory is on a
      remote machine.
%[ -- print one [ for each recursive editing level.  %] similar.
%% -- print %.   %- -- print infinitely many dashes.
Decimal digits after the % specify field width to which to pad.

So, if we'd like to include the host (system) name and some indication of the status of this buffer, we could do something like:

(setq frame-title-format
  '("emacs%@" (:eval (system-name)) ": " (:eval (if (buffer-file-name)
                (abbreviate-file-name (buffer-file-name))
                  "%b")) " [%*]"))

Of course, some of the information is available elsewhere already, but it might be clearer in the frame-title. Or not – there's a lot of room for tweaking and experimentation here.