quick note-taking with deft and org-mode

Emacs must be gathering a lot of enthusiasts lately; there's hardly a week where I don't discover some new gem. Recently, I discovered deft. And apparently, I wasn't the only one.

So what is it deft good for? Well, often I want to jot down some quick thing during a meeting or a telephone-call. Of course, I don't want to think about file names or anything else distracting me from my task, just get me that note already! In addition, at some later time, I want to be able to quickly search through the notes I made.

For MacOS, there's a program called Notational Velocity which does this. But really - it sounds like a typical task for emacs - wouldn't it be nice to have an emacs package that does roughly the same?

And that is what deft does - enable you to quickly write notes, and retrieving them later. The author has an excellent introduction on his website, so my job is very easy :) deft is not part of org-mode, but they can work together seamlessly. Here's my set-up:

;; http://jblevins.org/projects/deft/
(when (require 'deft nil 'noerror) 
   (setq
      deft-extension "org"
      deft-directory "~/Org/deft/"
      deft-text-mode 'org-mode)
   (global-set-key (kbd "<f9>") 'deft))

This blob goes in my .emacs. Note, the first line ensures that emacs starts without errors, even when I run on a system without deft. Apart from that, I make deft use org files for note taking, which makes it all very familiar.

All notes are saved ~/Org/deft - you can set it to something else of course. A Dropbox-folder seems to be a popular choice for synchronizing between machines.

Finally, the last line binds F9 to deft-mode. So, when I need a quick note, I can type F9 C-c C-n and start writing.

finding just about anything

Buffer-switching is one of those things I do all the time in Emacs. And it should be quick and efficient, or it will break my 'flow'. There are many ways to customize the buffer switching experience, and we already discussed quite a few of those here: using Ctrl-Tab, iswitchb/ido, ibuffer, the lusty explorer and others.

Some of those packages - ido and lusty-explorer - do not solely focus on buffer-switching - they also let you open files using the same user interface. But why stop at files and buffers - wouldn't it be nice if we could quickly find just about anything quickly?

Indeed that would be nice - and there's a way to do just that - using the aptly named anything package. I was always a bit put off by this package due to the screenshots (see link), but once I got over that, I've become a very happy user.

Anyhow, what can anything find? I mentioned buffers and files, but it can also find bookmarks, recently used files, any file on your system (using locate), man-pages, info-pages, emacs-function/variables, FIXME-comments, org-headlines, bbdb-contacts, google-suggests… and a million other things. It can probably find your car keys, too.

Installation is pretty straightforward, using the git-repository:

git clone git://repo.or.cz/anything-config.git

Then, go into the just-cloned directory and execute make. After that, add to your .emacs (or ~/.emacs.d/init.el):

;; path-to-anything is the path which has the 'anything' we just cloned
(add-to-list 'load-path "path-to-anything")
(require 'anything-config)

This will add a menu with various anything-commands, and a bunch of key-bindings, starting with F5. Play around a bit with it, it's nice. The results are shown in a little temporary buffer, and pressing Tab will let you do (search-type dependent) actions on the matches - for example ediff two buffers.

Of course, the real fun starts when we super-charge some of the normal emacs functions with anything-based ones. Let's look at buffer switching. Let's create an anything-based version, and assign it to C-x b, the good-old switch-to-buffer.

The thing to consider is that while anything can find just about anything, you (well, I) usually only want to search for a certain set of things; when I want to switch to another buffer, I don't want to match man-pages. Luckily, anything allows for making nifty function which use certain subsets of the search sources. So for buffer switching, I have:

(global-set-key (kbd "C-x b")
  (lambda() (interactive)
    (anything
     :prompt "Switch to: "
     :candidate-number-limit 10                 ;; up to 10 of each 
     :sources
     '( anything-c-source-buffers               ;; buffers 
        anything-c-source-recentf               ;; recent files 
        anything-c-source-bookmarks             ;; bookmarks
        anything-c-source-files-in-current-dir+ ;; current dir
        anything-c-source-locate))))            ;; use 'locate'

This will search in buffers, then in my recent-files, then in my bookmarks, the files in the current directory, and finally check with the locate tool. That last one is pretty neat, and finally gives me something back for the countless times I've wondered why the hard disk is grinding – indeed, it was locate, updating its database.

Then, I've defined another binding for searching general documentation on my system; I've put it under C-c I. This looks look something like the following:

(global-set-key (kbd "C-c I")  ;; i -> info
  (lambda () (interactive)
    (anything
      :prompt "Info about: "
      :candidate-number-limit 3
      :sources
      '( anything-c-source-info-libc             ;; glibc docs
         anything-c-source-man-pages             ;; man pages
         anything-c-source-info-emacs))))        ;; emacs 

These are sources I query quite regularly; there are many more to be found - for most packages with info-pages there's a corresponding anything-c-source-info-...; there's a list in anything-config.el.

Now, those are my general documentation sources; in specific modes, I have specialized information sources; for example, for elisp-mode:

(add-hook 'emacs-lisp-mode-hook
  (lambda()
  ;; other stuff...
  ;; ...
  ;; put useful info under C-c i
    (local-set-key (kbd "C-c i")
      (lambda() (interactive)
        (anything
          :prompt "Info about: "
          :candidate-number-limit 5
          :sources
          '( anything-c-source-emacs-functions
             anything-c-source-emacs-variables
             anything-c-source-info-elisp
             anything-c-source-emacs-commands
             anything-c-source-emacs-source-defun
             anything-c-source-emacs-lisp-expectations
             anything-c-source-emacs-lisp-toplevels
             anything-c-source-emacs-functions-with-abbrevs
             anything-c-source-info-emacs))))

This post just scratches the surface of what is possible – so go and experiment :) One interesting thing is to add your own sources; I played a bit with that already,

There are many things in anything I did not cover yet. First, there are many more sources to search - and it's pretty easy to write your own – see the EmacsWiki-page.

customizing the mode-line

The mode-line is the emacs 'status bar', the bar just above the minibuffer that shows various pieces of information, such as the buffer name, the major mode, maybe the current line number, some indicators for active minor modes, and so on. As I'm looking at it, it starts with 1<U:**- (which is: input-method: latin-1-alt-postfix, buffer-coding-system: utf8-unix, line-ending: unix-style, buffer is writable and buffer is modified – the tooltips help).
As with just about anything in emacs, the mode-line can be customized just the way you like. I give some example below, not because I think it is necessarily the best way, but just to give you a bit of an example to start with when making your own best-mode-line-ever.
I'm not going through all the details of the example, but let me highlight a few things that make it a bit easier to understand.
First of all, the mode-line can be customized by setting the variable mode-line-format; this variable becomes buffer-local automatically when changed, so if you want to set it for all buffers, you'll need to use setq-default in your .emacs (or equivalent). The format is quite similar to the one for frame-title-format, which we discussed in setting the frame title a while back.
mode-line-format is a list of items which are evaluated, and put together as a string which then ends up as the mode-line contents. These properties can be any string. The following types of items can be used:

• First, normal strings are just shown as-is;
• Then, there are some special format parameters which will be replaced with their value in the mode-line, from the Emacs-documentation:

  %b -- print buffer name.      %f -- print visited file name.
  %F -- print frame name.
  %* -- print %, * or hyphen.   %+ -- print *, % or hyphen.
        %& is like %*, but ignore read-only-ness.
        % means buffer is read-only and * means it is modified.
        For a modified read-only buffer, %* gives % and %+ gives *.
  %s -- print process status.   %l -- print the current line number.
  %c -- print the current column number (this makes editing slower).
        To make the column number update correctly in all cases,
        `column-number-mode' must be non-nil.
  %i -- print the size of the buffer.
  %I -- like %i, but use k, M, G, etc., to abbreviate.
  %p -- print percent of buffer above top of window, or Top, Bot or All.
  %P -- print percent of buffer above bottom of window, perhaps plus Top,
        or print Bottom or All.
  %n -- print Narrow if appropriate.
  %t -- visited file is text or binary (if OS supports this distinction).
  %z -- print mnemonics of keyboard, terminal, and buffer coding systems.
  %Z -- like %z, but including the end-of-line format.
  %e -- print error message about full memory.
  %@ -- print @ or hyphen.  @ means that default-directory is on a
        remote machine.
  %[ -- print one [ for each recursive editing level.  %] similar.
  %% -- print %.   %- -- print infinitely many dashes.
Decimal digits after the % specify field width to which to pad.

• Forms of the type (:eval ...) are evaluated each time the mode-line is drawn (just like the '%'-parameters) ; so, if you have a value that changes of the course your emacs session, you should use (:eval ...). For example, for your emacs-uptime you could use (:eval (emacs-uptime "%hh")); while the emacs-PID does not change, so simply you could simply use (format "PID:%d").
  The format parameter mentioned above are of evaluated each time as well. Note that you have to be a bit careful with evaluations - don't do too heavy operations there, and be careful the updates don't recurse.
  
• There are many others which I won't go into now - please check the Elisp reference. It's a rather baroque format…

Now, let's put this all together in an example (tested with emacs 23 and 24). As I said, this is for demonstration purposes only; but hopefully it gives you some inspiration. A lot of the 'magic' (colors, tooltips, faces) happens with the propertize function; again, the Elisp documentation can tell you a lot more about that. I'm (ab)using the various font-lock-faces to have colors that blend in nicely with your current theme.
And it has a limitation still, namely that it does not react to mouse clicks; how to that, I will discuss in some future article.

;; use setq-default to set it for /all/ modes
(setq mode-line-format
  (list
    ;; the buffer name; the file name as a tool tip
    '(:eval (propertize "%b " 'face 'font-lock-keyword-face
        'help-echo (buffer-file-name)))

    ;; line and column
    "(" ;; '%02' to set to 2 chars at least; prevents flickering
      (propertize "%02l" 'face 'font-lock-type-face) ","
      (propertize "%02c" 'face 'font-lock-type-face) 
    ") "

    ;; relative position, size of file
    "["
    (propertize "%p" 'face 'font-lock-constant-face) ;; % above top
    "/"
    (propertize "%I" 'face 'font-lock-constant-face) ;; size
    "] "

    ;; the current major mode for the buffer.
    "["

    '(:eval (propertize "%m" 'face 'font-lock-string-face
              'help-echo buffer-file-coding-system))
    "] "


    "[" ;; insert vs overwrite mode, input-method in a tooltip
    '(:eval (propertize (if overwrite-mode "Ovr" "Ins")
              'face 'font-lock-preprocessor-face
              'help-echo (concat "Buffer is in "
                           (if overwrite-mode "overwrite" "insert") " mode")))

    ;; was this buffer modified since the last save?
    '(:eval (when (buffer-modified-p)
              (concat ","  (propertize "Mod"
                             'face 'font-lock-warning-face
                             'help-echo "Buffer has been modified"))))

    ;; is this buffer read-only?
    '(:eval (when buffer-read-only
              (concat ","  (propertize "RO"
                             'face 'font-lock-type-face
                             'help-echo "Buffer is read-only"))))  
    "] "

    ;; add the time, with the date and the emacs uptime in the tooltip
    '(:eval (propertize (format-time-string "%H:%M")
              'help-echo
              (concat (format-time-string "%c; ")
                      (emacs-uptime "Uptime:%hh"))))
    " --"
    ;; i don't want to see minor-modes; but if you want, uncomment this:
    ;; minor-mode-alist  ;; list of minor modes
    "%-" ;; fill with '-'
    ))

Have fun playing with this!

toward balanced and colorful delimiters

Emacs has the very useful feature of blinking the corresponding left "(" when you type its partner ")" (and the same for other delimiters like "[]", "{}" etc., depending on the mode). This is very useful for programming, especially for languages like Lisp and Scheme and the like.


Further help can come from tools like autopair or paredit – although the latter is a bit too much bondage & discipline for me, many people love it.
Anyway, recently I discovered a new helper in the quest for balance in delimited universe: rainbow-delimiters. With this package, the delimiters all get different colors based on their nesting level. It works wonderfully well.
Installation is straightforward:

• download rainbow-delimiters and put it in your load-path
• add something like the following in your .emacs:

(when (require 'rainbow-delimiters nil 'noerror) 
  (add-hook 'scheme-mode-hook 'rainbow-delimiters-mode))

That's it – of course you'll need to do the equivalent for modes where you'd like to enable it. See the screenshot below – maybe a bit too colorful for some people, but I like it and I have found it actually useful to see the corresponding delimiters without having to move the cursor over them.

I already added some nice Zenburn colors for this to the Zenburn theme for Emacs-24, as can be seen in the screenshot.

porting the zenburn theme to emacs 24

Just a small note: I already mentioned the zenburn color theme a couple of times. I have now ported the full (big) set of customizations to the emacs-24 format – find it at github; so if you run emacs 24 (the development version) and enjoy zenburn like me, you can give it try. Suggestions, additions are welcome.

Note, there seems to be a limitation with the :inherit-attribute for themes – in practice it means that some aspects of the theme may not work if a package sets the face before the theme does. Therefore, best to load the theme early from your .emacs or init.el; or alternatively visit the theme file (zenburn-theme.el in this case) and evaluate the buffer.

nice-looking pdfs with org-mode and xetex

I've discussed the wonderful org-mode here a number of times already. It has become a pretty important part of my overall workflow. One thing I am using org-mode for, is to produce all kinds of PDF-documents that I can share with other people.

org-mode & LaTeX

In the past, I often used straight LaTeX for such things; I wrote my thesis with it, but also many other documents. There are many things I like about LaTeX, one of them being that I can use emacs for writing. Still, there are also a few things I do not particularly like. First, I think LaTeX is quite heavy with formatting directives, which hinder my writing flow (e.g., when I want to include an image, a table or a source code snippet). Another thing is that I find the default LaTeX styles a bit boring. Nothing wrong with it, but there just too many documents with the exact same lay-out.

Now, back to org-mode. One way to use org-mode is as a friendly way to generate LaTeX (and, consequently, PDFs). This is a big improvement! Much more than LaTeX itself, org-mode allows to focus on the contents of the document, rather than instructing LaTeX what to do. This comes at the price of small bit of flexibility, but, if needed org-mode allows you include straight LaTeX when needed – so while keeping easy things easy, hard things are still possible. The latter does require a bit of experience with LaTeX a though.

setting up XeTeX

Now, for the second issue, the way documents look, there are other solutions, and they live on the LaTeX side of things. I'm sure many have seen The Beauty of LaTeX. Using the XeTeX implementation of LaTeX and the fontspec package, you can create LaTeX documents with a bit 'refreshed' look.

So, the steps to get this working with org-mode:

• install the texlive-xetex packages on Ubuntu and Debian (this installs a huge set of packages)
• install the SIL fonts (I'm using ttf-sil-gentium and ttf-sil-charis, but there are more)
• I'm also using DejaVu Mono (ttf-dejavu)

teaching org-mode about the new XeTeX stuff

We now need to define some LaTeX document class for org-mode that uses XeTeX and some of these new fonts. Let's call the document class djcb-org-article (as I often use the djcb- prefix for my own stuff), it could be something like the following (add to your org-setup – e.g., in your .emacs, make sure there is a (require 'org) before this:

;; 'djcb-org-article' for export org documents to the LaTex 'article', using
;; XeTeX and some fancy fonts; requires XeTeX (see org-latex-to-pdf-process)
(add-to-list 'org-export-latex-classes
  '("djcb-org-article"
"\\documentclass[11pt,a4paper]{article}
\\usepackage[T1]{fontenc}
\\usepackage{fontspec}
\\usepackage{graphicx} 
\\defaultfontfeatures{Mapping=tex-text}
\\setromanfont{Gentium}
\\setromanfont [BoldFont={Gentium Basic Bold},
                ItalicFont={Gentium Basic Italic}]{Gentium Basic}
\\setsansfont{Charis SIL}
\\setmonofont[Scale=0.8]{DejaVu Sans Mono}
\\usepackage{geometry}
\\geometry{a4paper, textwidth=6.5in, textheight=10in,
            marginparsep=7pt, marginparwidth=.6in}
\\pagestyle{empty}
\\title{}
      [NO-DEFAULT-PACKAGES]
      [NO-PACKAGES]"
     ("\\section{%s}" . "\\section*{%s}")
     ("\\subsection{%s}" . "\\subsection*{%s}")
     ("\\subsubsection{%s}" . "\\subsubsection*{%s}")
     ("\\paragraph{%s}" . "\\paragraph*{%s}")
     ("\\subparagraph{%s}" . "\\subparagraph*{%s}")))

Of course, this can be customized to your own preference; e.g., North-Americans may not be using A4-paper.

org-mode takes care of the export from its own format to LaTeX, but we need to tell it to use xelatex to process the LaTeX to PDF:

(setq org-latex-to-pdf-process 
  '("xelatex -interaction nonstopmode %f"
     "xelatex -interaction nonstopmode %f")) ;; for multiple passes

That's all that's needed on the setup-side.

creating a document

Now, let's create a little test document, test.org, to show how it works:

#+LaTeX_CLASS: djcb-org-article
#+TITLE: My little document

* Introduction
  
  This is my document. There are many like it, but this is mine. It's easy to
  write without *too* _many_ /distractions/.
  
** Normal distribution

   Probability density of the normal distribution, using familiar TeX notation
   for formulae:
 
   $$\frac{1}{\sqrt{2\pi\sigma^2}}e^{ -\frac{(x-\mu)^2}{2\sigma^2} }$$

** Some table

| *Greek God* | *Roman God* | *Element*      |
|-------------+-------------+----------------|
| Zeus        | Jupiter     | Sky and clouds |
| Hera        | Juno        | Family         |
| Poseidon    | Neptune     | Sea            |
| Hades       | Pluto       | Underworld     |

We can export this to a PDF using C-c C-e p (or C-c C-e d to automatically open the PDF in a PDF-viewer). This should all work nicely; if it doesn't, note that when exporting, say, test.org, org-mode will create a file called test.tex, and visit in a buffer. There's also a buffer with the output from various commands, but sometimes it can be useful to run LaTeX (xelatex in this case) on the file by hand, to find any problems. The wonderful org-documentation about exporting to LaTeX has more information.

I think the result is pretty nice – it stays true to the class LaTeX article class, but freshens it up a bit with some news font. If you can make something better – which is not unlikely – you are of course invited to contribute your own!

Concluding

org-mode is a pretty convenient way to write nice-looking PDFs. Combined with xelatex, they don't have to look too plain :). However, I'm aware of my limitations when it comes to the coolness/aesthetic aspects, but I hope others can show the way here.

Maybe org-mode could ship with a number of ready-made templates to make it easy to make nice-looking documents, resumes, reference cards, reports, meeting notes and so on.

searching e-mails with wanderlust and mu

I have discussed the Wanderlust e-mail client a couple of times already. I'm still using it, so I keep on learning new tricks. Even though there has been quite a bit of action in the competing gnus e-mail client, for my particular use-case, Wanderlust is still the best option.

'My particular use-case' consists of storing my mail in Maildirs, which I fill with either offlineimap (which has fortunately found a new maintainer) or fetchmail.

mu

When dealing with e-mail, one particularly important feature for me is the ability to search my messages. In fact, it's so important for me that I wrote some software to do this for me; the software is called mu; it indexes the messages in my Maildirs, and then allows for searching them using queries, based on message contents, headers, or other message properties.

mu works through a command-line interface, although there is an experimental GUI available as well. The command-line interface makes it possible to hook mu up with various mail-clients, such as mutt, or Wanderlust. Some Linux distributions ship mu, but since the versions they ship are often a bit outdated, I recommend building it yourself from the sources linked on the mu website. The process is fairly straightforward; and there is plenty of documentation in the form of man pages.

mu and wanderlust

I've been combining mu and wanderlust for a while (see mu and wanderlust - the old way, below), but this week Sam B. on the mu mailing list showed a way to do so in a much more elegant way - using virtual or query folders.

How does this work? Well, after installing mu, add the following to your Wanderlust setup file (~/.wl or it's moral equivalent – see the older Wanderlust posts for the details):

(require 'elmo-search)
(elmo-search-register-engine
    'mu 'local-file
    :prog "/usr/local/bin/mu" ;; or wherever you've installed it
    :args '("find" pattern "--fields" "l") :charset 'utf-8)

(setq elmo-search-default-engine 'mu)
;; for when you type "g" in folder or summary.
(setq wl-default-spec "[")

So, to start with the last part, whenever you type g in folder or summary, in the mode-line you will get something like Folder name (.inbox): [. Now simply type your mu search expression and press Enter, and wanderlust opens a (temporary) folder with the search results. Brilliant!

Next, to add virtual folders for searches you do often, simply add some folder specifications like the following to your .folders file (again, check the older Wanderlust posts if you're not familiar with folders-file):

VFolders {
# message I received today
  [date:today..now]!mu  "Today"

# messages bigger than 1Mb  
  [size:1m..100m]!mu    "Big"

# signed messages i got in 2010 related to emacs
  [date:2010..2011 flag:signed emacs]!mu "Signed-Emacs2010"

# unread messages
  [not flag:seen]!mu    "Unread"
# or (for mu  >= 0.9.4):
# [flag:unread]! mu      "Unread"
}

After this, restart Wanderlust, and there you go! Wanderlust will display your brand new virtual folders with an icon that looks like a little whale.

You can put arbitrary mu search expressions between the [], matching whatever is useful in a certain case. Check the mu documentation to see how to do this.

Note, the messages you get in these virtual folders are links to the original messages. In practice, this means that changes you make to the links do no affect the originals – if you delete a link you're not deleting the message.

mu and wanderlust - the old way

This discussion would not complete without a description of the old way I used search. This method may still be useful for integrating mu with other clients such as mutt.

What I've been using for a while is a (in retrospect) rather clumsy way to integrate message searches with Wanderlust: based on the results of a query, I would create some special Maildir and fill it with symbolic links to the matched messages, and the visit this special Maildir with Wanderlust. I'll include the code here to contrast it with the more elegant solution that we saw before, but also because the approach taken might be easily adapted for other mail-clients.

;; search using mutt
(defvar mu-wl-mu-program     "/usr/local/bin/mu")
(defvar mu-wl-search-folder  "search")

(defun mu-wl-search ()
  "search for messages with `mu', and jump to the results"
   (let* ((muexpr (read-string "Find messages matching: "))
          (sfldr  (concat elmo-maildir-folder-path "/"
                    mu-wl-search-folder))
          (cmdline (concat mu-wl-mu-program " find "
                      "--clearlinks --format=links --linksdir='" sfldr "' "
                     muexpr))    
          (rv (shell-command cmdline)))
    (cond
      ((= rv 0)  (message "Query succeeded"))
      ((= rv 2)  (message "No matches found"))
      (t (message "Error running query")))
  (= rv 0)))

(defun mu-wl-search-and-goto ()
  "search and jump to the folder with the results"
  (interactive)
  (when (mu-wl-search)
    (wl-summary-goto-folder-subr
      (concat "." mu-wl-search-folder)
      'force-update nil nil t)
    (wl-summary-sort-by-date)))

;; search by pressing 'Q'
(define-key wl-summary-mode-map (kbd "Q") ;; => query
  '(lambda()(interactive)(mu-wl-search-and-goto))) 
(define-key wl-folder-mode-map (kbd "Q") ;; => query
  '(lambda()(interactive)(mu-wl-search-and-goto))) 

After installing mu and putting the above in your wanderlust startup file, you should be able to search by pressing Q. The mu documentation has an example for mutt as well.

conclusion

It's straightforward to integrate advanced searching capabilities to Wanderlust using mu, and thanks to Sam B., it's gotten a lot easier! The second (old) approach may be useful as 'inspiration' for use in other e-mail clients as well, if they do not provide the kind of hooks that the first solution needs.