executable source code blocks with org-babel

org-babel is the mechanism that org-mode offers for evaluating (executing) blocks of source code embedded in your org-mode-documents. This is useful for so-called reproducible research, i.e., where you allow your readers to go through the steps that led to your results.

Here, I'm just scratching the surface with some simple examples.

source code blocks

Let's first look at how it all began: source code blocks. I guess most org-mode-users will be familiar with those: the ability to include syntax-highlighted ('font-locked') snippets of source code in org-documents. We discussed source blocks before, they look like this:

#+begin_src perl
sub my_func {
        print "Hello, world!\n";
}
#+end_src

And note that code blocks can be edited in their 'native mode' using C-c'= (or, =org-edit-src-code).

When code block like this are exported to, say, HTML, they'll look like the following fully highlighted snippet (assuming you're reading Emacs-fu in it's full-color version):

sub my_func {
        print "Hello, world!\n";
}

evaluating source code

org-babel takes this a few steps further: instead of just looking at source code, we can actually evaluate (execute) it, using the org-mode-based system called org-babel.

If you have a recent version of org-mode (7.x or later), add the following to your .emacs (or equivalent):

(org-babel-do-load-languages
 'org-babel-load-languages
  '( (perl . t)         
     (ruby . t)
     (sh . t)
     (python . t)
     (emacs-lisp . t)   
   ))

This enables org-babel for the mentioned languages; there are many other languages available as well.

Now, suppose we have a snippet of python in an org-mode-buffer:

#+begin_src python
def hello(str):
        return "Hello, " + str + "!"
return hello ("dude")
#+end_src

You can move the cursor (point) inside the src-block and press C-c C-c (or, org-confirm-babel-evaluate). This causes the block of code to be evaluated (executed), after asking you for confirmation. The result will inserted below the block, like:

#+results:
: Hello, dude!

Note, in the hello example, the result of the block is the value of the evaluation - that is, the value of the last expression evaluated. This is the also the default, so we don't need to (but could) write:

#+begin_src python :results value

The alternative is to use the (standard) output of the function, which is activated with :results output, e.g.:

#+begin_src sh :results output
   echo "Hello $USER! Today is `date`"
#+end_src

Moving to this block and pressing C-c C=c would get you something like the following – probably with a different username and time:

˜#+results:
: Hello djcb! Today is Sun Feb 27 13:51:50 EET 2011

almost like functions

org-babel also allows you to refer to the code blocks from elsewhere in your document, by labeling your code-blocks with srcname. Let's say we have some Ruby code to revert a string:

#+begin_src ruby
def revert(s)
  if s == "" then 
    return ""
  else
    return s[-1].chr + revert(s.slice(0, s.length()-1))
  end
end
revert(str)
#+end_src

We can now 'call' this block; note that we get the result of evaluating the block. So if you want to use the result of a function in the block, you also need to add the call to that function (see the last line).

Now, we can use:

#+call: revert(str="VeryCoolStuff")

And we get:

: ffutSlooCyreV

Note, due to some limitation/bug in my version of org-babel, the strings should not contain spaces or other special characters, so the following will give result in an error note:

˜#+call: revert(str="Very Cool Stuff")

Whenever you try to evaluate a code block, emacs will ask for confirmation -- this is important, because of the obvious security implications of executing unknown code. Anyway, if you do trust the code, you can use the following to skip the confirmation:

(setq org-confirm-babel-evaluate nil)

These simple examples do not really capture the power that org-babel brings, but it's a start. There is quite a bit of documentation for org-babel to help you further. Finally, if you are already using org-babel, feel free to share your experiences in the comments!

keeping your secrets secret

If you want to keep your secrets secret, it is a good idea to encrypt your data; I usually do that for files with passwords for various services, banking data, and so on. Since version 23, Emacs includes a package called EasyPG (an interface to GnuPG) which makes this seamless – just make sure that you have GnuPG installed.

It's easy to use EasyPG – the only thing you need to do is adding the .gpg -extension to your files, and EasyPG will automatically encrypt/decrypt them when writing/reading. So, for example, to create an encrypted org-mode-file, simply visit (C-x C-f) a file with a name like myfile.org.gpg; emacs opens this in Org-Mode (just like any .org-file). When you want to save the file, emacs will ask you for a password, and with this same password, you can open it again. Don't forget that password!

account data

You can store any kind of secret data in your encrypted files. One type I find particularly useful is to store account data (user names, passwords) for various services there. For example, I use the emacs identi.ca-mode client, which gets its account data through variables identica-username and identica-password.

I do not want to put this information in my main .emacs file for safety reasons, but instead, put it an encrypted file, together with the account data for other services (mail, twitter etc.). Emacs' require does not understand encrypted files, but load-library does. To deal with that, I have two files, secrets.el and secrets.el.gpg (in my load-path):

;; secrets.el
(load-library "secrets.el.gpg")
(provide 'secrets)

and

;; secrets.el.gpg
(setq identica-username "djcb"
      identica-password "$ekr3t")
;; ... other passwords ...

Now, in my .emacs I have a function for various services, like:

(defun start-identica ()
  (interactive)
  (require 'secrets)
  (identica-friends-timeline))

This will prompt me for the password, but only if I use anything that requires the secret data, and only once per session.

Update: as Richard notes in the comments, you can also use require by explicitly specifying the filename (parameter two). That might actually be easier -- thanks Richard!

using public keys

By default, EasyPG performs symmetric encryption; if you want to use public key encryption instead (useful when you want to share the encrypted files with others), you can use:

;; 'silent to use symmetric encryption
;; nil to ask for users unless specified
;; t to always ask for a user
(setq epa-file-select-keys t) 

The 'users' in this snippet are the people in your GnuPG-keyring – EasyPG lists your keyring inhabitants, allowing for easy selection. You can also specify the people who can decrypt your file by putting something like the following at the beginning of the file you want to encrypt.

# -*- epa-file-encrypt-to: ("foo@bar.org") -*-

so

EasyPG brings more functionality for encryption, decryption, signing, managing your keyring and so on, but I haven't used that much yet. Anyhow, the automatic support for reading/writing file is really nice.

extending ERC with your own commands

ERC is the leading Emacs-based IRC-client; I already discussed ERC before. I have been using ERC a lot in recent times, as it's an essential way to communicate at work with team members in remote locations. There are other IRC-clients – most people around me seem to use either irssi or xchat, but these don't integrate so well with my emacs-based workflow, the easy with which it can be extended to do exactly what I want; in this although they have their own strenghts. One of the great strengths of ERC is article I give some examples.

Apart from chatting, you can send commands (long list) to the IRC-server, for example to request information about other users, change your 'nick', leave the channel, and so on. As in most IRC-clients, you can send these commands with ERC by prefixing them with /, so you'd type:

/nick ninjaturtle

to change your nickname.

The nice thing about ERC is how easy it to add your own commands to this. In your .emacs (after loading ERC), you can add something like:

(defun erc-cmd-MYSYSTEM ()
  "show some information about my system"
  (let ((str (shell-command-to-string "uname -a")))
    (when str (erc-send-message str))))

Or, add add a function called erc-cmd-XXXX (with the XXXX being the command name in capitals, will add command XXXX, which you can invoke with /XXXX or /xxxx). So, with the above function, I can now do something like:

ERC> /mysystem
<djcb> Linux cthulhu 2.6.35-25-generic #44 SMP Fri Jan 21 17:40:48 UTC
       2011 i686 GNU/Linux

Let's look at some other (somewhat) useful command: /calc; again, just a small example, I'm sure something can come up with something a bit more elegant - perhaps using emacs' built-in calc.

(defun erc-cmd-CALC (&rest args)
  "calculate value of some expression using bc"
  (let ((expr (mapconcat 'identity args " ")))
    (when (length expr)
      (let ((result (shell-command-to-string (concat "echo '" expr "' | bc "))))
        (when result (erc-send-message (concat expr " = " result)))))))

ERC> /calc 2 * (3 + 4) / 7
<djcb> 2 * (3 + 4) / 7  = 2

Now, a small warning, just because it's easy to dump the output of, say, cowsay in an IRC-channel using your own /cowsay command, does not mean it is a good idea – in fact, using something like that is guaranteed to get you kicked out fairly quickly from many channels.

That being said, I'm sure many people have come up with much more clever things than the examples here; feel free to share your inventions in the comments!