2009-08-03

interview with Chong Yidong and Stefan Monnier

Earlier this year, Chong Yidong and Stefan Monnier took over Emacs maintainership from Richard Stallman, and they successfully completed the Emacs 23.1 release. I asked them a couple of questions about the process, Emacs-development and some of the plans for the future.

I'd like to thank Chong (CYD) and Stefan (SM) to take the time to answer my (djcb) questions and even more so for doing an excellent job bringing us Emacs 23!


djcb: First of all, could you tell us a bit about yourself? For example, what you do when not hacking on Emacs?

CYD: I'm a postdoc in theoretical physics, specializing in photonic crystals and other optical phenomena.

SM: I'm a professor at the University of Montréal, teaching and researching theory of computer languages. More specifically, I design new type systems and try and abuse existing type systems for "type based formal methods" purposes.

djcb: Earlier this year, the two of you took over the maintainership of Emacs from Richard Stallman. How did you get involved in hacking on Emacs? How has the transition gone?

CYD: My first involvement in Emacs-related development was around 2004 or 2005—very recent by Emacs hacker standards—when I found myself with some free time on my hands after college. At that time, I wrote wikipedia-mode, a major mode for editing Wikipedia articles, plus some word-wrapping code that eventually became longlines-mode, and patches to emacs-devel fixing a few minor bugs. My level of involvement gradually grew, until eventually I was helping Richard to roll the pretest tarballs for the Emacs 22 release.

Because I was quite active in the Emacs 22 release process, I've been pretty comfortable with my role in Emacs 23. It helps, of course, that many parts of Emacs have their own dedicated and experienced maintainers, e.g. the major Lisp packages such as CC-mode, Gnus, and Org-mode.

SM: I started hacking on Emacs a fairly long time ago when I was waiting to start my PhD, but it only got more serious during my PhD when I decided that PCL-CVS was a neat idea but unusable as it stood (for lack of maintainership). It all went downhill from there.

The transition to maintainership happened very smoothly. I had already considered maintaining Emacs when Gerd [ Gerd Moellmann ] left (i.e. when 21.1 was released; at which point Richard ended up regaining maintainership for lack of any other volunteer), but it was a pretty busy time for me, so I decided not to. This time Richard kept a very active role, which coupled with the help of Chong made it very pleasant.

There's a fair bit of pressure, of course, because it's a very old package, so people have a lot invested in it, making some changes terribly delicate. As a maintainer, I did get to steer the direction of Emacs development, tho mostly by my own contributions and by imposing some contentious new defaults. The role of a maintainer as I see it is mostly to make sure the package keeps its integrity.

But I have to say, that while Chong started maybe a bit more of a "rookie maintainer" than I, he quickly took over and he deserves much of the credit for 23.1, while I was too busy with my work to do much good.

djcb: Talking about Emacs development: there are of course many people involved. Can you give a estimate of how many?

CYD: There are about 120 people who have commit access to the code repository; of these, I think around 20 contribute regularly. This does not count the packages that are maintained separately from Emacs.

Additionally, we do of course receive a steady stream of small patches from various users.

Emacs 23 has just been released (on July 29 2009), congratulations, a great accomplishment indeed! From your perspective, what are the most important improvements in Emacs 23 for end-users? And what about the internals? Are there any big changes in the way Emacs operates?

CYD: I'd describe the Emacs 23 release cycle as dominated by internals changes, in contrast with Emacs 22, where most of the major improvements occurred at the Lisp level. There are two fundamental changes. First, the internal character representation is now Unicode-based, which simplifies various aspects of multilingual editing. Second, the font engine has been revamped, and, among other things, we now support anti-aliasing on X. Both these changes are due largely to Kenichi Handa, who deserves a huge amount of credit for patiently developing the code a period of years.

One other major internals change is a restructuring of the terminal interaction code, by Károly Lőrentey, which allows a single Emacs process to display on X and text terminals simultaneously. Building on this "multi-tty" code, Dan Nicolescu implemented a small but clever hack, allowing Emacs to run as a daemon serving emacsclient connections.

There are several Lisp-level changes, large and small. For instance, Stefan revamped the minibuffer completion code, which is now more sophisticated about generating completions. And there are, as usual, new modes and packages: Doc-view mode, Ruby mode, nXml mode, etc.

SM: Better support for Unicode, and better support for fonts, multi-tty support, plus lots of new modes as always. Of course, I'm very happy with my new completion code, which makes partial-completion-mode obsolete (and enabled by default).

The new support for Unicode and for fonts required significant changes. Big thanks to Kenichi Handa for most of that.

djcb: Are there any features that you would have liked to add, but that were somehow not yet ready?

CYD: One feature that I'd have liked to include into 23.1 is CEDET, a set of packages by Eric Ludlum (the author of Speedbar), which turns Emacs into an IDE. There was no time to merge it for 23.1, but hopefully it will be included in 23.2.

SM: Several packages were planned for inclusion, but didn't make it. Support for GNUstep was planned (and is actually in there) but doesn't work. Also I hoped the new VC code would be developed further, but it sadly stayed at the stage where it mostly provides the same features as the old one (with all kinds of improvements in the way it supports them, tho).

djcb: There is always a bit of tension in Emacs between keeping things as they are, and changing things to be more like other programs - for example when thinking about key bindings and various defaults. What is your take on this? Should Emacs try to accommodate new users, or instead try to keep things as they are?

CYD: My impression is that I'm a little more conservative than Stefan with regards to changes, though I'm not sure what he thinks ;-) That said, we seem to arrive at the same conclusions with surprising frequency.

SM: Emacs standard key bindings (like C-x and C-c prefixes) clash badly with "standard" key bindings of other apps, so I don't think there's much hope to make Emacs like other applications. But yes, I generally believe that, all things being equal, it's better to be like others than to be different in this respect. But since changing bindings (or behaviors) is disruptive, I only consider it worthwhile if I believe the new default is really superior (not just for new users).

djcb: For example, in Emacs 23, transient-mark-mode is the default, but delete-selection-mode is not. How do you decide such things?

CYD: Typically, we try not to make flashy changes. The transient mark mode change is the exception that proves the rule: transient-mark-mode is so useful, and is so widely used (even Richard uses it), that it doesn't make sense to leave it off by default. But the rule of thumb is to improve Emacs on Emacs' own terms; for instance, CUA mode will not become the default anytime soon, I think.

SM: transient-mark-mode is an enabler: it allows some commands to behave differently depending on the activation state of the region. So it's a clear improvement. delete-selection-mode is not as important in this regard. We may see something along the lines of delete-selection-mode at some point, tho probably something more minor that only caters to the few cases where delete-selection-mode is more than just a way to avoid hitting C-w.

djcb: How do you see the competition with other text editors? Do you look for ideas elsewhere? Is there any other editor you would be using if Emacs did not exist?

CYD: I'm afraid I don't pay much attention to other editors.

SM: I used Zmacs, XEmacs, and Epoch at some point. That's about it. I do like structured editors, and I think Emacs should and will move in this direction (with more parsing going on).

djcb: It's a bit premature of course, but it's always interesting to speculate a bit about the future. Do you have any particular post-Emacs-23 plans? Obviously, this all depends on what people come up with, but are there any directions you would like Emacs to go?

CYD: The present plan is for Emacs 23.2 to contain a small number of new features, in addition to bugfixes. As mentioned above, I'd like to try to include CEDET. In general, I hope to move to shorter, more disciplined release cycles. Emacs 23 was a good step in that direction, as it was shorter than the previous cycle.

SM: My main goal for Emacs-23 was to shorten the release cycle. Hopefully, the quality has not been reduced accordingly. For 23.N there are several improvements planned (or even done), mostly about inclusion of packages like js2-mode and CEDET. In the longer term, the main goals for me are the integration of the lexical-scoped branch, the support for bidirectional display, and adding more parsing technology (basically replace syntax-tables with something like lex & yacc, maybe).

Thanks a lot Stefan and Chong!

15 comments:

Anonymous said...

No mention at all about the sorry state of affairs of the OSX port ?

Emacs 23.1 removed the well-tested, very stable carbon port and merged the shitty, pretty much unusable Emacs.app for mostly political reasons (GNUstep is dead, who gives a shit about a dead platform).

Sam Aaron said...

@Anonymous,

Perhaps it's the low sophistication of my Emacs mastery but to me the ns (GNUstep) build works remarkably well. I just compiled from source:

./configure --with-ns

and I'm very happy with what it produces - no crashes or hangs. What issues are you seeing?

Sam Aaron said...

Perhaps this post answers my questions:

http://atomized.org/2009/08/cocoa-emacs-231-cvs-builds-and-the-nextstep-port/

I'm sure that my Emacs mastery is at such a low level that I don't notice these things. However, it just suggests to me that things can only get better - which is a great thing!

I do hope that the ns branch is actively developed.

Leo said...
This comment has been removed by the author.
Leo said...

Excellent article.

Maybe someone can link this article to osnews or slashdot.

a said...

Thanks for the interview, as well as all the work with the other posts.

Anonymous said...

As I mentioned on the atomized.org site, I think the X11 version using gtk works great on the Mac. It is only a "sudo port install emacs +gtk" away using Macports (although the port needs to be upgraded to 23.1). This gives one the ability to use focus follows mouse and the same fonts as the Cocoa port.

And the Mac OS X X11 builds from Xquartz are very good these days.

Perry E. Metzger said...

The first anonymous poster and the URL Sam Aaron posted both seem to think that the Cocoa Emacs.app front end is a bad thing. I've been using it with absolutely no problems, no speed issues, no obvious functionality problems. In fact, the integration of the cocoa port is one of the reasons I really love Emacs 23.

Unknown said...

Thank you for the interview. Very rare and very informative.

∑ Xah

an0 said...

Nice info.

To the first complainer:
I like the Cocoa port, but it is a little slow when scrolling long buffers.

Anonymous said...

(I'm the first poster)

I can understand that some of you have not seen the problems i faced with the cocoa port, but i hope you understand that a 'works for me!' attitude is not really saying much.

I work with hundreds of buffers in Emacs and i simply can't afford to have it crash every half an hour (which is the case here with the cocoa port) and take them all down. The slowness is an issue too but there are others far more important
(broken unicode glyph rendering, frequent hangs for more than a minute, cursor issues, frame issues and so on and so forth)
Here is the list of open ns bugs:
http://emacsbugs.donarmstrong.com/cgi-bin/pkgreport.cgi?pkg=ns

To the poster who recommended using gtk emacs under x11 on osx, i hope you are joking. I despise x11 and everything related to it, i'd rather shoot myself or use *gasp* VIM than force myself to use that malignant cancer.

Robert said...

GNUStep?? Come on! Just develop for Cocoa. Nobody uses GNUStep. Ugh.

Anonymous said...

There's misconception here that the MacOSX Carbon port was working.
It was not, after the large infrastructure changes that happened for emacs-23 it was broken. It's maintainer abandoned it and nobody did anything to bring it up to life, that is why it was removed: broken and lack of effort to fix it.

It people want to make the Cocoa port better, please start fixing bugs, help is most welcome....

mituharu said...

Those who are not content with the NS port may want to try the "Mac port". It is a descendant of the Carbon port and uses Cocoa AppKit for its GUI implementation basis.

http://lists.gnu.org/archive/html/emacs-devel/2009-08/msg00091.html

I believe it is stable enough, but marked it as experimental/hackers-only for some other reasons. Please read the `README-mac' file in the tarball.

Jonathan Rogers said...

I had been using Carbon Emacs 22 until recently. I wanted to switch to Emacs 23, but I've had a hard time finding an acceptable configuration. For a while, I used an NS/Cocoa build. That worked quite well in most ways except it randomly hangs. The hangs may be related to sub-processes or switching between programs, but I wasn't able to reliably reproduce it. Magit commands seemed to cause the hangs more often than anything else.

Since I couldn't stand having to kill and restart Emacs every few hours, I tried a Gtk+ build and that seems quite acceptable. The anonymous poster's rejection of Gtk+ just because it displays with X11 is illogical and ill-informed. For me, the only way it's clearly inferior to the Cocoa one is that the fonts aren't rendered quite as well. However, the difference is slight.