Emacs 24 Package System Initialization Problems

Question

It seems to me that the new Package system that is built-in on Emacs 24 has some flaws when it comes to properly loading and initializing the installed packages.

Recently, I upgraded to Emacs 24.1.1 which was realeased on 6/10/2012 and I have been trying to use the built-in package system and have installed several packages using it, but they all have a similar problem related to autoload and initialization.

For example, I use a package called smex which provides enhancements for using the M-x chord. It requires you to define a key for M-x, so I added (global-set-key (kbd "M-x") 'smex) in my init.el file. But after starting emacs I press the M-x chord and I get the message "Symbol's function definition is void: smex" ... If I also put (require 'smex) in my init.el file I get the error message "File error: Cannot open load file, smex"

Adding the location of smex to the load-path variable makes it work as expected, however, that seems to defeat the whole purpose of having a package system in the first place...

Any thoughts? Is there a better way or do we live with this limitation for now?

Solution 1

The packages that you install with package.el are activated by default after your .emacs is loaded. To be able to use them before the end of your .emacs you need to activate them by using the commands:

(setq package-enable-at-startup nil)
(package-initialize)

Solution 2

It's worth noting why Emacs defers the package initialization:

See C-hig (emacs) Package Installation RET, and in particular:

The reason automatic package loading occurs after loading the init file is that user options only receive their customized values after loading the init file, including user options which affect the packaging system. In some circumstances, you may want to load packages explicitly in your init file (usually because some other code in your init file depends on a package). In that case, your init file should call the function package-initialize. It is up to you to ensure that relevant user options, such as package-load-list (see below), are set up prior to the package-initialize call. You should also set package-enable-at-startup to nil, to avoid loading the packages again after processing the init file. Alternatively, you may choose to completely inhibit package loading at startup, and invoke the command M-x package-initialize to load your packages manually.

So provided you make sure that your init file takes care of any non-default values you want for variables in the package customization group¹ before calling package-initialize -- and that you maintain this approach whenever customizing the package library config -- it should be okay to do this.

Alternatively, because after-init-hook runs after the standard package initialization has completed, you could use that to evaluate any init code which depends upon packages. So instead of calling package-initialize directly in init.el, you could instead write:

(add-hook 'after-init-hook 'my-after-init-hook)
(defun my-after-init-hook ()
  ;; do things after package initialization
  )

putting the code requiring the initialized package system within that function.

YMMV.

(n.b. I haven't tested the after-init approach, as I don't really use package.el; but I did confirm the sequence of events in the start-up code, so I believe it will work as described.)

¹ M-x customize-group RET package RET

Comments

[Erik Kaplun]

I think the answer by phils is much more valuable and even correct than this one; he also explains why.

[Erik Kaplun]

I've no idea why this answer has received so few votes and has not been accepted... In light of what you've explained, the solution offered in the accepted answer even begins to seem like a hack.

[Nicolas Dudebout]

@ErikAllik, downvoting is to report that information is inaccurate. You should only have upvoted the other more accurate answer. To understand why this answer was accepted, just look at the dates. It was written 15 months before the other one.

[Erik Kaplun]

It was only to balance the scores a little bit—a possibly inappropriate act :)

[Veritas]

Great answer! Do I still need to require my packages after loading them with package-initialize?

[phils]

Veritas: It varies by package. Well-written packages will typically include autoload cookies (which are processed by the package manager) for important commands, in which case you are unlikely to need to require them (which in turn will improve your start-up time). Packages with no autoloads will still need to be required (or loaded somehow, at any rate).