Summary: You can't actually do that. HOFs in a dynamically-typed language are not compatible with unboxed primitive invocation.

Our first pass used proxy to create a fn that implemented clojure.lang.AFn, a convenience implementation of IFn, which is the interface the compiler uses to recognize function-like things. For reasons that I am not well-versed in, IFn declares 21 arities of invoke (among other methods) that all take and return Objects. AFn implements these with throwing methods that can be overridden. Besides the fairly involved gen-class, proxy is the only API Clojure provides for subclassing a concrete class, which is a highly discouraged action. (We didn't want to reimplement AFn in order to use reify -- it was more work than necessary.) We finally managed to produce code like this:

user=> (binding [*print-meta* true] (clojure.pprint/pprint (macroexpand-1 '(org.baznex.imports/def-statics Math abs))))
(do
 (def
  ^{:doc "java.lang.Math/abs via def-statics", :private true} abs
  (clojure.core/proxy
   [clojure.lang.AFn clojure.lang.IFn$LL clojure.lang.IFn$DD] []
   (^java.lang.Object invoke [^java.lang.Object p_167]
                      (. java.lang.Math abs p_167))
   (^long invokePrim [^long p_168]
          (. java.lang.Math abs ^long p_168))
   (^double invokePrim [^double p_169]
            (. java.lang.Math abs ^double p_169)))))

This gets us an implementation not just of IFn, allowing the compiler to pick it up as a function, but also as an IFn$LL and IFn$DD -- 2 of 358 special interfaces introduced in Clojure 1.3.0 that indicate that the function is capable of being invoked with or returning an unboxed primitive.

And that's when we discovered that proxy only supports overloading by arity:

user=> (org.baznex.imports/def-statics Math abs)
CompilerException java.lang.IllegalArgumentException: Method 'invokePrim' redefined, compiling:(NO_SOURCE_PATH:4)

I bit the bullet and rewrote def-statics using reify instead. This meant writing something to loop through all 21 possible arities of invoke (as well as some extra methods), but it got done. The output was horrible (and this is the short version:)

user=> (binding [*print-meta* true] (clojure.pprint/pprint (macroexpand-1 '(org.baznex.imports/def-statics Math abs))))
(do
 (def
  ^{:doc "java.lang.Math/abs via def-statics", :private true} abs
  (clojure.core/reify

   clojure.lang.IFn

   ;; eliding .run, .call, and .applyTo
   (invoke
    [^clojure.lang.IFn this__144__auto__]
    (throw (clojure.lang.ArityException. 0 "java.lang.Math/abs")))
   (invoke
    [this__137__auto__ ^java.lang.Object p_449]
    (clojure.core/println "Invoked"
     "{:args nil, :ret java.lang.Object, :params (java.lang.Object), :prim nil}")
    (. java.lang.Math abs p_449))
   (invoke
    [^clojure.lang.IFn this__144__auto__ arg450 arg451]
    (throw (clojure.lang.ArityException. 2 "java.lang.Math/abs")))
   (invoke
    [^clojure.lang.IFn this__144__auto__ arg452 arg453 arg454]
    (throw (clojure.lang.ArityException. 3 "java.lang.Math/abs")))
   (invoke
    [^clojure.lang.IFn this__144__auto__ arg455 arg456 arg457 arg458]
    (throw (clojure.lang.ArityException. 4 "java.lang.Math/abs")))
   (invoke
    [^clojure.lang.IFn this__144__auto__ arg459 arg460 arg461 arg462 arg463]
    (throw (clojure.lang.ArityException. 5 "java.lang.Math/abs")))
   ;; eliding arities 6 through 19
   (invoke
    [^clojure.lang.IFn this__144__auto__ arg639 arg640 arg641 arg642 arg643
     arg644 arg645 arg646 arg647 arg648 arg649 arg650 arg651 arg652 arg653
     arg654 arg655 arg656 arg657 arg658]
    (throw (clojure.lang.ArityException. 20 "java.lang.Math/abs")))   
   ;; eliding the weird varargs arity

   clojure.lang.IFn$LL

   (^long invokePrim
    [this__137__auto__ ^long p_447]
    (clojure.core/println
     "Invoked"
     "{:prim clojure.lang.IFn$LL, :ret long, :params (long), :args [long]}")
    (. java.lang.Math abs (long p_447)))

   clojure.lang.IFn$DD

   (^double invokePrim
    [this__137__auto__ ^double p_448]
    (clojure.core/println
     "Invoked"
     "{:prim clojure.lang.IFn$DD, :ret double, :params (double), :args [double]}")
    (. java.lang.Math abs (double p_448))))))

Note the debugging printlns -- they were the only way I could tell if prim-invocation was occurring. Let's see it in action:

user=> (abs (long 5))
Invoked {:args nil, :ret java.lang.Object, :params (java.lang.Object), :prim nil}
5

Oops! We're only getting .invoke, not .invokePrim. With a little poking around in clojure.lang.Compiler, I discovered that arglists are important. Specifically, the type-hinting on the arglists. Here's an example of what they look like:

user=> (defn noop [^long x] nil)
#'user/noop
user=> (binding [*print-meta* true] (prn (:arglists (meta #'noop))))
([^long x])

This function is eligible for prim-invocation (that was a lie, and you'll see why in a minute) as confirmed by injecting some print statements into the compiler. So let's add some arglists to that reify!

(do
 (def
   ^{:doc "java.lang.Math/abs via def-statics", :private true
     :arglists `([^double x] [^long x] [x])} abs
  (clojure.core/reify

...

It doesn't work. Depending on exactly which function you try to reify, you'll get different results, so I'll just tell you the punchline: Compiler.java picks the first arity that matches the function call and runs with it. In this case, the [^double x] arglist is picked every time, leading to either casting, a cast exception, or a call to regular ol' .invoke. The compiler is completely incapable of choosing the correct IFn (or IFn$___ interface) invocation when there are multiple options for an arity.

But wait, there's more! After further investigation, I discovered that (map abs (range -3 3)) will always result in a call to invoke, or even worse, a runtime reflective call. Remember that :arglists is only present on the Var, not the IFn. (There's the lie exposed.) With (abs -5), Compiler.java sees that the call-position of the expression is occupied by a reference to a Var, and it is able to look up arglist info at parse-time. (There's some def-hoisting going on here that I don't entirely follow yet.) The IFn itself isn't available until runtime in all other cases, such as ((if P=NP? abs-LL abs-DD) -5), so bytecode cannot be generated with full knowledge of what optimizations might be available. The takeaway here is that the compiler can only work with whatever type info is statically available at the callsite, and since prim-invocation is a property of the IFn, only Vars of IFns can be optimized for unboxed primitive invocation.

And higher-order-functions? They pretty much never provide hinting information that the compiler could use. Remember that deep down inside (map abs (range -3 3)), the actual callsite we care about looks like (f (first s)). Completely vanilla. If we were using Java or Haskell or some other statically typed language, this information could be carried down inside the map call and used -- but Clojure is dynamic, so HOFs and prim invoke simply cannot mix.

Postscript: We plan to remove prim-invocation from the code and go back to proxy'ing AFn. We'll still be able to use hinting sometimes for its other, completely different purpose: Avoiding reflection. For instance, String/valueOf has a single ternary overload, so we can hint the actual call inside the ternary invoke. Farther down the road, we may combine def-statics and import-static so that users can choose whether they'd like each "import" to be a hinting macro or a boxing, reflecting function.

As it stands, the cloud is not safe for primary storage. I lost some data in the Magnolia bookmarking service fiasco myself, with my most recent backup several weeks before the crash. I only backed up my bookmarks when I remembered to, not having the time to work up an automated downloading script. Now I save all my bookmarks locally, and my twice-weekly backup takes care of everything. My online photo gallery is not in any similar danger, because I wrote it myself with such a failure mode in mind. The photos and tags and descriptions live on my laptop and propagate from there out to my public website. Images and tags that I don't want to be public never even leave my machine.

Why do people use the cloud, then? To put it simply, front-end convenience. I can create a Flickr or Numsum account in minutes, and right away I have a URL I can point my friends to to see what I have created. The cost (lost data) only comes much later. These hosted applications also present a reduced computer security risk to users, compared to desktop-based publishing software such as my photo gallery uses. If you want to use my application, you have to trust my code and run it on your own machine. However, Flickr only runs Javascript on your machine, safely sandboxed in your browser.

I want all of our current hosted-publishing systems move to the desktop. Clearly, a competitive answer to the cloud/hosted systems of today must compete on both convenience and easy security. Using an application must be as easy as surfing the web, and publishing photos, text, and video must only take a few clicks of the mouse.

What I envision is a desktop platform that allows users to download from the web and run (in a sandbox) any compatible software in a single click, where each application is given fine-grained permission to talk with other applications, read and write user data, and publish to the web, and a peer-to-peer communication system for non-public publishing (e.g. LiveJournal-like friend-locked blogging.) Since most users will not have always-on desktops, communications and storage would be optionally mediated by paid and freemium hosted services serving only as relays for encrypted user data.

With this imagined software, a user would be able to get started with photo publishing by finding a good-looking gallery application's website, clicking a special URL, and approving installation. Perhaps the user would then approve the application's request for 1) read-only access to a specified photo directory and 2) publishing permissions (to the public web, approved peers' inboxes, and relay servers). The application might provide a tagging and organization interface itself, or possibly just scrape data out of KPhotoAlbum, FSpot, and Picasa's databases. The user would connect certain tags to defined groups in their master contact list, allowing e.g. grandparents to be shown different sets of photos than coworkers.

Now imagine that at platform installation time, users are provided with a brand-new set of encryption keys and instructed how to hold key-signing parties, assisted by the platform's authentication module. Privacy-sensitive content (photos, blog entries, video, source code) would be created on the desktop, encrypted to group and personal public keys, distributed on personal and paid servers or via P2P, and downloaded to peer machines. At no point would the prime copy of a user's creations be offsite -- and at no time would personal information be out of the control of the user or those they trust with it.

This project is totally feasible in technical and social terms. It could revolutionize user privacy and even be the required bootstrap for layman's practical encrypted communication. It just needs the right group of people to start the work. Comment if you think you could be one of those people.

On local installs of Firefox (or in Windows), Firefox handles its own upgrades, and has the ability to warn the user if any extensions will be disabled due to incompatibility with the new Firefox version. However, when Firefox is managed by an external agent such as APT, no such warning can take place. (There's another problem here—upgrading Firefox when it is running results in gradual bitrot of the running process as it loads mismatched bits of itself from disk.)

This is totally a solvable problem. Deb packages can contain pre-install scripts that check for bad install or upgrade conditions, and I would bet they can also prompt the user for input in questionable circumstances, such as forced disabling of extensions. There's the minor issue of how to handle multiple user accounts, networked /home, etc., but these all seem solvable with a reasonable amount of effort.

I suspect this post will be too late to influence the Firefox 4 release, but I'm filing an enhancement request anyway. Hopefully the developers of other apps with package managers will take this advice as well.

NB: Emacs is not one thing. This post is abut GNU Emacs 23.

Most interaction with Emacs (besides normal typing of text) is via the issuance of commands. These commands almost exclusively involve modifiers like Control and Meta, which correspond to Ctrl and Alt on most keyboards. If you see the keyboard shortcut M-x, you should type the [X] key while holding down [Alt]. Similarly, C-h means [Control]-[h]. Some commands involve a sequence of keystrokes. You can remap all of these later if you wish. (You should also familiarize yourself with some other key and character conventions.)

Here are your first commands: C-g will cancel just about anything (a command in progress, a selection, etc.) and C-x C-c will quit the program. Note that C-x in most programs is "Cut to clipboard"—this is not what it is bound to in Emacs. In fact, none of the usual keyboard shortcuts apply. You'll learn about their replacements soon.

When you first launch Emacs, you will probably see a single window split into two panes, one above the other. (Emacs calls this a vertical split instead of a horizontal split, confusingly, but we'll stick with the convention.) These panes are called windows in the rest of this document. You have a cursor in each one, but only one will be blinking and active. This is the current window. Want to get rid of all windows except the current one? Use C-x 1 to do that. C-x 0 will close the current window.

The stuff you see inside each window is a buffer, which can be anything from a command line to a document to an interactive settings file. You can have a bunch of buffers open at any given time, and each window can be targeted to show any one of them. You will usually just have one window open and keep switching which buffer is being viewed in it. C-x b will give you an interactive prompt that will let you swap out which buffer you are using, and C-x C-b will give you the full list in the current window until you position your cursor over the one you want and hit [enter].

Emacs is always in one major mode at any given time. Major modes generally change with your programming language, and affect syntax highlighting, autocompletion, compilation, etc. There are also minor modes, which can coexist peacefully. Most notably, modes change what certain commands mean. There's a command to move forward by one word, but the meaning of "word" changes between programming languages. (If you're using an s-expression language, for instance, you'll really want the Paredit mode, which lets you sling parens like a master.)

Undo mistakes with C-/. If it really wasn't a mistake after all, you'll still be able to get it back later by undoing some more!

Now that you know the real basics, you're ready to look at all those other Emacs tutorials! Your next steps:

1. Open and save files
2. Keyboard navigation (and why the arrow keys can be trouble)
3. Undo, and redoing by undoing
4. Selection
5. Killing and yanking are not just different words for cutting and pasting
6. Understanding, installing, and enabling modes
7. How to use the amazing help system (C-h)

I'll add to and modify this document as my own understanding progresses.

Oh yeah, and here's some common stuff that isn't mentioned in regular guides:

• Trying to save to a new filename that has a space in it, and Emacs isn't letting you? C-q space will put in a literal space.
• Want to put a newline character in a search or replace pattern? C-q C-j inserts a ^J (line-feed character), which is usually what you want. In DOS mode you'll need to insert an ^M (carriage return) before the line feed, otherwise some programs might freak out.

(defn magic []
  (println "magic happens") ; one too few closing parens

(defn selfr [i]
  (if (< i 0)
    0
    [(selfr (dec i))
     (selfr (dec i))]))) ; one too many closing parens

(defn selfr-caller []
  (selfr 3))

(defn -main [& args]
;  (magic) ; uncomment to make everything work
  (println (selfr-caller)))

If you stare at the code for a bit, you'll see that selfr is defined inside the magic function. So why doesn't the compiler complain that selfr-caller can't see it? What I didn't realize until today (with the help of some wonderful people on IRC) was that defn hoists the declaration to the top-level, where everything can see it... but the assignment won't occur until the code block is actually run.

This kind of bug is extraordinarily tricky to find, since it is ultimately a syntax error but may not show symptoms depending on the execution order of the code. I suppose this could have been prevented by having a better editor, such as one with draconian indentation support (e.g. Emacs) or perhaps a matching-paren highlighting feature.