I am a fairly new member of the Lisp community, so new that most people have never heard of me. I’m one of the young programmers that have read the excellent writing from Paul Graham, Raganwald, and Steve Yegge and decided to try Lisp out for myself. I’ve had much more time available in small chunks, enough to read tutorials and articles on the internet but not enough to dive into heavy programming. As a result, I’ve gotten to know the community and history better than the language and tools, probably better than any new convert. Here are some observations I’ve made (summary then detail):
- Lisp is like a new language with lots of praise from important people.
- Lisp used to be very important but lost that position.
- Lisp lacks the infrastructure to get new users running quickly. The tools are there but they aren’t organized and streamlined.
- The Lisp community is fractured but composed of enough good people that it could develop into a positive, influential force.
1) Lisp feels like an emerging language (think Ruby circa 2003) that may or may not make it big but happens to have 50 years of history under its belt. I recently read that said SBCL and Clisp were the top two open source Common Lisp implementations, with single digit thousands of installs each. A decent number but nothing earth shaking. Interest could at this level for years, or given the right conditions, it could take off immensely (see Rails, Ruby on).
However, Lisp also has the burden and the blessing of decades of history. It was doing things 20 years ago that modern languages are just beginning to incorporate. Years of large scale development have driven improvements in the language, tools, and performance that blow the doors off of new scripting languages like Python and Ruby. It was the language of choice for the hardest problems of the 70s and 80s. Luminaries in all aspects of computing have publicly praised Lisp for being so great (Eric Raymond, Alan Kay, Peter Norvig, Guy Steele, etc).
But . . . decades of work by lots of brilliant people have caused Lisp’s family tree to splinter instead of remain close knit. The ease of writing software in Lisp means that people are more likely to roll their own tools instead of standardizing around a few common tools. And because it was so old and mature once the Internet started driving new users, the community infrastructure is not geared towards new users. This is more than a little weird compared to other languages.
2) Lisp used to be much more prominent in computing, and there are people around who remember that era. Lisp was the language of choice for CAD, artificial intelligence, expert systems, and many other challenging problems. There were multiple competing vendors offering Lisp environments, with new innovations in tools, improved compilers, and high performance. There were even two companies that made hardware to run Lisp natively and a complete stack on top of those machines! At that time, languages like Java, .Net, Perl, Python, Ruby, and even C++ didn’t even exist and Lisp was only competing with C, Cobol, Fortran, and Assembly.
To make a long story short, the game changed and Lisp fell from prominence. A number of factors torpedoed the Lisp machine companies, the DoD pulled funding for AI research, competing languages improved and incorporated some of Lisp’s features like garbage collection, and the Common Lisp standard took some of the innovation out of commercial Lisp vendors. Although it fell from prominence, during its heyday Lisp was a powerful contender in the computer world. Keep in mind that at this time, computer hardware was expensive (too expensive for individuals) and was almost exclusively owned by corporations and universities.
3) Today, most of the work in programming languages and development technology is done by individuals, on their own hardware and often on their own time. Since everyone is a volunteer, tools that cost money will reach many fewer people than free and especially open source tools will. Since most people are volunteers as opposed to employees, they care about stuff like “How long will it take me to get setup?” and “How hard is it to resolve issues?”. This is where languages like Ruby and Python shine. How long does it take to get setup? Just download the installer package and run it! Libraries included, directions just like the tutorial, easy. Oh yeah, and it works for Windows and Linux and Mac. Batteries Included.
With Lisp, newbies say “How do I get started?” and instead of a URL, they get “Lisp is a family of languages, with the primary branches being Common Lisp and Scheme. But these are just language standards with a number of implementations. Which implementation is best for you depends on your platform and computing needs.” Huh? From reading a bunch of blogs and discussions, the impression I got is that the two best environments are SBCL and PLT Scheme. But it seems rare for anyone to say “You should use implementation X” or even “You should use implementation X unless you need to do Y”. This confusion makes it hard to even know how to get started.
Enough has been said about emacs. I think it’s great fun and I’m enjoying learning it, but I can’t say that emacs/Slime is the most welcoming combination for newcomers.
And there are more and more of these types of things. In tutorials/demos, it’s not clear how your file structure should go. It’s not clear if a command should be entered in your code file, in the REPL, or in your SBCL init file (which is where, again?). Part of my problem is that I wanted to learn all the nitty gritty stuff, so I’m learning Linux, emacs, SBCL, Slime and Common Lisp at the same time. This means that I have to leave tons of time to work through issues, because I could run into something in any of these layers of my stack. I’m not sure what it is about me that keeps me from using Lispbox, but something like this would make life a lot easier. This should probably be the starting point for new Lispers, sort of like InstantRails is for the Ruby on Rails community.
Incidentally, there are good resources out there, but they need to be vetted and organized. Hey, I’m an organized person, maybe I’ll volunteer for this! (Oops, by publishing this, I think I just did (gulp)). (clearly my writing style predisposed me to write Lisp code).
(Note: If I had searched my feeds before I sat down to write, I would have found that Eli Bendersky basically wrote this post for me five months ago. Thanks Eli!)
4) The community is . . . special. There really is no community per se, but several communities bound together by one common name. There are the crusty old guys (the ones people warn you lurk on comp.lang.lisp), the generally good experienced developers (Peter Seibel had received several compliments), and the new people asking “dumb questions”. It’s a challenge having all these groups interact, and the results aren’t always pretty. I haven’t ventured onto c.l.l or IRC, but there are enough good Lisp bloggers that I feel connected (note to self: connect back more). People complain about the community (mainly because of group #1), but a couple dozen or so active, friendly, constructive people could create a positive impression for all visitors to the community. Just look what happened to Ruby with the Rails team.
And so that’s where I think I’ll leave off (now I just need to trim and edit the 1,000+ word ramble above):
So the Lisp “Golden Age” of #2 won’t be back (the computing world is too different), but the future could be very bright indeed. Let’s make it so.
(Of course, people have been making this observation for a while. Let’s see if anything changes.)
See Also:
My response to many of the comments
Comments on programming.reddit.com
*Original text: “better competing languages were developed,”
Great observations. I particularly agree with things being streamlined a bit more.
It pisses me off when the answer to “How do I program in Lisp?” kicks off a discussion about standards and scheme and Lisp-1 vs Lisp-2. If it were me, I would either point them to the Common Lisp Survey: http://common-lisp.net/~dlw/LispSurvey.html or just say SBCL. There is no need to be pedantic with newcomers. In time, they will figure out that SBCL on Windows may not be the best idea, but for now, it doesn’t matter.
If you are new to Lisp (as I am), please post about your experiences. I think it helps newcomers feel more comfortable. I personally just post on my blog because I think it is a way of giving back to the people who helped me.
I’d like to end this comment (rant?) by saying that Lisp is great for at least these reasons: 1) I don’t need to give a damn about syntax. 2) Except when I do. 3) Edi Weitz.
That is all.
Now replace Lisp for Smalltalk and 80% of this post still holds.
There was no golden age of Lisp. Rewriting history doesn’t help anyone.
“Lisp was the language of choice […]”
KI and expert systems have also been written in Prolog, Lisp didn’t dominate that market. And hardware Lisp machines were failures.
That said I’ve been very interested in Lisp since the beginning of the 80s when I first got an old print out of a lisp tutorial with macros.
I would’t use it to develop applications with it though.
Peace
-stephan
The streamlining is incredibly important. People just aren’t willing to accept a language now unless it’s part of a larger system, comprising a language-aware text editor, on-line help, and easily-installable packages. And quite rightly, too — it’s just easier to get stuff done in Ruby, so why bother? Lisp sometimes feels too much like watcing a small black-and-white TV. Y’know. With a REPL.
Also, the many implementations problem probably contributes. Limited developers re-invent the compiler over and over; the number of programmers working on the really important support stuff like documentation and installers and module libraries is low; poor support hinders takeup; that keeps the community small. Possibly, we need a squad of goons who threaten the developers of any lisp other than, say, Plt Scheme. (Plt isn’t a considered choice — it’s just the first one that popped into my head.)
Lastly, it might be a language searching for a problem; it’s been preying on me lately how much XSLT looks like half-arsed scheme with nasty pointy-bracket syntax. It might be that a lisp, with s-expressions written with pointy-brackets, would became popular for doing things with web pages and XML. Then you introduce the ‘shorthand notation’ using traditional parens, and voila — the world’s using scheme.
hey, i would try out clojure. Its (the only one) of those `bright’ lisps. As the author of clojure says: “Clojure is a Lisp not constrained by backwards compatibility”.
And so it feels. You can yust do the things not done in lisp before. For example, as a programming newbie, i _want_ to be able to show my pretty “hello world” string _not_ on the commandline, but in a nifty windows-alike messagebox: (. javax.swing.JOptionPane (showMessageDialog nil “Hello World”)).
I like that, even tough i like using emacs to edit my code, generally users are afraid of not using the mouse until they become more advanced. So try clojure, its the new lisp.
Sohail – your blog is one of the more useful resources I’ve found for giving summaries, evaluations, and starter tips on different Lisp libraries. Keep it up!
Razvan – sadly yes. I was debating between learning SmallTalk or Lisp, but bad OO experiences with C++ and Java drove me away from OO. Someday I’ll learn Smalltalk and heal the old wounds.
Stephan – It was a Golden Age from Lisp’s perspective, even though it was small compared to the mainstream. That era certainly beats the 90s and 00s for Lisp usage.
Steve – the diversity of work that people are doing is great – we just need more bodies (brains, I guess) to go around. Also, about XML, check out Steve Yegge’s take on this: http://steve.yegge.googlepages.com/the-emacs-problem
foobert – There are a lot of really cool Lisps out there: Clojure, Kawa, Gambit, etc. I’m sticking with SBCL because I want one of them to get out of the gate and become more mainstream. My definition of mainstream is that you can use it at work without having to start your own company.
A great Lisp add-in for eclipse. check out:
http://bitfauna.com/projects/cusp/index.html
And for Scheme, there’s SchemeScript:
http://schemescript.sourceforge.net
Oups, the link should have been:
http://sourceforge.net/projects/schemeway
It won’t do everything that you need but ABLE (http://phil.nullable.eu/) isn’t a bad place for a newbie Lisper to get started.
It’s very hard to know how many people are using Lisp. Some people who use it don’t admit that they do, for various reasons. Similarly, many people still use rule-based expert systems, but rarely advertise it, since the technology fell into disrepute after it was overhyped in the late 1980’s. Lisp got “collateral damage” from that episode.
There are actually a lot of ideas from Lisp in Java. But the Java people never admit that Lisp is one of the “parents” of Java, because it’s just not good public relations these days. Note that James Gosling actually wrote his own Lisp dialect once, for his Emacs-like text editor, long before Java happened. So he knows all about it. Saying “Lisp” just doesn’t sound good at this point in time.
The family tree is actually far less splintered than it used to be. In the early 1980’s, many divergent Lisp efforts came together to form the Common Lisp standard. The main competitor, Interlisp, is pretty much gone now. Of course, it’s quite easy to come up with your own Lisp dialect, and this has been done to great benefit by GNU Emacs, AutoCAD, and others (Interleaf, R.I.P.). But those are for very specfic purposes. There’s also Scheme, which has a rather different orientation, and doesn’t call itself “Lisp” anyway. So what’s left for general-purpose Lisp programming is Common Lisp. There are currently eleven maintained implementations of it, and as far as the language goes, they’re quite compatible. (The catch is that the language does not go as far as you’d like; more modern “standardized” libraries are needed.)
So it’s not really a family of languages. If you want Common Lisp, you do have a choice of implementations. I have written a survey of them, which should help you choose. SBCL is a fine choice (if you don’t need to run on Windows); the other free one that I like and know is Clozure. Either is fine. They’re very easy to download and install. Then read “Practical Common Lisp”, or any of the other good books linked to by the LispSurvey.
Regarding “I’m learning Linux, emacs, SBCL, Slime and Common Lisp at the same time.” Well, if you want to use Ruby, you’ll still have to learn Linux, and learn something to edit code such as Emacs. If you want an IDE, you’ll have to learn something like Slime (yes, maybe something easier to learn; check out “Cusp” as mentioned in the comments to your post), or in either language you can do without an IDE. And there isn’t ALL that much to learn about SBCL other than learning Common Lisp itself. And with any language, you have to learn ground rules like how to organize the code into files and what to put into your init file; that’s hardly unique to Common Lisp. So I don’t think you’re being quite fair in your comparison.
Indeed, the good resources need to be vetted and organized. There are people thinking about doing this; it’s a matter of pulling together the person-power, mostly.
Replying to some of the comments to your post:
Nobody has to worry about Lisp-1 vs. Lisp-2, other than to decide whether to use Common Lisp or Scheme. Why blame Common Lisp for the fact that Scheme exists and you “have to” choose between them? There are load of languages to choose from. Common Lisp and Scheme are just another two.
There really was a “golden age of Lisp” when Common Lisp was the only practical language around that had garbage collection! Most of you are probably too young to remember what the programming language scene was like about ten years ago: bleak, as the original post pointed out.
To see that I am not a raving Lisp bigot (as I sort of was in my youth, say thirty years ago), read this blog posting.
Lisp certainly has real drawbacks. Ten or fiftenn years ago, it truly looked like we would not be seeing interesting new languages any more (just as we are not seeing much innovation in operating systems right now). But the pendulum swings, and now there is a new golden age of computer languages happening. One that caught my eye recently is Scala.
Usually periods of creative birth and diversity are followed by periods of consolidation, where a few of the new entities become the popular ones. We’re in the creative outburst phase now. It’s fun watching all this happen.