Re: Recommendations for web servers (case study)
Chris,
--- "Christopher C. Stacy" <cstacy@dtpq.com> wrote:
> On the other hand, if you don't have much static content compared to
> the dynamic content that will be generated by your Lisp web server,
> maybe putting Apache into the loop will increase the overhead.
> (Rather than doing a regular expression match against every URL,
> and then having to do an extra protocol layer to talk to Lisp
> most of the time anyway, it might be faster to just cut out
> the middleman and run everything from inside Lisp.)
Though what you're saying is sensible and logical, practice doesn't show it to
be true.
My company manages americandiamond.com, and we took a very plain vanilla X86
tower, put Linux on it, with 1 GB of RAM, and did exactly what I said, using
Apache in the front, and then a full-blown Tomcat running alongside it, on the
same server.
We had Apache running, with over a dozen regexps/rewrite rules. The static
content was all served off of Apache, while all of the dynamic stuff was passed
off to an often 300 MB Tomcat process (Java app server, much slower than a Lisp
app would have been, probably).
We made it through Christmas that way, and though I'm not supposed to say how
much volume they sold, it was quite huge. I'll give you some stats on the
month of December:
o 50 GB of data was pumped out by the server
o 25,000 unique visitors, totaling 45,000 visits for that month
o avg of 150 hits/visit
o 400,000+ pages served
Not to mention that this server was also running PostGreSQL. So while what
you're saying makes sense, a good Apache configuration goes a long way.
I should note that in November, the site was crashing about once (and sometimes
twice/day), until I relieved Tomcat of serving static content, and had Apache
do this directly. Once I made the change, the site became several times
quicker, and never crashed.
Another thing that's interesting is that our site was extremely quick...pages
loaded up (and continue to load) very fast. Note that the site is very heavy
on the graphics.
We considered doing it in CL in the beginning, but I was not sure that the CL
community had developed sufficient Web development tools. Now that it's all in
the past, and I know how we did it with Java, I'm interested in Lisp frameworks
for developing sites like that.
We used Hibername (analogous to LispWorks Common SQL), though I'd say that
Hibernate is probably a bit more mature, and supports both OMG's OQL (Object
Query Lanuage and their own HQL, which is what we used)
We used an amazing framework/CMS called JPublish to manage (and separate) the
static content, pages, templates, actions, etc. Action scripts were not coded
in Java, but rather in BeanShell, which is very Java-like, but it's
interpreted, so it's like having your pages in Perl or even CL, but compiled
on-the-fly, so you can still edit those pages in-place, and see the changes
immediately (and it was still fast enough). This was a huge win, of course,
since writing in the scripted Java is much easier and faster than the
full-blown compiled Java, with all the declarations, etc.
JPublish, in turn, used the Velocity Template Engine for templating, and this
too helped a lot.
The only problem with Apache is that if you're developing an Lisp app that has
an embedded webserver (e.g. a Web configurator), then you can't really do the
whole Apache thing. So the whole mod_lisp thing isn't quite as attractive as
something like PortableAserve.
dave