Re: Best way to deploy to a cluster of servers?
On 17 Apr 2006, at 04:43, Chris Dean wrote:
> What's the best way to deploy an app to a cluster of servers?
>
> I have a LispWorks application that I deliver to a cluster of Linux
> boxes. Right now we run about a dozen copies of the same program
> (with slightly different configuration files) on a dozen different
> machines. We expect the number of machines to increase drastically
> over the next year or two. We/I have complete control over these
> machines and they don't do anything else except run our application.
Sounds like an interesting project: maybe you should write something
about it someday!
> On one of the development boxes we have a delivery script to make a
> single executable and we copy that executable out to the different
> machines. The final executable parses its command line and starts
> up -- not much different from a C program.
>
> What's the best way to deploy this app? For example, it would be nice
> to have a repl to connect to so that we could diagnose misbehaving
> apps. Here's what we've thought of:
>
> - Leave as is, no repl.
> - Embed a swank server.
> - Embed a simple repl listening on a TCP port.
> - Use detachtty.
> - Don't use deliver, just save an image (not sure if that is within
> our LispWorks license) and deploy using detachtty.
>
> Also, we can deploy the source code if that is useful for some reason.
In terms of licensing you cannot work with source code distribution:
that would require a file compiler and thus a full license on each
machine. 'Cannot' is too strong of course, 1100 per server is still
quite cheap compared to other server technologies.
I would add another requirement to your list:
- the ability to update running images with new code (patches,
extensions, ...) without taking them down.
My current system (which I would like to improve because it takes
some time to set up) is as follows:
I have a skeleton server image called lw-base: this is an almost
empty level-0 delivered tty mp image.
It contains some directory/location management code and the standard
asdf code.
When this image boots, it loads a config file from its own etc dir,
that in most case starts loading precompiled fasl files from asdf.
The asdf is set up to load from its own sys and dist directories - to
make everything self contained.
The image itself is managed by an lw-ctl script: this makes it easier
to start/stop/restart/kill/... the process.
The image is not started directly, but using screen (detaching
automatically). Reconnecting to the startup repl is done by
reattaching to the startup listener.
Updating server code is done from a special mirror setup on a
development machine: a single deploy script checks code out from
different repositories, compiles it using asdf in the local tree and
rsyncs the fasl files to the server.
On the server, a single asdf command is often all it takes to load
the (new) code; since everything is precompiled, the file compiler is
not needed. I ofter do small patches by copying and pasting functions
in the repl on the server; the evaluator is available in delivered
images.
On some projects I also load swank and use slime to connect to the
server image: it is a higher level, more functional interface.
Sven
PS: In the past I used to work with custom repl code but found that
unreliable when debugging certain mp conditions, a common situation
with server code.