Re: Security concerns in Common Lisp

On 25 Sep 2008, at 17:32, Jeff Massung wrote:
>
> The basic concern I have boils down to EVAL. Let me give a quick  
> example that doesn't seem to have problems, ask why, and that should  
> give an idea of where I'm going. Let's say I created a simple web  
> server with LispWorks that had a simple edit box that allowed a  
> visitor to play with Lisp, enter an expression, click submit, and  
> see the results. What prevents the user from entering something akin  
> to (run-shell-command "format c:")?
>

Nothing does unless you do.

You need (at least) to set reader control variables to be safe, and  
then walk the resulting form looking for toxins.

I have some stuff which originated with this in mind, which I could  
perhaps dig out.

However it's important to realise that if you want a language which  
completely unknown people can type completely unknown stuff at on your  
machine, CL is probably not an ideal match for that.  For a start you  
have the problem of defining what a "toxin" is above, cleaning up the  
image after code has run in it, and you also have to trust things like  
the reader not to misbehave when completely arbitrary junk is spewed  
at it.

Re: Security concerns in Common Lisp

Re: Security concerns in Common Lisp

Jeff Massung writes:
> I haven't really found much online about this, but I'm a little worried
> about security with a current application I'm writing. This is new territory
> for me, so I'd appreciate any links or insights other who have dealt with
> this before can share...
> The basic concern I have boils down to EVAL. Let me give a quick example
> that doesn't seem to have problems, ask why, and that should give an idea of
> where I'm going. Let's say I created a simple web server with LispWorks that
> had a simple edit box that allowed a visitor to play with Lisp, enter an
> expression, click submit, and see the results. What prevents the user from
> entering something akin to (run-shell-command "format c:")?

Your reputation is done!  ;-)

What would prevent a user to enter something like that would be to put
a chip in the brain of each newborn worldwide, to make them zombies
unable to go beyond the law.

In the meantime,  your web application will have to avoid executing
such an expression, because I can assure you that you will get them
with 100% probability.


> That example (the website) is probably a little easier given that one could
> make the Lisp process run as a guest and various shell commands would be
> completely unavailable. But, what if it was just an end application reading
> a configuration file with Lisp data for ease or (as in my case) a
> server/client sending Lisp data back and forth across the internet, where
> both users are most likely running as superusers (yay for Windows)?
> 
> Anyway, any advice people have on this is much appreciated.


The standard advice in security matters is to allow only what is
explicitely allowed.

1- Either implement a safe lisp reader handling #\: safely
   (eg. prevent access to unallowed packages such as SYSTEM or SB-EXT)
   with only "safe" reader macros (eg. #\; and #\( would be ok, but
   not #\# #\.).

2- Implement a minimal compiler.

3- Minimaly compile the expression (this removes the macro calls,
   which can only come from the allowed safe macros, and therefore
   cannot execute anything unsafe during macro expansion).

4- Walk again the resulting form to check that all the remaining
   special operators and functions are in the safe list.

5- Eventually, you can either call EVAL, or run your own compiler or
   interpreter. 

(Don't forget to handle all the conditions).

CL:+     could be added to the safe operator list.
CL:OPEN  shouldn't be added to the safe operator list;
         you could have a SAFE-VIRTUAL-CL:OPEN instead.
etc...


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

PLEASE NOTE: Some quantum physics theories suggest that when the
consumer is not directly observing this product, it may cease to
exist or will exist only in a vague and undetermined state.

Security concerns in Common Lisp

Well, yeah, running an EVAL server means deliberately opening up
your system and a free-for-all.  Web "scripting languages" and
libraries (and all web applications) go to some trouble to make
sure that such a service never happens.

If you are just trying to READ some data, call READ (or more
likely, READ-FROM-STRING) and bind *READ-EVAL* to NIL.

If you are trying to give the user a server to some language,
then write an interpreter that checks the input you READ as above.
This language could be your own carefully implemented Lisp interpreter.

If you are intending to give the user a server to the underlying Lisp,
using EVAL will let the user do anything the Lisp process can do, 
so you should take steps to limit that (eg. run in an operating system
sandbox where they can't open any files, limit cpu and memory usage, etc).  
That's a mouthful and I'm not sure how easy/possible it would be to do so.

Your Lisp implementation may have features that allow it to be
re-programmed from within itself.  This might make it harder
to secure than some other dynamic languages.   But opening up
what is essentially a general computing server to the web is
a formidable idea regardless of the language.

Re: Security concerns in Common Lisp

Jeff Massung wrote:
>
> Let's say I created a simple web server with LispWorks that had a
> simple edit box that allowed a visitor to play with Lisp, enter an
> expression, click submit, and see the results. What prevents the user
> from entering something akin to (run-shell-command "format c:")?
Nothing, really.  Such a server is inherently highly unsafe.  I know of
no Common Lisp
implementation that attempts to implement a "sandbox" of the sort that
Java attempts
to implement.  You have to limit its privileges using the underlying
operating system's
facilities.
>
> That example (the website) is probably a little easier given that one
> could make the Lisp process run as a guest and various shell commands
> would be completely unavailable.
Yes, that's what I mean.
> But, what if it was just an end application reading a configuration
> file with Lisp data for ease or (as in my case) a server/client
> sending Lisp data back and forth across the internet, where both users
> are most likely running as superusers (yay for Windows)?
Ah.  First, it's really not a great idea to design your system so that your
configuration files are Lisp forms to be evaluated.  First, it's just too
easy for mistakes to have arbitrary, bad effects.  Second, full Lisp
is usually "too powerful", in the sense that a simple declarative
format can be parsed easily by a program, whereas a Lisp file
might have arbitrarily complex Lisp forms that are hard for
a program to understand. Yes, there are good reasons for
programs that can read and understand config files.  You may not
have one now, but you should design your architecture to allow
for them in the future, IMHO based on years of experience.

If you really are going to use Lisp, but not Lisp forms,
then you'll find yourself using the Lisp "read" but not
the Lisp "eval".  That's a lot safer, but there is one
hazard that you need to know about: there is a reader
macro, #., that means "read one form and evaluate it".

Back in the Symbolics days, our mail reader supported
a "Font:" header.  It would use the Lisp reader to read
the name of the font.  One day some hacker figured out
that you could send mail whose "Font:" header used this
trick, and the victim would suffer from anything at all
when reading that mail.  (Of course, nobody did anything
that really hurt anyone; usually it popped up a window
that said something silly.)

There is a specific feature that exists entirely to deal
with this problem (which we put in very shortly
after the "Font:" hack was invented!).  What you do
is bind the special variable *read-eval* to false (nil).
(Or you could make a special readtable of your
own, but that's more work unless you have some
other reason to be doing it.)

-- Dan


>
> Anyway, any advice people have on this is much appreciated.
>
> Thanks!
>
> Jeff M.

-- 
________________________________________
Daniel Weinreb
http://danweinreb.org/blog/
Lisp: Implementation, Semantics, and Practice
http://il09.org

Re: Security concerns in Common Lisp

Re: Security concerns in Common Lisp

Re: Security concerns in Common Lisp

Unable to parse email body. Email id is 8680

Re: Security concerns in Common Lisp

Gary King writes:
> Hi Pascal,
> 
> What you wrote below is a good summary. Maybe we should look for  
> volunteers to implement it for next years Summer of Code.
> 
> >> The standard advice in security matters is to allow only what is
> > explicitely allowed.
> >
> > 1- Either implement a safe lisp reader handling #\: safely
> >   (eg. prevent access to unallowed packages such as SYSTEM or SB-EXT)
> >   with only "safe" reader macros (eg. #\; and #\( would be ok, but
> >   not #\# #\.).


Done.
 http://darcs.informatimago.com/public/lisp/common-lisp/reader.lisp
 http://darcs.informatimago.com/public/lisp/common-lisp/source-text.lisp

> > 2- Implement a minimal compiler.

I'm slowlydottedly working on it.

> > 3- Minimaly compile the expression (this removes the macro calls,
> >   which can only come from the allowed safe macros, and therefore
> >   cannot execute anything unsafe during macro expansion).
> >
> > 4- Walk again the resulting form to check that all the remaining
> >   special operators and functions are in the safe list.
> >
> > 5- Eventually, you can either call EVAL, or run your own compiler or
> >   interpreter.
> >
> > (Don't forget to handle all the conditions).
> >
> > CL:+     could be added to the safe operator list.
> > CL:OPEN  shouldn't be added to the safe operator list;
> >         you could have a SAFE-VIRTUAL-CL:OPEN instead.
> > etc...

Then you may need some additionnal stuff, like a securised file
system.  I started to implement a "VIRTUAL-FS", which implements all
CL I/O, to a pseudo-file system in memory. (As is, it relies on saving
the lisp image for persistency, but it could be extended to
persistently store the data to a store such as S3 using Xach's CL-S3).


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.

Re: Security concerns in Common Lisp

Rainer Joswig writes:
> 
> 
> Am 25.09.2008 um 20:53 schrieb Pascal J. Bourguignon:
> >
> 
> > The standard advice in security matters is to allow only what is
> > explicitely allowed.
> >
> > 1- Either implement a safe lisp reader handling #\: safely
> >   (eg. prevent access to unallowed packages such as SYSTEM or SB-EXT)
> >   with only "safe" reader macros (eg. #\; and #\( would be ok, but
> >   not #\# #\.).
> >
> > 2- Implement a minimal compiler.
> >
> > 3- Minimaly compile the expression (this removes the macro calls,
> >   which can only come from the allowed safe macros, and therefore
> >   cannot execute anything unsafe during macro expansion).
> >
> > 4- Walk again the resulting form to check that all the remaining
> >   special operators and functions are in the safe list.
> >
> > 5- Eventually, you can either call EVAL, or run your own compiler or
> >   interpreter.
> 
> Better to run a special interpreter/compiler that runs the code
> in some 'safe' environment (protected against taking all cpu time,
> using all file handles, allocating all memory, ...).

Yes.


> > (Don't forget to handle all the conditions).
> >
> > CL:+     could be added to the safe operator list.
> > CL:OPEN  shouldn't be added to the safe operator list;
> >         you could have a SAFE-VIRTUAL-CL:OPEN instead.
> > etc...
> 
> Executing Lisp expressions typed by a remote user in a server Lisp is  
> not a good idea:
> 
> (loop)  will use 100% CPU and run until the thing gets killed.

So you don't put LOOP in the safe list.
 

> Something like
> 
> (loop (make-thread)) will allocate all threads

So you don't put MAKE-THREAD in the safe list. 


> (loop collect (make-array 100)) will allocate all memory.
> 
> (loop (write-string "foo0000" file-stream)) will keep the disk busy  
> until some file gets too large.
> 
> (loop (open-file ...)) will get all file handles ...

I already mentionned CL:OPEN above.


> Having declarations and optimize safety 0 will create unsafe code that  
> can crash the Lisp.

Yes, in addition to mere code, we have also to filter declarations.


I only presented the alternatives.  I agree with you that soon enough,
it becomes easier to control things when you implement your own
interpreter/compiler layer.  

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

CONSUMER NOTICE: Because of the "uncertainty principle," it is
impossible for the consumer to simultaneously know both the precise
location and velocity of this product.

Re: Security concerns in Common Lisp

Daniel Weinreb writes:
> 
> 
> 
> Jeff Massung wrote:
> >
> > Let's say I created a simple web server with LispWorks that had a
> > simple edit box that allowed a visitor to play with Lisp, enter an
> > expression, click submit, and see the results. What prevents the user
> > from entering something akin to (run-shell-command "format c:")?
>
> Nothing, really.  Such a server is inherently highly unsafe.  I know
> of no Common Lisp implementation that attempts to implement a
> "sandbox" of the sort that Java attempts to implement.  You have to
> limit its privileges using the underlying operating system's
> facilities.

Well indeed, one good way would be to move the border further out. Use
a virtual unix machine, such as a user-mode-linux VM or a xen VM,
configured to avoid leaking network priviledges, and run the lisp
inside it.  Configure it to allocate it a bound CPU time percentage.
When a remote user connects, make a copy (or CoW) of the image, and
then the remote user may do whatever evil it wants, it'll stay inside
the VM.  When he's done, delete the image.



> > But, what if it was just an end application reading a configuration
> > file with Lisp data for ease or (as in my case) a server/client
> > sending Lisp data back and forth across the internet, where both users
> > are most likely running as superusers (yay for Windows)?
>
> Ah.  First, it's really not a great idea to design your system so that your
> configuration files are Lisp forms to be evaluated.  First, it's just too
> easy for mistakes to have arbitrary, bad effects.  Second, full Lisp
> is usually "too powerful", in the sense that a simple declarative
> format can be parsed easily by a program, whereas a Lisp file
> might have arbitrarily complex Lisp forms that are hard for
> a program to understand. 

I don't agree. '.bashrc' and '.emacs' are two examples showing that
it's good to have executable configuration, (not to mention ~/.sbclrc
and such).


> Yes, there are good reasons for
> programs that can read and understand config files.  You may not
> have one now, but you should design your architecture to allow
> for them in the future, IMHO based on years of experience.
> 
> If you really are going to use Lisp, but not Lisp forms,
> then you'll find yourself using the Lisp "read" but not
> the Lisp "eval".  That's a lot safer, but there is one
> hazard that you need to know about: there is a reader
> macro, #., that means "read one form and evaluate it".

Of course, if the application itself is not "an emacs", that is, if is
not mainly a programming environment in which the application happens
to _be_ developed, there's little need for executable configuration
files.  But your application should probably be develoed as "an
emacs"!


Configuration files are (or should be) entirely controled by the user,
the same user who can delete his own files himself already, so you're
gaining nothing preventing him to delete his files from a
configuration file.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

HANDLE WITH EXTREME CARE: This product contains minute electrically
charged particles moving at velocities in excess of five hundred
million miles per hour.

Re: Security concerns in Common Lisp

Unable to parse email body. Email id is 8687