You can support this project by donating at:

Or see the list of project sponsors.

trivial-mmapsystemposixtrivial

Documentation	🤨
Docstrings	🤨
Tests	🥺
Examples	🤨
RepositoryActivity	🥺s
CI	🥺

This library provides a few functions and macros to work with memory mapped files from Lisp.

Memory-mapped files are useful when you need to process a large amount of data which may not fit into the RAM. At the end I'll give a link to the article which more detail, but now let's see how it works on a small example.

The main entry point is the "with-mmap-file" it opens the file and maps its content into the memory.

Also, "trivial-mmap" provides few functions for sequential reads from the file by "char" or by "byte":

POFTHEDAY> (trivial-mmap:with-mmap-file (file "/tmp/the-file.dat")
             (loop for i upto 16
                   collect (trivial-mmap:mmap-read-char file)))
(#\H #\e #\l #\l #\o #\  #\L #\i #\s #\p #\  #\W #\o #\r #\l #\d #\!)
POFTHEDAY> (coerce * 'string)
"Hello Lisp World!"

Of cause, in this synthetic example we aren't having any profits from using mmap. To feel the difference we need to operate on gigabytes of data.

Usually, this technique is used by databases and image/video processing software.

When using mmap, you can access the content of the file like you do with a usual memory array. But in Common Lisp, you will need to use CFFI.

Here is a function which reads a "byte" from the memory-mapped file. It increments a pointer and fetches an unsigned byte from the array:

(defun %mmap-read-byte (pointer-to-mmap-file offset)
  "Reads and returns one byte from a memory-mapped file pointed to by
the POINTER-TO-MMAP-FILE pointer, offset by OFFSET bytes."
  (cffi:mem-aref (cffi:inc-pointer pointer-to-mmap-file offset) :uint8))

On top of this, you may implement your own data structures. By the way, for serialization/deserialization you may try cl-conspack, reviewed a week ago.

Finally, I want to give a link to the article describing when the memory-mapped files can be useful.