OpenRPC for Common Lisp

This framework is built on top of JSON-RPC and Clack. Comparing to JSON-RPC library, it provides these key features:

• Automatic OpenRPC spec generation.

• Automatic JSON-RPC client building by OpenRPC spec. This includes creation of Common Lisp classes and methods for making RPC requests and returning native CL objects.

• On both server and client sides your code looks very lispy, all JSON marshalling is done under the hood.

Server

OPENRPC-SERVER ASDF System Details

• Description: OpenRPC server implementation for Common Lisp.

• Licence: BSD

• Author: Alexander Artemenko

• Homepage: https://40ants.com/openrpc/

• Source control: GIT

• Depends on: alexandria, clack-cors, clack-prometheus, closer-mop, jsonrpc, lack-request, lambda-fiddle, local-time, log4cl, log4cl-extras, serapeum, str, websocket-driver, yason

Defining Methods

Here is an example of openrpc-server ASDF system allows to define JSON-RPC methods and data-structures they return.

Let's see how we can define an api for usual PetShop example.

Simple Example

First, we will operate on usual Common Lisp class:

(defclass pet ()
  ((id :initarg :id
       :type integer
       :reader pet-id)
   (name :initarg :name
         :type string
         :reader pet-name)
   (tag :initarg :tag
        :type string
        :reader pet-tag)))

Now we can define an RPC method to create a new pet:

(openrpc-server:define-rpc-method create-pet (name tag)
  (:summary "Short method docstring.")
  (:description "Lengthy description of the method.")
  (:param name string "Pet's name"
          :description "This is a long description of the parameter.")
  (:param tag string "Old param, don't use it anymore." :deprecated t)
  (:result pet)
  (let* ((new-id (get-new-id))
         (pet (make-instance 'pet
                             :id new-id
                             :name name
                             :tag tag)))
    (setf (gethash new-id *pets*)
          pet)
    pet))

Here we should explicitly specify type for each parameter and result's type.

Pay attention, the result type is PET. openrpc-server system takes care on serializing objects and you can retrieve an OpenRPC spec for any type, using type-to-schema generic-function:

CL-USER> (serapeum:toggle-pretty-print-hash-table)
T
CL-USER> (openrpc-server:type-to-schema 'pet)
(SERAPEUM:DICT
  "type" "object"
  "properties" (SERAPEUM:DICT
                 "id" (SERAPEUM:DICT
                        "type" "integer"
                       )
                 "name" (SERAPEUM:DICT
                          "type" "string"
                         )
                 "tag" (SERAPEUM:DICT
                         "type" "string"
                        )
                )
  "required" '("tag" "name" "id")
  "x-cl-class" "PET"
  "x-cl-package" "COMMON-LISP-USER"
 )

This method is used to render response requests to /openrpc.json handle of your api.

There is also a second generic-function which transform class instance into simple datastructures according to a scheme. For example, here is how we can serialize our pet:

CL-USER> (openrpc-server:transform-result
          (make-instance 'pet :name "Bobik"))
(SERAPEUM:DICT
  "name" "Bobik"
 ) 
CL-USER> (openrpc-server:transform-result
          (make-instance 'pet
                         :name "Bobik"
                         :tag "the dog"))
(SERAPEUM:DICT
  "name" "Bobik"
  "tag" "the dog"
 )

Returning Lists

To return result as a list of objects of some kind, use (:result (list-of pet)) form:

(openrpc-server:define-rpc-method list-pets ()
  (:result (list-of pet))
  (retrieve-all-pets))

Paginated Results

Sometimes your system might operate on a lot of objects and you don't want to return all of them at once. For this case, framework supports a keyset pagination. To use it, your method should accept LIMIT argument and PAGE-KEY argument. And if there are more results, than method should return as a second value the page key for retrieving the next page.

In this simplified example, we'll return (list 1 2 3) for the first page, (list 4 5 6) for the second and (list 7 8) for the third. Pay attention how VALUES form is used for first two pages but omitted for the third:

(openrpc-server:define-rpc-method list-pets (&key (limit 3) page-key)
  (:param limit integer)
  (:param page-key integer)
  (:result (paginated-list-of integer))

  (cond
    ((null page-key)
     (values (list 1 2 3)
             3))
    ((= page-key 3)
     (values (list 4 5 6)
             6))
    (t
      (list 7 8))))

Of cause, in the real world application, you should use PAGE-KEY and LIMIT arguments in the WHERE SQL clause.

Using Clack to Start Server

Framework is based on Clack. Use make-clack-app to create an application suitable for serving with CLACK:CLACKUP.

Then just start the web application as usual.

(clack:clackup (make-clack-app)
               :address interface
               :port port)

Also, you might use any Lack middlewares. For example, here is how "mount" middleware can be used to make api work on /api/ URL path, while the main application is working on other URL paths:

(defparameter *app*
  (lambda (env)
    '(200 (:content-type "text/plain") ("Hello, World!"))))

(clack:clackup
 (lambda (app)
   (funcall (lack.util:find-middleware :mount)
            app
            "/api"
            (make-clack-app)))
 *app*)

OpenRPC Spec

The key feature of the framework, is an automatic OpenRPC spec generation.

When you have your api up and running, spec will be available on /openrpc.json path. For our example project it will looks like:

{
  "methods": [
    {
      "name": "rpc.discover",
      "params": [],
      "result": {
        "name": "OpenRPC Schema",
        "schema": {
          "$ref": "https://raw.githubusercontent.com/open-rpc/meta-schema/master/schema.json"
        }
      }
    },
    {
      "name": "list-pets",
      "params": [
        {
          "name": "page-key",
          "schema": {
            "type": "integer"
          }
        },
        {
          "name": "limit",
          "schema": {
            "type": "integer"
          }
        }
      ],
      "result": {
        "name": "list-pets-result",
        "schema": {
          "type": "object",
          "properties": {
            "items": {
              "type": "array",
...

API

Client

OPENRPC-CLIENT ASDF System Details

• Description: OpenRPC client implementation for Common Lisp.

• Licence: BSD

• Author: Alexander Artemenko

• Homepage: https://40ants.com/openrpc/

• Source control: GIT

• Depends on: alexandria, dexador, jsonrpc, kebab, log4cl, serapeum, str, usocket, yason

openrpc-client ASDF system provides a way to build CL classes and methods for working with JSON-RPC API. All you need is to give it an URL and all code will be created in compile-time as a result of macro-expansion.

Generating

For example, this macro call:

(generate-client petshop
                 "http://localhost:8000/openrpc.json")

Will generate the whole bunch of classes and methods:

(defclass petshop (jsonrpc/class:client) nil)

(defun make-petshop () (make-instance 'petshop))

(defmethod describe-object ((openrpc-client/core::client petshop) stream)
  (format stream "Supported RPC methods:~2%")
  (format stream "- ~S~%" '(rpc-discover))
  (format stream "- ~S~%"
          '(list-pets &key (page-key nil page-key-given-p)
            (limit nil limit-given-p)))
  (format stream "- ~S~%" '(create-pet (name string) (tag string)))
  (format stream "- ~S~%" '(get-pet (id integer))))

(defclass pet nil
  ((id :initform nil :initarg :id :reader pet-id)
   (name :initform nil :initarg :name :reader pet-name)
   (tag :initform nil :initarg :tag :reader pet-tag)))

(defmethod print-object ((openrpc-client/core::obj pet) stream)
  (print-unreadable-object (openrpc-client/core::obj stream :type t)
    (format stream " ~A=~S" 'id (pet-id openrpc-client/core::obj))
    (format stream " ~A=~S" 'name (pet-name openrpc-client/core::obj))
    (format stream " ~A=~S" 'tag (pet-tag openrpc-client/core::obj))))

(defmethod rpc-discover ((openrpc-client/core::client petshop))
  ...)

(defmethod list-pets
    ((openrpc-client/core::client petshop)
     &key (page-key nil page-key-given-p) (limit nil limit-given-p))
  ...)

(defmethod create-pet
    ((openrpc-client/core::client petshop) (name string) (tag string))
  ...)

(defmethod get-pet ((openrpc-client/core::client petshop) (id integer))
  ...)

Using

When client is generated, you need to make an instance of it and to connect it to the server:

(let ((cl (make-petshop)))
    (jsonrpc:client-connect cl :url "http://localhost:8000/" :mode :http)
    cl)

You can use any transport, supported by JSONRPC library.

DESCRIBE-OBJECT method is defined for a client, so you might see which methods are supported right in the REPL:

OPENRPC-EXAMPLE/CLIENT> (defvar *client* (make-test-client))
#<PETSHOP {1007AB2B13}>

OPENRPC-EXAMPLE/CLIENT> (describe *client*)
Supported RPC methods:

- (RPC-DISCOVER)
- (LIST-PETS &KEY (PAGE-KEY NIL PAGE-KEY-GIVEN-P)
             (LIMIT NIL LIMIT-GIVEN-P))
- (CREATE-PET (NAME STRING) (TAG STRING))
- (GET-PET (ID INTEGER))

And then to call these methods as usually you do in Common Lisp. Pay attention, that the library returns not JSON dictionaries, but ready to use CL class instances:

OPENRPC-EXAMPLE/CLIENT> (create-pet *client* "Bobik" "the dog")
#<PET  ID=1 NAME="Bobik" TAG="the dog">

OPENRPC-EXAMPLE/CLIENT> (create-pet *client* "Murzik" "the cat")
#<PET  ID=2 NAME="Murzik" TAG="the cat">

OPENRPC-EXAMPLE/CLIENT> (create-pet *client* "Homa" "the hamster")
#<PET  ID=3 NAME="Homa" TAG="the hamster">

Now, pay attention how pagination does work.

OPENRPC-EXAMPLE/CLIENT> (list-pets *client* :limit 2)
(#<PET  ID=1 NAME="Bobik" TAG="the dog">
 #<PET  ID=2 NAME="Murzik" TAG="the cat">)
#<FUNCTION (FLET OPENRPC-CLIENT/CORE::RETRIEVE-NEXT-PAGE :IN LIST-PETS) {1006D1F3CB}>

This call has returned a list of objects as the first value and a closure, which can be called to retrive the next page. Let's retrieve it now!

OPENRPC-EXAMPLE/CLIENT> (funcall #v167:1)
(#<PET  ID=3 NAME="Homa" TAG="the hamster">)

Now this is the last page and there is now a closure to retrieve the next page. Learn more how to implement pagination on server-side in the Paginated Results section.

Our Ask...

If you use this or find value in it, please consider contributing in one or more of the following ways:

1. Sponsor project at Patreon or Boosty and make a contribution.

2. Star it!

3. Share posts about it in social networks!

4. Fix an issue.

5. Add a feature (post a proposal in an issue first!).

Contributors

These people have contributed to OpenRPC. I'm so grateful to them!

• Kilian M. Haemmerle