Kawa web collection

Project versioned at https://github.com/arvyy/kawa-web-collection

Compiled documentation can be viewed at https://arvyy.github.io/kawa-web-collection/README.html

If you want inspiration on how a real project might look like using this collection, checkout scheme index https://github.com/arvyy/r7rs-index-site

Goals and target audience

Kawa web collection is an attempt to take mature and feature-complete web libraries and tooling, put them in a minimal wrapper, and enable them to be used through vanilla R7RS syntax. All modules below are optional; you pay in final-artifact size cost only for what you choose to use.

If you want to create web applications using Scheme, and don’t mind learning maven, hopefully this collection will be just for you.

Prerequisites

You must have these tools installed (commonly available through package managers)

JDK 11 (test by running javac -version)
Maven 3 (test by running mvn -version)

You should eventually read up about maven in general (https://maven.apache.org/guides/getting-started/) and maven kawa plugin (https://github.com/arvyy/kawa-maven-plugin), however if you use project generation command below, it will output all command line commands you need to run and package the application.

Setting up

Run mvn install from the root of this repository to install collection in local maven repository, which will make individual modules available for local machine.

To generate a scaffold project, navigate to some directory, and execute

mvn archetype:generate "-DarchetypeGroupId=com.github.arvyy.kawa-web-collection" "-DarchetypeArtifactId=auth-archetype" "-DarchetypeVersion=0.0.1"

When asked for groupId enter some domain you control, when asked for artifactId enter a short name for the project.

This will create a simple application, that includes RDBS based authorization system (including proper password salting and hashing), MPA web interface, loggin with Logback. To add other dependencies, use maven dependency block mentioned under each module, and paste it under <dependencies> tag. Likewise, if you don’t need some of the dependencies in the scaffold, remove corresponding <dependency> entry.

As mentioned, generated project will include a slim readme file, containing most important CLI commands to run and compile.

Modules

DB Utils

Module for interacting with relational databases through JDBC drivers. A thin wrapper of Apache DbUtils.

Maven dependencies

To include module itself, add following dependency.

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>dbutils</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Note, however, this alone is not enough. As mentioned, this operates through JDBC, and therefore you’ll also have to include an appropriate JDBC driver for the kind of DB you plan using. For example, to use sqlite, you’d also add

<dependency>
    <groupId>org.xerial</groupId>
    <artifactId>sqlite-jdbc</artifactId>
    <version>3.32.3.2</version>
</dependency>

Exported procedures

Library `(arvyy dbutils)`

(make-query-runner driver url username password) → query-runner

constructs query runner with given connection details to a database. driver is a driver java class name, and will depend on database / JDBC (eg for sqlite this would be "org.sqlite.JDBC"). url is the jdbc url, which specifies database location and other properties. As with driver, also entirely depends on database (eg for sqlite "jdbc:sqlite:testdb"). username and password are credentials as strings, used for connection. If DB isn’t protected, these strings may be empty.

(query query-runner sql handler arg …) → query-result

execute a query (ie, select statement). query-runner is an object as constructed with make-query-runner. sql is SQL statement as a string. The statement may have placeholders in using ? (eg., select * from users where name = ?). During execution, each ? placeholder is replaced with positionally corresponding arg. handler is what handles database result and transforms it to the value, which is what ultimately gets returned from query call (see handler/* procedures below for handler construction).

(handler/scalar) → handler

creates a scalar handler, which expects for the query to have returned 1 row, and simply returns said row’s value of first column. If query had returned 0 or 2+ rows, a dbutils-error? is raised.

(handler/single-of row-handler) → handler

creates a handler, which expects for the query to have returned 1 row, and maps the row in accordance to row-handler before returning (see row-handler/* procedures below for row-handler construction).

(handler/list-of row-handler) → handler

creates a handler which maps the rows in accordance to row-handler, and returns them as a list (see row-handler/* procedures below for row-handler construction).

(handler/generator-of row-handler consumer) → handler

creates a handler which defers reading of the rows and creates a generator (ie, thunk, which when called repeatedly keeps returning next value, or eof-object if the values were exhausted), with generator producing values in accordance to row-handler. The result of query using this handler is (consumer generator). (see row-handler/* procedures below for row-handler construction).

(row-handler/vector) → row-handler

creates a row handler, which reads a row into a vector.

(row-handler/alist) → row-handler

creates a row handler, which reads a row into an alist, using symbol to represent column name.

(update query-runner sql arg …) → count

executes an update (either DML, or a DDL statement), and returns a number of affected rows. query-runner is an object as constructed with make-query-runner. sql is SQL statement as a string. The statement may have placeholders in using ? (eg., update users set name = ? where name = ?). During execution, each ? placeholder is replaced with positionally corresponding arg.

(insert query-runner sql arg …) → new-id

same as update, except only usable with sql insert statement, and returned value corresponds to inserted row’s generated id if the table has autoincrement.

(call-in-transaction query-runner use-existing? thunk)

invokes thunk and for its duration executes SQL statements inside a single transaction, and returns whatever thunk returns. query-runner is an object as constructed with make-query-runner. use-existing? specifies what do in the case when current context is already in transaction - if #f, a new transaction is started. When using call-in-transaction, it’s users responsibility to commit (or rollback) for the changes to take place, see commit and rollback below. If an uncaught exception gets raised by thunk, transaction is automatically rolled back.

(in-transaction?) → boolean

returns if we’re in a transaction context started by a call-in-transaction.

(commit)

(rollback)

Commit / rollback current transaction. Raises an error if in-transaction? is false.

(dbutils-error? obj) → boolean

return whether or not given object is an error caused by DB interaction.

(dbutils-error-message err) → string

returns error’s message.

(dbutils-error-statement sql) → string

returns sql statement, execution of which caused the error

Interface abstraction for a set of functions

This library provides a way to define an interface (a set of procedures with defined names and parameter count), define interface’s implementation(s), and call interface method with specific implementation reference.

While generally interface abstraction is less needed compared to statically typed and / or object oriented languages, it can be handy in certain contexts, such as when implementing "Ports and Adapters" (also known as "Hexagon") architecture, which was the main motivation for the creation of this library.

Maven dependencies

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>interface</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Exported procedures

Library `(arvyy interface)`

Interface is defined with

(define-interface
  interface-implementation-constructor
  method-signature ...)

Where method-signature is either

(method-name arg …)
(method-name arg … . rest)

For example,

(define-interface
  foo
  (bar arg1 arg2))

interface-implementation-constructor is bound to syntax for constructing implementation of this interface, which shall be used so

(interface-implementation-constructor
    (method-signature method-body) ...)

For example (using foo interface from above),

(define foo-impl
  (foo
    ((bar arg1 arg2) (list arg1 arg2))))

Finally, to invoke interface’s method, call it by supplying implementation instance as a first parameter

For example,

(display (bar foo-impl 1 2)) ;; should show "(1 2)"

During construction of implementation instance, the syntax verifies that

implementation provides same amount of methods, in same order, with same method names (parameter names are not checked)
each method provides call-compatible amount of arguments. Implementation method cannot declare more required arguments than there were in interface declaration. The method may declare less required arguments if it also provides a rest argument.

Mustache templating

This library implements version 1.2.1 of the mustache spec. For general semantics, see https://mustache.github.io/mustache.5.html

Maven dependencies

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>mustache</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Base usage

(import (arvyy mustache))

First, templates need to be compiled, using one of following ways.

(compile root partial-locator). Here root should be a string of the "root" partial which will be the entry point during execution. partial-locator must be a procedure, that accepts partial’s name as a single argument, and returns either #f if it cannot be found, or string or textual input port.

(define (locator name)
  (cond
    ((string=? name "root") "Hello {{>foo}}")
    ((string=? name "foo") "{{world}}")
    (else #f))
(define hello-world-compiled (compile "root" locator))

(compile template). Compiles given template string.

(define hello-world-compiled (compile "Hello {{world}}"))

Compiled template can be executed using

(execute compilation data out). compilation is result of compile invokation; data is the root object for interpolations; out is a textual output port, to which result will be writen. Returns unspecified value

(call-with-output-file "result.txt"
                       (lambda (out)
                         (execute hello-world-compiled
                                  '((world . "Scheme"))
                                   out)))

(execute compilation data). Instead of writing to a port, returns interpolated string result as a function value.

(display (execute hello-world-compiled '((world . "Scheme"))))

By default, vectors and streams are accepted for list interpolation, alists for object field lookup, data values are writen as with display.

Behavior customization

Behavior can be altered by use of parameterization.

Field lookup in object

Lookup is a procedure, that finds corresponding value given an object and a fragment (fragment being elements of list after splitting tag name by .), of the following form

(lookup object name found not-found). Object is currently examined datum; name is a fragment / field of string type; found is a function that should be invoked in tail position if field was found in the object with corresponding value as an argument; not-found is a 0 argument function that should be invoked in tail position if field was not found in the object, or if the object cannot be inspected with this lookup function.

For example, creating a lookup for a record type:

(define-record-type <foo> (foo bar) foo? (bar foo-bar))
(define (foo-lookup obj name found not-found)
  (cond
    ((not (foo? obj)) (not-found))
    ((string=? "bar" name) (found (foo-bar obj)))
    (else (not-found))))

Use compose-lookups to merge multiple lookup implementations into one:

(define alist+foo (compose-lookups alist-lookup foo-lookup))

When composed, each lookup is tried to be applied to the object until one of lookups invokes found, in given order. If all lookups return not-found, then composition also returns not-found

The lookup used during execution is retrieved from current-lookup parameter. It defaults to alist-lookup.

Collection

Collection is a reference to a set of methods of an iterable multivalue object type, which is used to expand sections. Collection is created with

(collection pred?-proc empty?-proc for-each-proc) where pred?-proc is a predicate for given collection, empty?-proc is procedure returning if given collection is empty, for-each-proc is a procedure that given a one argument function and this collection, executes given function for each element.

(define list-collection (collection list? null? for-each))

Use compose-collections to merge multiple collections into one

(define vec+list (compose-collections vector-collection list-collection))

The collection used during execution is retrieved from current-collection parameter. It defaults to (compose-collections vector-collection stream-collection). While the library does export list-collection, it is not used by default as to not clash with alist lookup for objects.

Value writing

Writer is a procedure of form

(writer obj out) where obj is a scheme object, and out is textual output port. Writer should write appropriate representation of obj to out.

The writer used during execution is retrieved from current-writer parameter. It defaults to (lambda (obj out) (when obj (display obj out))).

SLF4J logging API

Module for using logging APIs.

Maven dependencies

To include module itself, add following dependency.

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>slf4j</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Note, however, this alone is not enough. SLF4J is an API, it’s a frontend providing endpoints to log things, however you’d also need to include one of loggers that provide SLF4J implementation. For development / tests, you can use simple implementation

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>1.7.30</version>
</dependency>

For production uses however, you should look into using log4j or logback implementations.

Exported procedures

Library `(arvyy slf4j)`

(get-logger name) → logger

create a logger by name.

(logger-name logger) → name

return logger’s name. logger is a logger constructed with get-logger.

(log-trace logger message arg …)

(log-debug logger message arg …)

(log-info logger message arg …)

(log-warn logger message arg …)

(log-error logger message arg …)

log a message at appropriate level. logger is a logger constructed with get-logger. message is a string to be logged. The string may have placeholders {} which would be substituted by arg value. The last arg which isn’t matching a {} may be an error object (as caught by with-excption-handler). Note that these are macros - each arg may be a resource intensive expression, which will only be evaluated if appropriate logging level is enabled.

(trace-enabled? logger) → boolean

(debug-enabled? logger) → boolean

(info-enabled? logger) → boolean

(warn-enabled? logger) → boolean

(error-enabled? logger) → boolean

determine if specified logging level is enabled. logger is a logger constructed with get-logger.

Solrj

Module for interacting with Solr. A thin wrapper of Solrj. If you wish to use embedded solr, add solr-embedded module.

Maven dependencies

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>solrj</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Exported procedures

Library `(arvyy solrj)`

(create-http-solr-client url) → solr-client

create a client, that connects to solr server running at given url.

commit-within

a parameter (used through parameterize form) controlling commitWithin solr parameter when calling add, query, delete-by-query, delete-by-id. See https://solr.apache.org/guide/6_6/updatehandlers-in-solrconfig.html#UpdateHandlersinSolrConfig-commitWithin .

(commit core)

(rollback core)

force commit / rollback for a given core. core is a core name as a string.

(add solr-client core docs)

Add documents to solr index. core is a core name as a string. docs is a vector of documents, where each document is an alist using symbol as a key.

(query solr-client core handler params) → solr-result

Execute solr query, returning json response in sexpr form as defined in srfi 180. core is a core name as a string. handler is request handler name as a string. params is an alist of parameters to be passed to query, using symbols as keys, and strings, integers, or vector of the later as values.

(delete-by-query solr-client core query)

Delete solr documents matching the query. core is a core name as a string. query is a string of lucene query syntax, defining what to delete.

(delete-by-id solr-client core ids)

Delete solr documents with given ids. core is a core name as a string. ids is a vector of strings of documents to delete.

Solr embedded

Module for embeddeding Solr. For interaction with with it, see solrj module.

Maven dependencies

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>solr-embedded</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Exported procedures

Library `(arvyy solrj)`

(create-embedded-solr-client solrhome default-core) → solr-client

create a client that wraps embedded lucene index. solrhome is path as a string to solr home folder containing solr configuration. default-core specifies core name that acts as a default.

Spark web server

Module provides http server functionality. Thin wrapper over sparkjava.

Maven dependencies

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>spark</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Exported procedures

Library `(arvyy kawa-spark)`

In all occurances below, route corresponds to a procedure parameter, that takes two arguments - request and response objects - and returns a value to be used in response. In all occurances below, filter corresponds to a procedure parameter, that takes two arguments - request and response objects - interacts with them in some way, but returns unspecified value. In all occurances below, req corresponds to a request object, provided as the first argument to route or filter. In all occurances below, resp corresponds to a response object, provided as the second argument to route or filter. In all occurances below, session corresponds to a session object, retrieved through req/session / req/create-session!.

(get path route)

(post path route)

(put path route)

(delete path route)

(options path route)

Add a controller mapping for a GET / POST / PUT / DELETE / OPTIONS method. path is a path as a string.

(before-all filter)

(after-all filter)

(after-after-all filter)

Execute filter before / after / very-after ("finally" block) for any request.

(before path filter)

(after path filter)

(after-after path filter)

Execute filter before / after / very-after ("finally" block) for requests to a given path. path is a path as a string.

(redirect/get from to)

(redirect/post from to)

(redirect from to)

Redirect GET / POST / any requests coming through from path towards to. from and to are paths as strings.

(path path body …)

Mappings defined in body will have path prefix added to them. path is path as a string. body … are statements to be wrapped in a lambda.

(path "rest"
    (get "foo" (lambda (req resp) ;; maps to /rest/foo
                  ...)))

(req/attributes req)

Get request attributes as list of strings.

(req/attribute req attr)

Get attribute value. attr is attribute name as a string.

(req/set-attribute! req attr value)

Set attribute value. attr is attribute name as a string. value is any value.

(req/body req)

Get request body as a string.

(req/body-as-bytes req)

Get request body as a vector of bytes.

(req/content-length req)

Get request length in bytes as an integer.

(req/content-type req)

Get request content-type as a string.

(req/context-path req)

Get request context-path as a string.

(req/cookies req)

Get request cookies as an alist.

(req/cookie req name)

Get a cookie by name as a string.

(req/headers req)

Get a list of header names present in request.

(req/header req name)

Get value for the provided header as a string.

(req/host req)

Get host as a string.

(req/ip req)

Get IP as a string.

(req/params req)

Get params of the request as an alist. Param here refers to path variables encoded in the form of :name, for example

;;  matches "GET /rest/users/u1"
(get "/rest/users/:id"
     (lambda (req resp)
       (define p (req/params req)) ;; ((":id" . "u1"))
       ...
     ))

(req/param req name)

Get param of the request for given name as a string.

(req/path-info req)

Get path info as a string.

(req/port req)

Get port as an integer.

(req/protocol req)

Get protocol as a string.

(req/query-string req)

Get a query string (ie., the bit after ?) as a string.

(req/query-params req)

Get query param names present in request as a list of strings.

(req/query-param req name)

Get first value for a given query parameter as a string.

(req/query-param-values req)

Get all values for a given query parameter (ie., when parameter is repeated multiple times) as a list of strings.

(req/request-method req)

Get HTTP method as a string.

(req/scheme req)

Get scheme as a string.

(req/session req)

Get session object, if session had been created. Returns #f if it wasn’t.

(req/create-session! req)

Get session object; creates the session if it didn’t exist.

(req/splat req)

Get splat of the request as a list of strings. Splat here refers to path variables encoded in the form of *, for example

;;  matches "GET /rest/foo/bar/baz"
(get "/rest/*/bar/*"
     (lambda (req resp)
       (define s (req/splat req)) ;; ("foo" "baz")
       ...
     ))

(req/uri req)

Get request URI as a string.

(req/url req)

Get request URL as a string.

(req/user-agent req)

Get user agent as a string.

(resp/body resp)

Get response body as string.

(resp/set-body! resp body)

Set response body as string.

(resp/set-header! resp name value)

Set response header.

(resp/redirect resp location)

Mark response to do a redirect to a location.

(resp/status resp)

Get response status code as an integer.

(resp/set-status! resp status-code)

Set response status code as an integer.

(resp/type resp)

Get response content type as a string.

(resp/set-type! resp type)

Set response content type as a string.

(resp/set-cookie! resp name value)

Set cookie for the response.

(resp/remove-cookie! resp name)

Remove cookie by name from response.

(session/attribute session name)

Get attribute value from session by name.

(session/set-attribute! session name value)

Set session attribute value.

(session/remove-attribute! session name)

Remove session attribute.

(session/attributes session)

Get attribute names present in session as a list of strings.

(session/id session)

Get session unique id as a string.

(session/new? session)

Returns true if the session hadn’t been communicated with client yet

(halt! code message)

Halt current request with given code status code as an integer, message as a string.

(not-found route)

Handle request which didn’t match any handler.

(internal-server-error route)

Handle request which caused server-side error.

(exception handler)

Handle request which raised an exception. handler is like route, except it has an extra parameter in first position exception, corresponding to the raised object.

(static-files/location folder)

Enable serving static files from folder as a string relative to classpath (ie, inside src/main/resources).

(static-files/external-location folder)

Enable serving static files from file system from folder location as a string.

(static-files/expire-time seconds)

Set expire timing for static resources in seconds as an integer.

(static-files/header key value)

Append given header when serving static resources.

(init)

Manually start the server (The server is automatically started when you do something that requires the server to be started, and thus usually isn’t needed).

(stop)

Stop the server.

(port value)

Set port on which to run server. Must be done before declaring routes and filters. Defaults to 4567.

(secure keystoreFilePath keystorePassword truststoreFilePath truststorePassword)

Secure server with SSL. Must be done before declaring routes and filters.

(thread-pool count)

Configure threadpool specifying the amount of threads.

(await-initialization)

Block until server is ready

Password encoder

Module to cryptographically encode and compare passwords or other sensitive data. Thin wrapper over spring security crypto library.

Maven dependencies

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>spring-password-encoder</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Exported procedures

Library `(arvyy spring-password-encoder)`

(encode-password password-encoder password) → encoded-password

encodes given password and returns it as a string. password-encoder is an encoder instantiated with one of make-*-password-encoder procedures below. password is a password as a string.

Note that generally speaking, encode-password isn’t referentially transparent and may return different results with same inputs (due to generating and using a different salt). Therefore this procedure is only useable for encoding, but not comparing. See passwords-match for validating user supplied password with stored password.

(passwords-match password-encoder raw-password encoded-password) → matches?

compares given raw password with an encoded password, and returns whether they match as a boolean. password-encoder is an encoder instantiated with one of make-*-password-encoder procedures below. raw-password is a raw password as a string. encoded-password is a previously encoded password by an encoder as a string.

(make-pbkdf2-password-encoder) → password-encoder

(make-pbkdf2-password-encoder secret) → password-encoder

(make-pbkdf2-password-encoder secret salt-length) → password-encoder

(make-pbkdf2-password-encoder secret iterations hash-width) → password-encoder

(make-pbkdf2-password-encoder secret salt-length iterations hash-width) → password-encoder

Constructs pbkdf2 based password encoder.

(make-bcrypt-password-encoder [strength]) → password-encoder

Constructs bcrypt based password encoder.

(make-scrypt-password-encoder [cpu-cost mem-cost parallelization key-length salt-length]) → password-encoder

Constructs scrypt based password encoder.

(make-argon2-password-encoder [salt-length hash-length parallelization memory iterations]) → password-encoder

Constructs argon2 based password encoder.

(make-custom-password-encoder encode-proc matches-proc) → password-encoder

Constructs a custom password encoder. encoder-proc must be a procedure that takes raw password as a string and returns encoded password as a string. matches-proc must be a procedure that takes raw password as a string, encoded password as a string, and returns boolean whether or not passwords match.

SRFI 180 (json)

Plain SRFI 180 reference implementation for using json.

Maven dependencies

<dependency>
    <groupId>com.github.arvyy.kawa-web-collection</groupId>
    <artifactId>srfi-180</artifactId>
    <packaging>kawalib</packaging>
    <version>0.0.1</version>
</dependency>

Exported procedures

For details, please see https://srfi.schemers.org/srfi-180/srfi-180.html .

Wishlist

These are some of the modules that will hopefuly find their way to be included into this collection with time:

REST client
JWT library
Email sender
Excel (Apache POI) integration

Kawa web collection

Goals and target audience

Prerequisites

Setting up

Modules

DB Utils

Maven dependencies

Exported procedures

Library (arvyy dbutils)

Interface abstraction for a set of functions

Maven dependencies

Exported procedures

Library (arvyy interface)

Mustache templating

Maven dependencies

Base usage

Behavior customization

Field lookup in object

Collection

Value writing

SLF4J logging API

Maven dependencies

Exported procedures

Library (arvyy slf4j)

Solrj

Maven dependencies

Exported procedures

Library (arvyy solrj)

Solr embedded

Maven dependencies

Exported procedures

Library (arvyy solrj)

Spark web server

Maven dependencies

Exported procedures

Library (arvyy kawa-spark)

Password encoder

Maven dependencies

Exported procedures

Library (arvyy spring-password-encoder)

SRFI 180 (json)

Maven dependencies

Exported procedures

Wishlist

Library `(arvyy dbutils)`

Library `(arvyy interface)`

Library `(arvyy slf4j)`

Library `(arvyy solrj)`

Library `(arvyy solrj)`

Library `(arvyy kawa-spark)`

Library `(arvyy spring-password-encoder)`