Refactor API servers into one modular server

.dockerignore

@@ -0,0 +1,2 @@
+**/node_modules/
+**/package-lock.json

.gitignore

@@ -1,6 +1,7 @@
 .*
 config/docker-compose.prod.yaml
 config/docker-compose.prod.yml
+kansa-common*.tgz
 node_modules/
 package-lock.json
 

.travis.yml

@@ -8,6 +8,7 @@ node_js:
   - 'node'
 
 before_install:
+  - mv common/$(cd common && npm pack) server/kansa-common.tgz
   - docker-compose
     -f config/docker-compose.base.yaml
     -f config/docker-compose.dev.yaml

Dockerfile

@@ -0,0 +1,24 @@
+FROM node:10
+
+ARG NODE_ENV
+ENV NODE_ENV $NODE_ENV
+
+WORKDIR /kansa
+COPY server/package.json .
+COPY common/package.json ./common/
+RUN npm install && \
+    npm install ./common && \
+    npm cache clean --force
+
+COPY modules/raami/package.json modules/raami/
+RUN cd modules/raami && npm install && npm cache clean --force
+
+COPY modules/slack/package.json modules/slack/
+RUN cd modules/slack && npm install && npm cache clean --force
+
+COPY common ./common
+COPY modules ./modules
+COPY server .
+
+EXPOSE 80
+CMD [ "npm", "start" ]

README.md

@@ -5,24 +5,30 @@
 <h1>Kansa</h1>
 </div>
 
-Kansa is a convention member management system originally developed for [Worldcon 75](http://www.worldcon.fi),
+Kansa is a convention member management system originally developed for [Worldcon 75],
 the World Science Fiction Convention organised in Helsinki in 2017. It is also used by
-[Dublin 2019: An Irish Worldcon](https://dublin2019.com/).
+[Dublin 2019: An Irish Worldcon] and [CoNZealand], the 2020 Worldcon.
 
-The system is modular and extensible. Together with its
-[front-end client](https://github.com/maailma/kansa-client) it provides the following services:
+The system is modular and extensible. Together with its [front-end client] it provides
+the following services:
 
 - Member admin services, including an easy-to use admin front-end
 - Support for multiple membership types, as well as for non-member accounts
-- [Stripe](https://stripe.com/) integration for membership and other purchases (via credit cards or
+- [Stripe] integration for membership and other purchases (via credit cards or
   SEPA direct debit)
 - Individual and bulk import of member data and transactions from other systems
 - Member-facing front-end for e.g. name and address changes
 - Passwordless authentication using login links sent by email
-- Emails sent using [Sendgrid](https://sendgrid.com/) and customisable
-  [templates](config/message-templates/)
+- Emails sent using [Sendgrid] and customisable [templates](config/message-templates/)
 - Synchronisation of contact info to Sendgrid for mass mailings
 
+[worldcon 75]: http://www.worldcon.fi
+[dublin 2019: an irish worldcon]: https://dublin2019.com/
+[conzealand]: https://conzealand.nz/
+[front-end client]: https://github.com/maailma/kansa-client
+[stripe]: https://stripe.com/
+[sendgrid]: https://sendgrid.com/
+
 To help with at-con registration, Kansa has:
 
 - Pre-con badge preview and customisation (with Unicode support)
@@ -101,21 +107,26 @@ Email messages are based on message templates, which are
 
 ### Directory Overview
 
-- **`config`** - System configuration
-- **`hugo`** - Provides the Hugo Nominations and Awards parts of the [REST API](docs/index.md)
-- **`integration-tests`** - Tests for the REST API, targeting the Stripe and Sendgrid interfaces in particular
-- **`kansa`** - Provides the core parts of the [REST API](docs/index.md)
-- **`kyyhky`** - Internal mailing service & [SendGrid](https://sendgrid.com/) integration for hugo & kansa
-- **`nginx`** - An SSL-terminating reverse proxy & file server, using [OpenResty](https://openresty.org/)
+- **`common`** - Shared utilities for the server & modules, published on npm as [`@kansa/common`][kc]
+- **`config`** - System configuration, see in particular [`config/kansa.yaml`](config/kansa.yaml)
+- **`integration-tests`** - Tests for the REST API endpoints
+- **`kyyhky`** - Internal mailing service & [SendGrid] integration for hugo & kansa
+- **`modules`** - Optional server modules providing additional functionality
 - **`postgres`** - Configuration & schemas for our database
-- **`raami`** - Art show management [REST API](docs/raami.md)
+- **`proxy`** - An SSL-terminating reverse proxy & file server, using [OpenResty]
+- **`server`** - Provides the core parts of the [REST API](docs/index.md)
 - **`tools`** - Semi-automated tools for importing data, and for other tasks
 - **`tuohi`** - Fills out a PDF form, for `GET /people/:id/ballot`
 
-[Kansa](https://en.wiktionary.org/wiki/kansa#Finnish) is Finnish for "people" or "tribe", and it's
-the name for our member registry. The [Hugo Awards](http://www.thehugoawards.org/) are awards that
-are nominated and selected by the members of each year's Worldcon. Kyyhky is Finnish for "pigeon",
-Raami is "frame", and Tuohi is the bark of a birch tree.
+The [Hugo Awards] are awards that are nominated and selected by the members of each year's Worldcon.
+[Kansa] is Finnish for "people" or "tribe", Kyyhky is "pigeon", Raami is "frame", and Tuohi is the
+bark of a birch tree.
+
+[kc]: https://www.npmjs.com/package/@kansa/common
+[sendgrid]: https://sendgrid.com/
+[openresty]: https://openresty.org/
+[hugo awards]: http://www.thehugoawards.org/
+[kansa]: https://en.wiktionary.org/wiki/kansa#Finnish
 
 ### Common Issues
 
@@ -125,7 +136,7 @@ The particular places that may need manual adjustment are:
   [self-signed certificate](http://www.selfsignedcertificate.com/) for `localhost`. This will not
   be automatically accepted by browsers or other clients. If you have a signed certificate you can
   use (and therefore a publicly visible address), you'll want to add the certificate files to
-  `nginx/ssl/` and adjust the environment values set for the `nginx` service in
+  `proxy/ssl/` and adjust the environment values set for the `proxy` service in
   [docker-compose.override.yaml](config/docker-compose.override.yaml) and/or your
   `docker-compose.prod.yaml`.
 
@@ -134,7 +145,7 @@ The particular places that may need manual adjustment are:
   default, the value should match the `http://localhost:8080` address of the client Webpack dev
   servers.
 
-- If you're running the server on a separate machine or if you've changed the `nginx` port
+- If you're running the server on a separate machine or if you've changed the `proxy` port
   configuration, you may need to tell clients where to find the server, using something like
   `export API_HOST='remote.example.com'` before running `npm start`.
 

common/README.md

@@ -0,0 +1,123 @@
+# @kansa/common
+
+Common objects and utilities shared across the
+[Kansa](https://github.com/maailma/kansa) server & modules.
+
+## User authentication
+
+```js
+const { isSignedIn, hasRole, matchesId } = require('@kansa/common/auth-user')
+```
+
+### `function isSignedIn(req, res, next)`
+
+Express.js middleware function that verifies that the current user is signed in.
+On failure, calls `next(new AuthError())`.
+
+### `function hasRole(role: string | string[]): function(req, res, next)`
+
+Function returning an Express.js middleware function that verifies that the
+current user is signed in and has the specified `role` (or one of the roles, if
+an array). On failure, calls `next(new AuthError())`.
+
+### `function matchesId(db, req, role?: string | string[]): Promise<number>`
+
+Verifies that the `id` parameter of the request `req` grants access to the
+session user's `email`. If set, `role` may define one or multiple roles that the
+user may have that grant access without a matching `email`.
+
+`db` should be a `pg-promise` instance, or e.g. a `task` extended from such an
+instance. Returns a promise that either resolves to the `id` value, or rejects
+with either `AuthError`, `InputError`, or a database error.
+
+### `new AuthError(message: string)`
+
+## Configuration
+
+```js
+const config = require('@kansa/common/config')
+```
+
+The server configuration, sourced from `config/kansa.yaml` and read during
+server start.
+
+## Errors
+
+```js
+const { AuthError, InputError } = require('@kansa/common/errors')
+```
+
+### `new AuthError(message: string)`
+
+### `new InputError(message: string)`
+
+Handled by the server's error handling. May also have their `status` set.
+
+## Log entries
+
+```js
+const LogEntry = require('@kansa/common/log-entry')
+new LogEntry(req, 'Log message').write(db)
+```
+
+### `new LogEntry(req, message: string)`
+
+Creates a new log entry for tracking changes. The entry's fields such as
+`author`, `description` and `subject` are modifiable before the entry gets
+stored with its `write(db): Promise` method.
+
+## Mail
+
+### `function sendMail(type: string, data: any, delay?: number): Promise`
+
+Schedule an email message of `type` with `data` to be sent, with an optional
+`delay` in minutes. The message should have a correspondingly named template
+available under `config/message-templates/`.
+
+### `function updateMailRecipient(db, email: string): Promise`
+
+Using the `pg-promise` instance `db`, fetch the appropriate data for `email`
+and forward it to be sent to Sendgrid. Returns a promise that will never reject.
+
+## Split names
+
+```js
+const splitName = require('@kansa/common/split-name')
+
+splitName('Bob Tucker')
+// ['', 'Bob Tucker']
+
+splitName('Bob Tucker', 8)
+// ['Bob', 'Tucker']
+
+splitName('Arthur Wilson "Bob" Tucker')
+// [ 'Arthur Wilson', '"Bob" Tucker' ]
+```
+
+### `function splitName(name: string, maxLength = 14): [string, string]`
+
+Splits a name (or other string) prettily on two lines.
+
+If the name already includes newlines, the first will be used to split the
+string and the others replaced with spaces. If the name is at most
+`maxLength` characters, it'll be returned as `['', name]`. Otherwise, we
+find the most balanced white space to use as a split point.
+
+## Trueish
+
+```js
+const isTrueish = require('@kansa/common/trueish')
+
+isTrueish() // false
+isTrueish('0') // false
+isTrueish(' False') // false
+isTrueish(-1) // true
+```
+
+### `function isTrueish(v: any): boolean`
+
+Casts input into boolean values. In addition to normal JavaScript casting
+rules, also trims and lower-cases strings before comparing them to the
+following: `''`, `'0'`, `'false'`, `'null'`. Matches to these result in a
+`false` value. The primary intent is to enable human-friendly handling of query
+parameter values.

common/auth-user.js

@@ -0,0 +1,39 @@
+const { AuthError, InputError } = require('./errors')
+
+module.exports = { isSignedIn, hasRole, matchesId }
+
+function isSignedIn(req, res, next) {
+  const { user } = req.session
+  if (user && user.email) next()
+  else next(new AuthError())
+}
+
+function hasRole(role) {
+  return function _hasRole(req, res, next) {
+    const { user } = req.session
+    if (user && user.email) {
+      if (Array.isArray(role)) {
+        if (role.some(r => user[r])) return next()
+      } else {
+        if (user[role]) return next()
+      }
+    }
+    next(new AuthError())
+  }
+}
+
+function matchesId(db, req, role) {
+  const id = parseInt(req.params.id)
+  const { user } = req.session
+  if (isNaN(id) || id < 0)
+    return Promise.reject(new InputError('Bad id number'))
+  if (!user || !user.email) return Promise.reject(new AuthError())
+  if ((Array.isArray(role) && role.some(r => user[r])) || (role && user[role]))
+    return Promise.resolve(id)
+  return db
+    .oneOrNone('SELECT email FROM people WHERE id = $1', id)
+    .then(data => {
+      if (data && user.email === data.email) return id
+      throw new AuthError()
+    })
+}

kansa/lib/config.js → common/config.js

@@ -10,12 +10,14 @@ const shape = {
   name: 'string',
   paid_paper_pubs: 'boolean',
   auth: {
+    admin_roles: ['string'],
     key_timeout: {
       admin: 'duration',
       normal: 'duration'
     },
     session_timeout: 'duration'
-  }
+  },
+  modules: 'object'
 }
 
 const getIn = (config, path) => {
@@ -40,6 +42,15 @@ function parseConfig(config, path, shape) {
         `Expected value for '${key}' to match regular expression ${shape}`
       )
     }
+  } else if (Array.isArray(shape)) {
+    if (!Array.isArray(value)) {
+      throw new Error(
+        `Expected array value for '${key}', but found ${typeof value}`
+      )
+    }
+    value.forEach((v, i) => {
+      parseConfig(config, path.concat(i), shape[0])
+    })
   } else if (typeof shape === 'object') {
     if (typeof value !== 'object') {
       throw new Error(

common/log-entry.js

@@ -0,0 +1,39 @@
+const logFields = [
+  // id SERIAL PRIMARY KEY
+  'timestamp', // timestamptz NOT NULL DEFAULT now()
+  'client_ip', // text NOT NULL
+  'client_ua', // text
+  'author', // text
+  'subject', // integer REFERENCES People
+  'action', // text NOT NULL
+  'parameters', // jsonb NOT NULL
+  'description' // text NOT NULL
+]
+
+class LogEntry {
+  constructor(req, desc = '') {
+    const { user } = req.session
+    this.timestamp = null
+    if (user && user.member_admin && req.body && req.body.timestamp) {
+      const ts = new Date(req.body.timestamp)
+      if (ts > 0) this.timestamp = ts.toISOString()
+    }
+    this.client_ip = req.ip
+    this.client_ua = req.headers['user-agent'] || null
+    this.author = (user && user.email) || null
+    this.subject = null
+    this.action = req.method + ' ' + req.baseUrl + req.path
+    this.parameters = Object.assign({}, req.query, req.body)
+    delete this.parameters.key
+    this.description = desc
+  }
+
+  write(db) {
+    const fields = logFields.filter(fn => this[fn] !== null)
+    const values = fields.map(fn => `$(${fn})`).join(', ')
+    const sqlValues = `(${fields.join(', ')}) VALUES(${values})`
+    return db.none(`INSERT INTO log ${sqlValues}`, this)
+  }
+}
+
+module.exports = LogEntry