daq-tools · amotl · Jun 9, 2023 · Jun 10, 2023 · Jun 17, 2023 · Jun 18, 2023
diff --git a/.env b/.env
@@ -6,6 +6,11 @@ MOSQUITTO_VERSION=2.0
 MOSQUITTO_MQTT_PORT=1883
 MOSQUITTO_WS_PORT=9001
 
+# CrateDB
+CRATEDB_VERSION=6.2
+CRATEDB_HTTP_PORT=4200
+CRATEDB_POSTGRESQL_PORT=5432
+
 # InfluxDB
 INFLUXDB_VERSION=1.8
 INFLUXDB_HTTP_PORT=8086

diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
@@ -35,13 +35,15 @@ jobs:
           "3.14",
         ]
         mosquitto-version: [ "2.0" ]
+        cratedb-version: [ "6.2" ]
         influxdb-version: [ "1.8" ]
         grafana-version: [ "13.1.0" ]
 
         include:
           - grafana-version: "7.5.17"
             python-version: "3.14"
             mosquitto-version: "2.0"
+            cratedb-version: "6.2"
             influxdb-version: "1.8"
             os: "ubuntu-22.04"
 #          - grafana-version: "8.5.27"
@@ -67,12 +69,19 @@ jobs:
           - grafana-version: "12.3.4"
             python-version: "3.14"
             mosquitto-version: "2.0"
+            cratedb-version: "6.2"
             influxdb-version: "1.8"
             os: "ubuntu-22.04"
 
     # https://docs.github.com/en/free-pro-team@latest/actions/guides/about-service-containers
     services:
 
+      cratedb:
+        image: crate:${{ matrix.cratedb-version }}
+        ports:
+          - 4200:4200
+          - 5432:5432
+
       influxdb:
         image: influxdb:${{ matrix.influxdb-version }}
         ports:
@@ -95,7 +104,7 @@ jobs:
       OS: ${{ matrix.os }}
       PYTHON: ${{ matrix.python-version }}
 
-    name: Python ${{ matrix.python-version }}, Grafana ${{ matrix.grafana-version }}, Mosquitto ${{ matrix.mosquitto-version }}, InfluxDB ${{ matrix.influxdb-version }}
+    name: Py ${{ matrix.python-version }}, Grafana ${{ matrix.grafana-version }}, Mosquitto ${{ matrix.mosquitto-version }}, InfluxDB ${{ matrix.influxdb-version }}, CrateDB ${{ matrix.cratedb-version }}
     steps:
 
       - name: Acquire sources

diff --git a/CHANGES.rst b/CHANGES.rst
@@ -10,8 +10,8 @@ in progress
 - Export: Repaired ``png`` rendering with ``matplotlib >= 3.8.0``
 - General: Dropped support for Python 3.7
 - DAQ: Masked ``PASSKEY`` variable coming from HTTP, emitted by Ecowitt
-
 - **Grafana:** Validated support with Grafana v13
+- CrateDB: Add support for data acquisition and data export
 
 .. _kotori-0.28.1:
 

diff --git a/CONTRIBUTORS.rst b/CONTRIBUTORS.rst
@@ -14,39 +14,52 @@ contributed to Kotori in one way or another.
 * Anne-Julia Scheuermann <annejulia@gmail.com>
 * Carolin Johannsen <caro@hiverize.org>
 * Clemens Gruber <clemens@hiveeyes.org>
+* Georg Traar <georg@crate.io>
 * Georges Gagliano <https://github.com/ggeorges39>
 * Géraldine Farcy-Merlo <https://mellisphera.com/>
+* Hernan Cianfagna <hernan.cianfagna@crate.io>
 * Jan Hoffmann <j.hoffmann@isarengineering.de>
 * Jens Schmelkus <jens@schmelkus.net>
 * Karsten Harazim <email@karstenharazim.de>
 * Lorenzo Pons <https://mellisphera.com/>
+* Marija Selakovic <marija@crate.io>
 * Markus Euskirchen <m@euse.de>
 * Matthias Büchner <m.buechner@gmail.com>
 * Matthias Mehldau <wetter@hiveeyes.org>
 * Michael Haberler <mah@mah.priv.at>
+* Niklas Schmidtmer <niklas@crate.io>
 * Richard Pobering <einsiedlerkrebs@ginnungagap.org>
 * Roh <incognito>
 * Rui Alexandre Teixeira Sousa Pinto <https://github.com/RuiPinto96>
 * Sebastian Donner <info@thunder-gfx.de>
 * Sebastian Henneke <s.henneke@isarengineering.de>
+* Sebastian Utz <sebastian@crate.io>
 * Smilie <smilie@posteo.de>
 * Thorsten Kluß <tox@uni-bremen.de>
+* Walter Behmann <walter@crate.io>
 * Weef <incognito>
 
-Thanks a bunch!
+Thanks a stack.
 
 
 ****************
 Acknowledgements
 ****************
+
 Kotori would not have been possible without all the amazing people
-working on the infrastructure the software is leveraging.
+working on the infrastructure the software is leveraging and building
+upon.
 
 - Glyph Lefkowitz et al. for conceiving, building and maintaining the
   Twisted network programming framework.
+- Dr. Andy Stanford-Clark and Arlen Nipper for inventing the MQTT
+  messaging protocol.
 - Roger Light et al. for conceiving, building and maintaining the
   Mosquitto MQTT broker.
 - Torkel Ödegaard and his team for creating and maintaining Grafana.
+- Bernd Dorn, Bernd Rössl, Jodok Batlogg, Bernhard Kuzel, Mathias
+  Fußenegger, Sebastian Utz, et al., for conceiving and maintaining
+  CrateDB.
 - Paul Dix and his team for creating and maintaining InfluxDB.
 - The PostgreSQL and MongoDB developers for creating and maintaining
   their database systems.
@@ -59,4 +72,4 @@ working on the infrastructure the software is leveraging.
   to put everything together.
 
 Thank you so much for providing such great infrastructure
-components and resources to the community! You know who you are.
+components and resources to the community. You know who you are.
diff --git a/README.rst b/README.rst
@@ -72,6 +72,10 @@ Kotori DAQ
         :target: https://github.com/grafana/grafana
         :alt: Supported Grafana versions
 
+  .. image:: https://img.shields.io/badge/CrateDB-4.2%20--%205.x-blue.svg
+        :target: https://github.com/crate/crate
+        :alt: Supported CrateDB versions
+
   .. image:: https://img.shields.io/badge/InfluxDB-1.6%2C%201.7%2C%201.8-blue.svg
         :target: https://github.com/influxdata/influxdb
         :alt: Supported InfluxDB versions
@@ -104,7 +108,7 @@ Details
 
 Kotori is a data and protocol broker within `SCADA`_ / `MDE`_
 systems, exclusively built upon industry-grade `free and open-source software`_
-like Grafana_, Mosquitto_, or InfluxDB_. It is written in Python_,
+like Grafana_, Mosquitto_, CrateDB_, or InfluxDB_. It is written in Python_,
 and uses the Twisted_ networking library.
 
 The best way to find out what you can do with Kotori, is by looking at
@@ -230,6 +234,7 @@ excellent development tooling.
 .. _all installation options: https://kotori.readthedocs.io/setup/
 .. _Autobahn: https://crossbar.io/autobahn/
 .. _contributors: https://kotori.readthedocs.io/project/contributors.html
+.. _CrateDB: https://github.com/crate/crate
 .. _Create an issue: https://github.com/daq-tools/kotori/issues/new
 .. _data historian: https://en.wikipedia.org/wiki/Operational_historian
 .. _development sandbox: https://kotori.readthedocs.io/setup/sandbox.html

diff --git a/doc/source/_meta.rst b/doc/source/_meta.rst
@@ -16,5 +16,5 @@
     :keywords lang=es: adquisición de datos, gráficas, conjunto de herramientas, telemetría
     :keywords lang=fr: acquisition de données, graphiques, boîte à outils, télémétrie
     :keywords:         m2m, iot, mqtt, wamp, http, rest, udp, binary struct,
-                       Kotori, Grafana, InfluxDB, Mosquitto, MongoDB, XMPP, SMTP
+                       Kotori, Grafana, CrateDB, InfluxDB, Mosquitto, MongoDB, XMPP, SMTP
 
diff --git a/doc/source/_resources.rst b/doc/source/_resources.rst
@@ -170,6 +170,8 @@
 .. _Raumfahrtagentur: http://www.raumfahrtagentur.org/
 
 .. NEW
+.. _CrateDB: https://github.com/crate/crate
+.. _CrateDB reference documentation: https://crate.io/docs/crate/reference/
 .. _curl: https://en.wikipedia.org/wiki/CURL
 .. _Flux data scripting language: https://docs.influxdata.com/flux/
 .. _Funky v3: https://harizanov.com/product/funky-v3/
@@ -178,12 +180,15 @@
 .. _InfluxDB line protocol: https://docs.influxdata.com/influxdb/v1.8/write_protocols/line_protocol_reference/
 .. _InfluxDB storage engine: https://docs.influxdata.com/influxdb/v2.7/reference/internals/storage-engine/
 .. _LoRaWAN: https://en.wikipedia.org/wiki/LoRa#LoRaWAN
+.. _Lucene: https://lucene.apache.org/
 .. _MongoDB manual: https://www.mongodb.com/docs/manual/
 .. _MongoDB Wire Protocol: https://www.mongodb.com/docs/manual/reference/mongodb-wire-protocol/
 .. _OpenXC: https://openxcplatform.com/
 .. _OpenXC for Python: http://python.openxcplatform.com/
 .. _multi-tenant: https://en.wikipedia.org/wiki/Multitenancy
+.. _near real-time: https://en.wikipedia.org/wiki/Near_real-time#Near_real-time
 .. _NodeUSB: https://web.archive.org/web/20210621192219/http://www.nodeusb.com/
+.. _PostgreSQL wire protocol: https://www.postgresql.org/docs/16/protocol.html
 .. _trunking: https://en.wikipedia.org/wiki/Trunking
 .. _webhook: https://en.wikipedia.org/wiki/Webhook
 .. _webhooks: https://en.wikipedia.org/wiki/Webhook

diff --git a/doc/source/about/scenarios.rst b/doc/source/about/scenarios.rst
@@ -85,8 +85,8 @@ We are standing on the shoulders of giants:
 - Leverage the open infrastructure based on Twisted_ - an event-driven networking engine -
   to implement custom software components.
 - Listen and talk M2M_ using the *MQ Telemetry Transport* connectivity protocol and software bus (MQTT_).
-- Store data points into InfluxDB_, a leading open source time series database suitable
-  for realtime analytics and sensor data storage.
+- Store data points into CrateDB_, InfluxDB_, or other open source time series databases
+  suitable for realtime analytics and sensor data storage.
 - Automate dashboard management in the context of data arriving on different data channels
   using Grafana_, an open source, feature rich metrics dashboard and graph editor.
 - Make Browser clients first-class citizens of the underpinning software bus framework

diff --git a/doc/source/about/technologies.rst b/doc/source/about/technologies.rst
@@ -26,14 +26,16 @@ Infrastructure components
 
 - Kotori_, a data acquisition, graphing and telemetry toolkit
 - Grafana_, a graph and dashboard builder for visualizing time series metrics
-- InfluxDB_, a time-series database
+- CrateDB_, a time series database with document features and more ¹
+- InfluxDB_, a time series database ¹
 - Mosquitto_, an MQTT message broker
-- MongoDB_, a document store (optional) ¹²
+- MongoDB_, a document store (optional) ²³
 
-| ¹ MongoDB is only required when doing CSV data acquisition, so it is completely
+| ¹ Kotori can either use CrateDB or InfluxDB as time series database.
+| ² MongoDB is only required when doing CSV data acquisition, so it is completely
 | optional for regular operations of Kotori.
-| ² As MongoDB - strictly speaking - stopped being free software recently (2018/2019),
-| it will probably be phased out gradually and replaced by PostgreSQL_.
+| ³ As MongoDB - strictly speaking - stopped being free software recently (2018/2019),
+| it will probably be phased out gradually and replaced by PostgreSQL_ or CrateDB_.
 
 
 Supported protocols

diff --git a/doc/source/attic/setup/debian-amd64.rst b/doc/source/attic/setup/debian-amd64.rst
@@ -16,7 +16,7 @@ Intro
 Install the whole stack on a Debian-based system. It is currently made of these free and open source software components:
 
 - Mosquitto_, a MQTT message broker
-- InfluxDB_, a time-series database
+- InfluxDB_, a time series database
 - Grafana_, a graph and dashboard builder for visualizing time series metrics
 - :ref:`Kotori`, a data acquisition, graphing and telemetry toolkit