diff --git a/documentation/src/main/resources/_posts/2018-02-07-milestone-announcement-020-M1.md b/documentation/src/main/resources/_posts/2018-02-07-milestone-announcement-020-M1.md
index 5b633fe117b..3934374a0e9 100644
--- a/documentation/src/main/resources/_posts/2018-02-07-milestone-announcement-020-M1.md
+++ b/documentation/src/main/resources/_posts/2018-02-07-milestone-announcement-020-M1.md
@@ -16,7 +16,7 @@ The Ditto team is proud to announce the next milestone release.
Have a look at the Milestone [0.2.0-M1 release notes](release_notes_020-M1.html).
The main changes are
-* being able to [search in namespaces](httpapi-search.html#query-parameters) which can speed up search queries when applied
+* being able to [search in namespaces](httpapi-search.html#examples) which can speed up search queries when applied
to a large population of digital twins
* the enhancement of our [Feature entity](basic-feature.html) by [Definitions](basic-feature.html#feature-definition)
which lays the foundation for using Features in a typesafe way (later by enforcing the schema with the help of an
diff --git a/documentation/src/main/resources/_posts/2018-12-05-example-command-and-control.md b/documentation/src/main/resources/_posts/2018-12-05-example-command-and-control.md
index bd30c0e47df..248a283b953 100644
--- a/documentation/src/main/resources/_posts/2018-12-05-example-command-and-control.md
+++ b/documentation/src/main/resources/_posts/2018-12-05-example-command-and-control.md
@@ -193,7 +193,7 @@ The request is now open to receive a command for 60 seconds before it is termina
### Send a Ditto message
-Now we can use the [Ditto Messages API](protocol-specification-things-messages.html#using-the-messages-api) to send a
+Now we can use the [Ditto Messages API](protocol-specification-things-messages.html#examples) to send a
message to the device waiting for a command:
```bash
curl -i -X POST 'https://ditto.eclipseprojects.io/api/2/things/org.eclipse.ditto:teapot/inbox/messages/brew?timeout=60' \
diff --git a/documentation/src/main/resources/_posts/2021-09-29-kafka-connectivity.md b/documentation/src/main/resources/_posts/2021-09-29-kafka-connectivity.md
index ac7b44a4a5a..dc9ec233325 100644
--- a/documentation/src/main/resources/_posts/2021-09-29-kafka-connectivity.md
+++ b/documentation/src/main/resources/_posts/2021-09-29-kafka-connectivity.md
@@ -18,7 +18,7 @@ The time has come to support consuming as well.
A Kafka connection behaves slightly different from other consuming Connections in Ditto.
The following aspects are special:
* [Backpressure by using acknowledgements](connectivity-protocol-bindings-kafka2.html#quality-of-service)
-* [Preserve order of messages on redelivery due to failed acknowledgements](connectivity-protocol-bindings-kafka2.html#backpressure-by-using-acknowledgements)
+* [Preserve order of messages on redelivery due to failed acknowledgements](connectivity-protocol-bindings-kafka2.html#backpressure-via-acknowledgements)
* [Expiry of messages](connectivity-protocol-bindings-kafka2.html#message-expiry)
### Scalability
diff --git a/documentation/src/main/resources/_posts/2021-11-03-oauth2.md b/documentation/src/main/resources/_posts/2021-11-03-oauth2.md
index 6ab14f5f88e..465c1423265 100644
--- a/documentation/src/main/resources/_posts/2021-11-03-oauth2.md
+++ b/documentation/src/main/resources/_posts/2021-11-03-oauth2.md
@@ -82,7 +82,7 @@ placeholder. The unencoded headers and body are as follows.
# Create the connection
-[Create a connection](connectivity-manage-connections.html#create-connection)
+[Create a connection](connectivity-manage-connections.html#create-a-connection)
publishing twin events to the event publishing webhook using OAuth2 credentials.
The `tokenEndpoint` field is set to the access token webhook.
diff --git a/documentation/src/main/resources/_posts/2021-12-20-http-live-channel.md b/documentation/src/main/resources/_posts/2021-12-20-http-live-channel.md
index b671610401b..de17d7e82a1 100644
--- a/documentation/src/main/resources/_posts/2021-12-20-http-live-channel.md
+++ b/documentation/src/main/resources/_posts/2021-12-20-http-live-channel.md
@@ -11,21 +11,21 @@ toc: true
---
The upcoming release of Eclipse Ditto **version 2.3.0** will support sending commands via the HTTP API
-directly to devices using the [live channel](protocol-twinlive.html#live) by just adding the `channel=live`
-query parameter to the same HTTP API request which would target the [twin](protocol-twinlive.html#twin).
+directly to devices using the [live channel](protocol-twinlive.html#live-channel) by just adding the `channel=live`
+query parameter to the same HTTP API request which would target the [twin](protocol-twinlive.html#twin-channel).
## HTTP Live channel
-Ditto supports sending all kind of [Thing commands](protocol-specification-things.html#commands) via
+Ditto supports sending all kind of [Thing commands](protocol-specification-things.html#create-and-modify-commands) via
the `live` channel directly to devices.
When sending a `live` command to a device, the device is responsible for sending a correlated and correct response
in [Ditto Protocol](protocol-overview.html).
Ditto supports two types of `channel`s for communication.
-* [twin](protocol-twinlive.html#twin): The default value of the channel parameter is `twin`
+* [twin](protocol-twinlive.html#twin-channel): The default value of the channel parameter is `twin`
to communicate with the persisted **twin** representation.
-* [live](protocol-twinlive.html#live): The `channel` parameter can be changed to `live`
+* [live](protocol-twinlive.html#live-channel): The `channel` parameter can be changed to `live`
to communicate with the real device.
{% include note.html content="In order to use the live channel, the device receiving live commands must be able to understand
diff --git a/documentation/src/main/resources/_posts/2021-12-22-live-channel-condition.md b/documentation/src/main/resources/_posts/2021-12-22-live-channel-condition.md
index 508e1e10488..6e5d487a244 100644
--- a/documentation/src/main/resources/_posts/2021-12-22-live-channel-condition.md
+++ b/documentation/src/main/resources/_posts/2021-12-22-live-channel-condition.md
@@ -10,11 +10,11 @@ sidebar: false
toc: true
---
-After the added option to target the [live channel](protocol-twinlive.html#live) by adding the `channel=live` to HTTP
+After the added option to target the [live channel](protocol-twinlive.html#live-channel) by adding the `channel=live` to HTTP
requests (see also [blog post about "HTTP live channel"](2021-12-20-http-live-channel.html)), Eclipse Ditto
**version 2.3.0** will in addition support to define a
[live channel condition](basic-conditional-requests.html#live-channel-condition), which, when evaluating to `true`,
-will retrieve data from a device via the live channel, or fall back to the persisted [twin](protocol-twinlive.html#twin)
+will retrieve data from a device via the live channel, or fall back to the persisted [twin](protocol-twinlive.html#twin-channel)
otherwise.
{% include note.html content="In order to use the live channel, the device receiving live commands must be able to understand
@@ -72,7 +72,7 @@ GET /api/2/things/my.namespace:my-polling-device-1/features/temperature/properti
```
The specified `live-channel-condition` will evaluate to `true`, meaning that the retrieve is transformed to a
-[live command](protocol-twinlive.html#live) and sent to the device, e.g. connected via a
+[live command](protocol-twinlive.html#live-channel) and sent to the device, e.g. connected via a
[managed connection](basic-connections.html).
Upon receiving the "live retrieve" at the device, the device can create a command response correlated with the same
`correlation-id` and send it back to Ditto with the current value.
@@ -80,7 +80,7 @@ This value is then returned as result of the `GET`, the HTTP response header `ch
sent by the device by having the value `live`.
If the device does not answer with a correctly correlated response within the given `timeout`, the request will fall back
-to the [twin](protocol-twinlive.html#twin) channel, retrieving the data from the last known persisted temperature value
+to the [twin](protocol-twinlive.html#twin-channel) channel, retrieving the data from the last known persisted temperature value
in the twin managed by Ditto.
The HTTP response header `channel` will indicate that the data was received by the persisted twin by having the value
`twin`.
diff --git a/documentation/src/main/resources/_posts/2022-01-21-release-announcement-230.md b/documentation/src/main/resources/_posts/2022-01-21-release-announcement-230.md
index 6a958137555..a6b5e036f84 100644
--- a/documentation/src/main/resources/_posts/2022-01-21-release-announcement-230.md
+++ b/documentation/src/main/resources/_posts/2022-01-21-release-announcement-230.md
@@ -12,11 +12,11 @@ toc: false
The Eclipse Ditto teams announces availability of Eclipse Ditto [2.3.0](https://projects.eclipse.org/projects/iot.ditto/releases/2.3.0).
-It contains mainly new features around the Ditto ["live" channel](protocol-twinlive.html#live) which can be used to
+It contains mainly new features around the Ditto ["live" channel](protocol-twinlive.html#live-channel) which can be used to
directly interact with devices.
Such live commands may now be easily [created via the HTTP API](2021-12-20-http-live-channel.html) - and in addition
-a conventional API call targeting the persisted [twin](protocol-twinlive.html#twin) may now be automatically converted
+a conventional API call targeting the persisted [twin](protocol-twinlive.html#twin-channel) may now be automatically converted
to a live command, based on a passed [live channel condition](2021-12-22-live-channel-condition.html).
With that, we are proud that we now can provide a really powerful addition to Ditto's "Digital Twin pattern":
diff --git a/documentation/src/main/resources/_posts/2023-10-09-ditto-benchmark.md b/documentation/src/main/resources/_posts/2023-10-09-ditto-benchmark.md
index d66b930b612..867b54943ec 100644
--- a/documentation/src/main/resources/_posts/2023-10-09-ditto-benchmark.md
+++ b/documentation/src/main/resources/_posts/2023-10-09-ditto-benchmark.md
@@ -220,7 +220,7 @@ Example message:
device_id:org.eclipse.ditto:test-thing-1,ditto_message:y!{"topic":"org.eclipse.ditto/test-thing-1/things/twin/commands/modify","path":"features/coffee-brewer/properties/brewed-coffees","value":"10"}
```
-In this scenario, connectivity service is used to create a ditto kafka connection, which reads messages from the provided topic, maps them to a ditto modify command and forwards it to things service. The things service then executes mongodb update query and generates the [thing modified event](protocol-specification-things-create-or-modify.html#event), which is pushed to the MMock service instance via an HTTP Push connection. Also, the kafka connection is configured with [qos=1](connectivity-protocol-bindings-kafka2.html#quality-of-service), which means if there is no acknowledgement that the thing is persisted, the operation will be retried.
+In this scenario, connectivity service is used to create a ditto kafka connection, which reads messages from the provided topic, maps them to a ditto modify command and forwards it to things service. The things service then executes mongodb update query and generates the [thing modified event](protocol-specification-things.html#create-or-modify-a-thing), which is pushed to the MMock service instance via an HTTP Push connection. Also, the kafka connection is configured with [qos=1](connectivity-protocol-bindings-kafka2.html#quality-of-service), which means if there is no acknowledgement that the thing is persisted, the operation will be retried.
The HTTP Push connection looks like the following:
@@ -354,7 +354,7 @@ Scaling connectivity instance and changing our connection to have **clientCount*
### Device live messages(commands)
-This scenario executes HTTP POST requests to ditto's [live channel](protocol-twinlive.html#live). An HTTP Push connection is subscribed for them and in turn pushes to a MMock instance that acts as a 'dummy' device receiver of live messages/commands and simply responds with pre-configured ditto response.
+This scenario executes HTTP POST requests to ditto's [live channel](protocol-twinlive.html#live-channel). An HTTP Push connection is subscribed for them and in turn pushes to a MMock instance that acts as a 'dummy' device receiver of live messages/commands and simply responds with pre-configured ditto response.
The HTTP POST request looks like the following:
diff --git a/documentation/src/main/resources/css/customstyles.css b/documentation/src/main/resources/css/customstyles.css
index 24542d119eb..c69d1b7ce09 100644
--- a/documentation/src/main/resources/css/customstyles.css
+++ b/documentation/src/main/resources/css/customstyles.css
@@ -297,43 +297,68 @@ p.post-meta {font-size: 80%; color: #777;}
.entry-date{font-size:14px; line-height:1.71429; margin-bottom:0; text-transform:uppercase;}
-/* search area */
-#search-demo-container ul#results-container {
- list-style: none;
- font-size: 12px;
- background-color: white;
- position: absolute;
- top: 40px; /* if you change anything about the nav, you'll prob. need to reset the top and left values here.*/
- left: 20px;
- z-index: -1;
- width:223px;
- border-left: 1px solid #dedede;
+/* pagefind search overrides */
+.pagefind-search-container {
+ position: relative;
+ margin-top: 7px;
+ margin-left: 10px;
}
+.pagefind-search-container .pagefind-ui {
+ --pagefind-ui-scale: 0.6;
+ --pagefind-ui-primary: #3a8c9a;
+}
-ul#results-container a {
- background-color: transparent;
+.pagefind-search-container .pagefind-ui__form {
+ padding: 0;
+ margin: 0;
}
-ul#results-container a:hover {
- color: black;
+.pagefind-search-container .pagefind-ui__search-input {
+ padding: 2px 8px;
+ width: 12em;
+ font-size: 13px;
+ font-family: 'Open Sans', sans-serif;
+ box-sizing: border-box;
+ height: 24px;
+ border-radius: 3px;
+}
+
+.pagefind-search-container .pagefind-ui__search-clear {
+ top: auto;
+ height: 24px;
+ padding: 0 4px;
}
+.pagefind-search-container .pagefind-ui__drawer {
+ position: absolute;
+ right: 0;
+ top: 100%;
+ z-index: 10000;
+ background: white;
+ width: 400px;
+ max-height: 70vh;
+ overflow-y: auto;
+ border: 1px solid #dedede;
+ border-radius: 0 0 4px 4px;
+ box-shadow: 0 4px 12px rgba(0,0,0,0.15);
+}
-#search-demo-container a:hover {
- color: black;
+.pagefind-search-container .pagefind-ui__result-link {
+ color: #3a8c9a;
}
-#search-input {
- padding: .5em;
- margin-left:15px;
- width:15em;
- font-size: 0.8em;
- -webkit-box-sizing: border-box;
- -moz-box-sizing: border-box;
- box-sizing: border-box;
- margin-top:10px;
+
+.pagefind-search-container .pagefind-ui__result-link:hover {
+ color: #248ec2;
+}
+
+@media (max-width: 768px) {
+ .pagefind-search-container .pagefind-ui__drawer {
+ width: 90vw;
+ right: -50px;
+ }
}
-/* end search */
+/* end pagefind search */
.filter-options {
margin-bottom: 20px;
diff --git a/documentation/src/main/resources/openapi/ditto-api-2.yml b/documentation/src/main/resources/openapi/ditto-api-2.yml
index 3d7b9e27312..9ce1b899d8d 100644
--- a/documentation/src/main/resources/openapi/ditto-api-2.yml
+++ b/documentation/src/main/resources/openapi/ditto-api-2.yml
@@ -4238,7 +4238,7 @@ paths:
No special permission is required to issue a claim message.
### Example
- See [Claiming](https://www.eclipse.dev/ditto/protocol-specification-things-messages.html#sending-and-handling-claim-messages) concept in detail and example in GitHub.
+ See [Claiming](https://www.eclipse.dev/ditto/protocol-specification-things-messages.html#claim-messages) concept in detail and example in GitHub.
However, in that scenario, the policy should grant you READ and WRITE permission on
the "message:/" resource in order to be able to send the message and read the response.
Further, the things-client which handles the "claim" message, needs permission to change the policy itself
diff --git a/documentation/src/main/resources/openapi/sources/paths/messages/inbox-claim.yml b/documentation/src/main/resources/openapi/sources/paths/messages/inbox-claim.yml
index bdbd4c7d102..f2b6149cb7c 100644
--- a/documentation/src/main/resources/openapi/sources/paths/messages/inbox-claim.yml
+++ b/documentation/src/main/resources/openapi/sources/paths/messages/inbox-claim.yml
@@ -38,7 +38,7 @@ post:
No special permission is required to issue a claim message.
### Example
- See [Claiming](https://www.eclipse.dev/ditto/protocol-specification-things-messages.html#sending-and-handling-claim-messages) concept in detail and example in GitHub.
+ See [Claiming](https://www.eclipse.dev/ditto/protocol-specification-things-messages.html#claim-messages) concept in detail and example in GitHub.
However, in that scenario, the policy should grant you READ and WRITE permission on
the "message:/" resource in order to be able to send the message and read the response.
Further, the things-client which handles the "claim" message, needs permission to change the policy itself
diff --git a/documentation/src/main/resources/pages/ditto/advanced-data-by-pass.md b/documentation/src/main/resources/pages/ditto/advanced-data-by-pass.md
index ca68151735f..efab018d299 100644
--- a/documentation/src/main/resources/pages/ditto/advanced-data-by-pass.md
+++ b/documentation/src/main/resources/pages/ditto/advanced-data-by-pass.md
@@ -5,84 +5,51 @@ tags: [advanced]
permalink: advanced-data-by-pass.html
---
-This pattern centers around the idea to delegate the data transmission to external services, by-passing the Ditto cluster,
-while still being able to benefit from Ditto's [policy system](basic-policy.html) and IoT architecture.
+The Data By-Pass pattern delegates large data transfers to an external high-performance proxy while still using Ditto's [policy system](basic-policy.html) for access control.
-## Context
+{% include callout.html content="**TL;DR**: When you need to transfer data volumes too large for the Ditto cluster (e.g., database query results), route the negotiation through Ditto messages for authentication, then redirect the client to a high-performance proxy for the actual data retrieval." type="primary" %}
-You have services exposing their functionality transparently though Ditto's messaging API as part of your digital twin.
-E.g. a history service providing the actual interface to your timeseries database as part of the things interface such
-that a client may not need to know if the history actually is managed by the thing itself or any other program.
-You use Ditto's [policy system](basic-policy.html) to secure access to your services that way.
+## Overview
-Your services provide data in quantities that are not suited for transmission through the Ditto cluster directly,
-because of (de-)serialization costs, round-trip-times etc.
+You may have services that expose functionality through Ditto's messaging API as part of your digital twin -- for example, a history service providing access to a time-series database through a Thing's interface. You use Ditto's [policy system](basic-policy.html) to secure access to these services.
-## Problem
+However, some data volumes are too large for the Ditto cluster. Serialization costs, roundtrip times, and message size quotas make it impractical to return large query results as Ditto message responses.
-You want to query a greater amount of data (e.g. database query result) by issuing a Ditto message to a thing which is
-picked up by a service speaking with you databases. It does not work to just let this service return the result
-as a response to the Ditto message, since the messaging system in the Ditto cluster is not designed for big quantities
-of data and will reject them based on tight quotas. Also the costs due to many (de-)serialization steps are high.
+## How it works
-## Solution
+### Architecture
-The solution consists of the following systems:
+The pattern involves five components:
-* **database**: where your bigger chunks of data reside and wait to be delivered / queried
-* **database provider mirco-service**: the service managing the database connection and exposing it to clients through
- things messaging API
-* **thing**: a digital twin with extended API through a micro-service
-* **client**: a client-application trying to receive bigger quantities of data via a things messaging API in the scope
- of that thing and secured via ditto policies
-* **high-performance data proxy** (or just proxy): a third-party application proxy sitting in-between the database and
- the provider micro-service managing data delivery
+* **Database**: Where your larger data sets reside
+* **Database provider microservice**: Manages the database connection and exposes it through the Thing messaging API
+* **Thing**: A digital twin with an extended API through the microservice
+* **Client**: An application requesting data via the Thing messaging API
+* **High-performance data proxy**: A third-party proxy (e.g., nginx-based) that sits between the database and the client for actual data delivery
-{% include image.html file="pages/advanced/data-by-pass-architectural-design.jpg" alt="Architectural Design"
-caption="Architectural design of the data by-pass pattern showing all actors and their interactions." max-width=800 %}
+{% include image.html file="pages/advanced/data-by-pass-architectural-design.jpg" alt="Architectural Design" caption="Architectural design of the data by-pass pattern showing all actors and their interactions." max-width=800 %}
-In order for the client-application to retrieve the requested data in a secure and performant way we introduce a
-high-performance proxy (e.g. based on nginx, example below). The proxy will not have any credentials by itself,
-it's just serving prepared queries on a randomly, hard-to-guess URL with an expiration time of 5 minutes.
-It features an admin API which the micro-service has credentials to access.
+### Request flow
-The provider micro-service hooks into a twin (e.g. via websockets) and listens for queries.
-If a query arrives it will formulate the query, store it at the high-performance proxy (which might already query the data)
-and return a randomly generated URL to the proxy together with a Location-header as a response to the client-application.
-The client then needs to follow the response in order to retrieve the data from the proxy.
+1. The client sends a query as a Ditto message to a Thing.
+2. The provider microservice (listening via WebSocket) receives the message.
+3. The microservice formulates the database query and stores it at the high-performance proxy under a randomly generated, hard-to-guess URL with a 5-minute expiration.
+4. The microservice returns the proxy URL with a `Location` header (HTTP 303) as the Ditto message response.
+5. The client follows the redirect to retrieve the actual data from the proxy.
-With this approach the access to the database is secured via Ditto [policies](basic-policy.html) and in scope of single
-things while the data retrieval happens via a performant proxy application without the Ditto cluster ever seeing those packages.
+The proxy itself has no credentials -- it only serves pre-prepared queries on expiring random URLs. The provider microservice holds the admin credentials for the proxy's configuration API.
-*Note: Keep in mind that security in this situation is highly dependent of the micro-service implementation.
-You have to make sure that your implementation uses provided information of ditto properly and that the contents of
-a message do not allow a violation of the policy. E.g. through SQL-Injections.*
+### Security
-## Discussion
-
-**Benefits**:
-
-* Higher performance compared to using just Ditto
-* The Ditto [policy system](basic-policy.html) can be utilized to scope and secure data access from clients to databases/-stores
-
-**Drawbacks**:
-
-* A third-party application for the high-performance proxy has to be added and maintained
-* A custom messaging API is necessary in the first place introducing a higher complexity
-* A translation of certain query-languages from messages to the actual databases / applications has to be implemented
-
-**Issues**:
+Access to the database is secured through Ditto [policies](basic-policy.html) and scoped to individual Things. The actual data transfer bypasses the Ditto cluster entirely.
-* Managing and communicating custom messaging APIs is not natively supported in Ditto, other ways have to be explored to
- keep APIs consistent
+{% include warning.html content="Security depends heavily on your microservice implementation. Ensure that message contents cannot violate policy boundaries, for example through SQL injection attacks." %}
-## Policies
+## Configuration
-Policies can be used to restrict access to the provider micro-service and through that eventually to the database using
-restrictions on the `message:/` resource.
+### Policy setup
-Let's assume that the provider micro-service registers via websockets and expects requests to the message-topic
-`/services/history`. With the following policy entry we can allow access to this resource:
+Restrict access to the provider microservice using `message:/` resources. If the microservice listens for messages on the topic `/services/history`:
```json
{
@@ -104,34 +71,36 @@ Let's assume that the provider micro-service registers via websockets and expect
}
```
-The first resource entry revokes any access to messages for subjects of this type. This is optional. The next entry
-allows the provider micro-service to read messages from the topic `/services/history`. Note that we've decided to insert
-another "namespace" `/services` here to distinguish these messages from other device faced messages. The last section
-than allows the provider micro-service to reply to the received requests with it's 303 response.
+The first entry revokes all message access for subjects of this type (optional). The second entry allows the microservice to read incoming queries. The third entry allows the microservice to send responses.
-This can also be built against single features. Since features have to be stated explicitly in the policy, this is not
-as general but can provide a more fine-grained access control when using distinct policies for different things,
-or features with same names over multiple things.
+You can also scope this to individual Features for more fine-grained access control.
-## Proxy Implementations
+## Examples
+
+### Proxy implementation
+
+The [ceryx proxy project](https://github.com/sourcelair/ceryx) was used for the [reference implementation](https://github.com/w4tsn/ceryx). It was enhanced with delegation features (a modified nginx with Redis for storing randomly generated IDs correlated with prepared queries and authentication). See the [fork's source code](https://github.com/w4tsn/ceryx) or the [container image](https://quay.io/repository/w4tsn/ceryx).
+
+### Known uses
+
+**[othermo GmbH](https://www.othermo.de)** uses this pattern for a history service. The service connects to Ditto via WebSocket and responds to `/history` messages. It translates the messaging API into InfluxDB queries, stores them at the proxy with a 5-minute expiring URL, and returns the URL to the client. The provider service uses InfluxDB tags to scope data by `thingId`, policy, and path information.
+
+## Discussion
-The [ceryx proxy project](https://github.com/sourcelair/ceryx) was used for the
-[PoC (or reference implementation)](https://github.com/w4tsn/ceryx) of this pattern.
-It was enhanced with delegation features which still have to be contributed upstream.
-Have a look at the [forks source code](https://github.com/w4tsn/ceryx) or
-the [corresponding container image](https://quay.io/repository/w4tsn/ceryx) until then.
+**Benefits:**
+* Higher data throughput compared to routing everything through Ditto
+* Ditto's policy system secures and scopes data access from clients to databases
-The ceryx proxy is a modified nginx with a redis-database to store the randomly generated IDs correlating with prepared queries.
-It is not suited for this use-case on its own so capabilities to store queries (including Authentication) behind
-expiring random URLs was added, but not send upstream yet.
+**Drawbacks:**
+* You must add and maintain a third-party proxy application
+* You need a custom messaging API, which adds complexity
+* You must implement query language translation from messages to your database
-## Known uses
+**Open issues:**
+* Ditto does not natively support managing and communicating custom messaging APIs
-**[othermo GmbH](https://www.othermo.de) uses this for a history-service**: The history service connects to Ditto via
-websockets and hooks into things by answering specific `/history` messages. The messages API is translated to InfluxDB queries
-which then are stored with a randomly generated URL and expiration of 5 minutes at the high-performance proxy.
-The service then returns the random URL to the client which then follows the 303 to retrieve the actual data.
+## Further reading
-The messages contain InfluxDB-similar query elements while the query is only constructed at the provider service.
-That's because the provider service uses the databases specifics like Tags in InfluxDB to assign thingId, policy and
-path information in order to get the stored data into the right scopes and to be able to retrieve the correct sets of data.
+* [Policies](basic-policy.html)
+* [Messages](basic-messages.html)
+* [WebSocket Protocol Binding](httpapi-protocol-bindings-websocket.html)
diff --git a/documentation/src/main/resources/pages/ditto/architecture-overview.md b/documentation/src/main/resources/pages/ditto/architecture-overview.md
index 1bcc9aa3775..1d90b493d29 100644
--- a/documentation/src/main/resources/pages/ditto/architecture-overview.md
+++ b/documentation/src/main/resources/pages/ditto/architecture-overview.md
@@ -5,47 +5,50 @@ tags: [architecture]
permalink: architecture-overview.html
---
-The architecture chapter describes the overall architecture of Eclipse Ditto and in detail which sub-components fulfill
-which responsibilities.
+Eclipse Ditto consists of five microservices that communicate over a Pekko cluster to provide digital twin management, access control, search, and connectivity to external systems.
-## Top level component view
+{% include callout.html content="**TL;DR**: Ditto runs five services -- Policies, Things, Things-Search, Gateway, and Connectivity -- all in one Pekko cluster. They communicate via TCP using serializable signals, with MongoDB as the persistence layer and nginx as a reverse proxy." type="primary" %}
-This overview shows the Ditto services (components), the externally provided and consumed API endpoints,
-the external dependencies (MongoDB and nginx) and the relations of the services to each other.
+## Overview
+
+The architecture of Eclipse Ditto is built around five cooperating microservices, each with a clear responsibility. This page describes the top-level component view, how services are defined, and how they communicate.
+
+## How it works
+
+### Component view
{% include image.html file="pages/architecture/ditto-architecture-overview-2022.png" alt="Ditto services and context" caption="Ditto services in blue and context with nginx as reverse proxy and MongoDB" max-width=800 %}
-The components have the following tasks:
+The five services and their responsibilities:
+
+| Service | Responsibilities |
+|---------|-----------------|
+| [Policies](architecture-services-policies.html) | Persist and enforce (authorize) [Policies](basic-policy.html) |
+| [Things](architecture-services-things.html) | Persist and enforce (authorize) [Things](basic-thing.html) and [Features](basic-feature.html) |
+| [Things-Search](architecture-services-things-search.html) | Track changes to `Things`, `Features`, and `Policies`; maintain an optimized search index; execute search queries |
+| [Gateway](architecture-services-gateway.html) | Provide [HTTP](httpapi-overview.html) and [WebSocket](httpapi-protocol-bindings-websocket.html) APIs |
+| [Connectivity](architecture-services-connectivity.html) | Persist [Connections](basic-connections.html); send and receive [Ditto Protocol](protocol-overview.html) messages to/from external message brokers |
-* [Policies](architecture-services-policies.html): persistence + enforcement (authorization) of [Policies](basic-policy.html)
-* [Things](architecture-services-things.html): persistence + enforcement (authorization) of [Things](basic-thing.html)
- and [Features](basic-feature.html)
-* [Things-Search](architecture-services-things-search.html): tracking changes to `Things`, `Features`, `Policies` and
- updating an optimized search index + executes queries on this search index
-* [Gateway](architecture-services-gateway.html): provides HTTP and WebSocket API
-* [Connectivity](architecture-services-connectivity.html):
- * persistence of [Connections](basic-connections.html)
- * sends [Ditto Protocol](protocol-overview.html) messages to external message brokers and receives messages from them
+All services run in the same [Pekko cluster](https://pekko.apache.org/docs/pekko/current/typed/cluster-concepts.html) and reach each other via TCP without requiring an additional message broker between them.
-All services run in the same [Pekko cluster](https://pekko.apache.org/docs/pekko/current/typed/cluster-concepts.html) and can
-reach each other via TCP without the need for an additional message broker in between.
+### Microservice definition
-## Components
+Each Ditto microservice follows three rules:
-Ditto consists of multiple "microservices" as shown in the above component view.
+1. **Own data store**: Only the owning microservice can access and write to its data store.
+2. **Signal-based API**: The service exposes its API as [signals](basic-signals.html) (commands, command responses, events).
+3. **Signal-only access**: Other services interact with it exclusively through these signals.
-A "microservice" in Ditto is defined as:
+### Communication
-* has its own data store which only this microservice may access and write to
-* has an API in form of [signals](basic-signals.html) (commands, command responses, events)
-* can be accessed by other services only via the defined [signals](basic-signals.html)
+All microservices communicate asynchronously within the Ditto cluster using [Pekko remoting](https://pekko.apache.org/docs/pekko/current/general/remoting.html). Each service acts as both a TCP server (accepting connections) and a TCP client (sending messages to other services).
-## Communication
+All messages sent between services are serializable. Ditto [signals](basic-signals.html) serialize from Java objects to JSON and deserialize back from JSON to Java objects.
-All microservices can communicate asynchronously in a Ditto cluster. Communication is done via
-[Pekko remoting](https://pekko.apache.org/docs/pekko/current/general/remoting.html) which means that each service acts as server,
-providing a TCP endpoint, as well as client sending data to other services.
+## Further reading
-All messages which are sent between Ditto microservices must in a way be serializable and deserializable.
-All Ditto [signals](basic-signals.html) can be serialized from Java objects to JSON representation and deserialized back
-from JSON to Java objects.
+* [Policies Service](architecture-services-policies.html)
+* [Things Service](architecture-services-things.html)
+* [Things-Search Service](architecture-services-things-search.html)
+* [Gateway Service](architecture-services-gateway.html)
+* [Connectivity Service](architecture-services-connectivity.html)
diff --git a/documentation/src/main/resources/pages/ditto/architecture-services-connectivity.md b/documentation/src/main/resources/pages/ditto/architecture-services-connectivity.md
index dc47211c60e..a716b5322ed 100644
--- a/documentation/src/main/resources/pages/ditto/architecture-services-connectivity.md
+++ b/documentation/src/main/resources/pages/ditto/architecture-services-connectivity.md
@@ -5,55 +5,58 @@ tags: [architecture, connectivity]
permalink: architecture-services-connectivity.html
---
-The "connectivity" service enables Ditto to establish and manage client-side connections to external service endpoints.
-You can communicate with your connected things/twins over those connections via [Ditto Protocol] messages. The
-connectivity service supports various transport protocols, which are bound to the [Ditto Protocol] via specific
-[Protocol Bindings].
-
-If you don't have the option to transform your payload to a [Ditto Protocol Message] on the client-side, the
-connectivity service offers a flexible and customizable [payload mapping] on top.
+The Connectivity service establishes and manages client-side connections to external message brokers, translating between [Ditto Protocol] messages and external transport protocols.
-## Model
+{% include callout.html content="**TL;DR**: The Connectivity service persists connection configurations, manages connections to external endpoints (AMQP, MQTT, Kafka, HTTP), translates messages to/from Ditto Protocol, and supports custom payload mapping." type="primary" %}
-The model of the connectivity service is defined around the entity `Connection`:
+## Overview
+You use the Connectivity service to integrate Ditto with external systems. It creates client-side connections to message brokers and other endpoints, sends and receives [Ditto Protocol] messages over those connections, and maps custom payloads when needed.
-* [model](https://github.com/eclipse-ditto/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model)
+If you cannot transform your payload to a [Ditto Protocol Message] on the client side, the Connectivity service provides flexible and customizable [payload mapping].
-## Signals
+## How it works
-Other services can communicate with the connectivity service via:
+### Model
-* [ConnectivityCommands](https://github.com/eclipse-ditto/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model/signals/commands/ConnectivityCommand.java):
- implementing classes provide commands which are processed by this service
-* [ConnectivityEvents](https://github.com/eclipse-ditto/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model/signals/events/ConnectivityEvent.java):
- implementing classes represent events which are emitted when entities managed by this service were modified
+The service is built around the `Connection` entity:
-## Persistence
+* [Connection model](https://github.com/eclipse-ditto/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model)
-The connectivity service uses [Pekko persistence](https://pekko.apache.org/docs/pekko/current/persistence.html?language=java) and
-with that [Event sourcing](basic-signals.html#architectural-style) in order to persist changes to
-and restore persisted [connections](basic-connections.html).
+### Signals
-## Enforcement
+Other services communicate with the Connectivity service through two signal types:
-The connectivity service does not enforce/authorize [connection signals](#signals) by a [policy](basic-policy.html) as
-the connection does not contain a `policyId`.
+* [ConnectivityCommands](https://github.com/eclipse-ditto/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model/signals/commands/ConnectivityCommand.java): Commands for managing connections (create, modify, open, close, delete)
+* [ConnectivityEvents](https://github.com/eclipse-ditto/ditto/tree/master/connectivity/model/src/main/java/org/eclipse/ditto/connectivity/model/signals/events/ConnectivityEvent.java): Events emitted when connection entities change
-## Tasks
+### Persistence
-* create/remove connections (by persisting them)
-* connect/disconnect to endpoints
-* restore existing connections upon restart/failover
-* translate incoming [Ditto Protocol] messages to [commands](basic-signals-command.html)
- and translate [command responses](basic-signals-commandresponse.html) back to [Ditto Protocol] response messages
-* map custom message protocols to the [Ditto Protocol]
+The Connectivity service uses [Pekko persistence](https://pekko.apache.org/docs/pekko/current/persistence.html?language=java) with [event sourcing](basic-signals.html#overview) to persist and restore [connections](basic-connections.html).
+### Enforcement
+The Connectivity service does not enforce authorization through [policies](basic-policy.html) because connections do not reference a `policyId`. Access control for connection management is handled through the [DevOps user](operating-devops.html) or via the HTTP API with appropriate permissions.
+
+### Tasks
+
+The Connectivity service performs these core tasks:
+
+* Create and remove connections by persisting their configuration
+* Connect to and disconnect from external endpoints
+* Restore existing connections on restart or failover
+* Translate incoming [Ditto Protocol] messages to [commands](basic-signals-command.html) and translate [command responses](basic-signals-commandresponse.html) back to [Ditto Protocol] response messages
+* Map custom message protocols to the [Ditto Protocol] via [Protocol Bindings]
+
+## Further reading
+
+* [Connections concept](basic-connections.html)
+* [Protocol Bindings](protocol-bindings.html)
+* [Ditto Protocol Overview](protocol-overview.html)
+* [Architecture Overview](architecture-overview.html)
-
[Ditto Protocol]: protocol-overview.html
[Ditto Protocol Message]: protocol-specification-things-messages.html
-[payload mapping]: protocol-specification-things-messages.html
+[payload mapping]: connectivity-mapping.html
[Protocol Bindings]: protocol-bindings.html
diff --git a/documentation/src/main/resources/pages/ditto/architecture-services-gateway.md b/documentation/src/main/resources/pages/ditto/architecture-services-gateway.md
index fc444ea7644..00d3245b6ea 100644
--- a/documentation/src/main/resources/pages/ditto/architecture-services-gateway.md
+++ b/documentation/src/main/resources/pages/ditto/architecture-services-gateway.md
@@ -5,28 +5,37 @@ tags: [architecture]
permalink: architecture-services-gateway.html
---
-The "gateway" service is responsible for providing Ditto's [HTTP](httpapi-overview.html) +
-[WebSocket](httpapi-protocol-bindings-websocket.html) API.
+The Gateway service provides Ditto's external-facing [HTTP](httpapi-overview.html) and [WebSocket](httpapi-protocol-bindings-websocket.html) APIs, translating between HTTP/WebSocket requests and internal Ditto signals.
-## Model
+{% include callout.html content="**TL;DR**: The Gateway service is the entry point for all HTTP and WebSocket traffic. It translates HTTP requests into internal commands, routes Ditto Protocol messages from WebSocket and Cloud Events, and streams change notifications back to connected clients." type="primary" %}
-The gateway service has no model by its own, but uses the model of all the services it provides the HTTP + WebSocket API for.
+## Overview
-## Signals
+The Gateway service acts as Ditto's API layer. It does not own any entities or persistence -- instead, it translates external requests into internal [signals](basic-signals.html) and forwards them to the appropriate services within the cluster.
-The gateway service has no signals by its own, but uses the signals of all the services it provides the HTTP + WebSocket API for.
+## How it works
-## Persistence
+### Model and signals
-The gateway service has no persistence by its own.
+The Gateway service does not define its own entity model or signal types. It uses the models and signals from all other Ditto services to provide a unified API surface.
-## Tasks
+### Persistence
-* translate HTTP request to [commands](basic-signals-command.html) and translates [command responses](basic-signals-commandresponse.html)
- back to HTTP responses
-* translate [Ditto Protocol](protocol-overview.html) messages incoming via the [WebSocket](httpapi-protocol-bindings-websocket.html)
- to [commands](basic-signals-command.html) and translates [command responses](basic-signals-commandresponse.html) back
- to [Ditto Protocol](protocol-overview.html) response messages
-* accepts [Ditto Protocol](protocol-overview.html) messages incoming via the [Cloud Events HTTP Binding](httpapi-protocol-bindings-cloudevents.html)
-* subscribe for [events](basic-signals-event.html) in Ditto cluster and emits [change notifications](basic-changenotifications.html)
- via connected [WebSocket](httpapi-protocol-bindings-websocket.html) clients or via [SSEs](httpapi-sse.html)
+The Gateway service does not maintain any persistence of its own.
+
+### Tasks
+
+The Gateway service performs these core tasks:
+
+* **HTTP API**: Translate incoming HTTP requests to [commands](basic-signals-command.html) and translate [command responses](basic-signals-commandresponse.html) back to HTTP responses
+* **WebSocket API**: Translate [Ditto Protocol](protocol-overview.html) messages arriving via [WebSocket](httpapi-protocol-bindings-websocket.html) to commands, and translate responses back to Ditto Protocol messages
+* **Cloud Events**: Accept [Ditto Protocol](protocol-overview.html) messages via the [Cloud Events HTTP Binding](httpapi-protocol-bindings-cloudevents.html)
+* **Change notifications**: Subscribe to [events](basic-signals-event.html) in the Ditto cluster and stream [change notifications](basic-changenotifications.html) to connected [WebSocket](httpapi-protocol-bindings-websocket.html) clients and [SSE](httpapi-sse.html) consumers
+
+## Further reading
+
+* [HTTP API Overview](httpapi-overview.html)
+* [WebSocket Protocol Binding](httpapi-protocol-bindings-websocket.html)
+* [Cloud Events Binding](httpapi-protocol-bindings-cloudevents.html)
+* [SSE (Server-Sent Events)](httpapi-sse.html)
+* [Architecture Overview](architecture-overview.html)
diff --git a/documentation/src/main/resources/pages/ditto/architecture-services-policies.md b/documentation/src/main/resources/pages/ditto/architecture-services-policies.md
index ea231875a62..4b208e93ce9 100644
--- a/documentation/src/main/resources/pages/ditto/architecture-services-policies.md
+++ b/documentation/src/main/resources/pages/ditto/architecture-services-policies.md
@@ -5,30 +5,39 @@ tags: [architecture]
permalink: architecture-services-policies.html
---
-The "policies" service takes care of persisting [Policies](basic-policy.html).
+The Policies service persists and enforces [Policies](basic-policy.html), which control access to all resources in Ditto.
-## Model
+{% include callout.html content="**TL;DR**: The Policies service owns all Policy entities, persists them via event sourcing in MongoDB, and enforces authorization on policy-related commands using the policy itself." type="primary" %}
-The model of the policies service is defined around the entity `Policy`:
+## Overview
+The Policies service is responsible for the complete lifecycle of [Policy](basic-policy.html) entities -- creation, modification, retrieval, and deletion. Every authorization decision for policy-level operations flows through this service.
+
+## How it works
+
+### Model
+
+The service is built around the `Policy` entity:
* [Policy model](https://github.com/eclipse-ditto/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model)
-## Signals
+### Signals
+
+Other services communicate with the Policies service through two signal types:
+
+* [PolicyCommands](https://github.com/eclipse-ditto/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model/signals/commands/PolicyCommand.java): Commands that the service processes (create, modify, retrieve, delete)
+* [PolicyEvents](https://github.com/eclipse-ditto/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model/signals/events/PolicyEvent.java): Events emitted when policy entities change
-Other services can communicate with the policies service via:
+### Persistence
-* [PolicyCommands](https://github.com/eclipse-ditto/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model/signals/commands/PolicyCommand.java):
- implementing classes provide commands which are processed by this service
-* [PolicyEvents](https://github.com/eclipse-ditto/ditto/tree/master/policies/model/src/main/java/org/eclipse/ditto/policies/model/signals/events/PolicyEvent.java):
- implementing classes represent events which are emitted when entities managed by this service were modified
+The Policies service uses [Pekko persistence](https://pekko.apache.org/docs/pekko/current/persistence.html?language=java) with [event sourcing](basic-signals.html#overview) to persist changes and restore [policies](basic-policy.html).
-## Persistence
+### Enforcement
-The policies service uses [Pekko persistence](https://pekko.apache.org/docs/pekko/current/persistence.html?language=java) and
-with that [Event sourcing](basic-signals.html#architectural-style) in order to persist changes to
-and restore persisted [policies](basic-policy.html).
+The service authorizes all [policy signals](#signals) using the policy's own rules. In other words, you need the right permissions in a policy to modify that policy.
-## Enforcement
+## Further reading
-The policies service enforces/authorizes [policy signals](#signals) by the "own" [policy](basic-policy.html).
+* [Policy concept](basic-policy.html)
+* [Architecture Overview](architecture-overview.html)
+* [Things Service](architecture-services-things.html)
diff --git a/documentation/src/main/resources/pages/ditto/architecture-services-things-search.md b/documentation/src/main/resources/pages/ditto/architecture-services-things-search.md
index 05e64b1f221..55d30cad1f1 100644
--- a/documentation/src/main/resources/pages/ditto/architecture-services-things-search.md
+++ b/documentation/src/main/resources/pages/ditto/architecture-services-things-search.md
@@ -5,40 +5,45 @@ tags: [architecture, search]
permalink: architecture-services-things-search.html
---
-The "things-search" service takes care of:
+The Things-Search service maintains an optimized search index of all Things and executes search queries against it.
-* updating an optimized search index of `Things` based on the [events](basic-signals-event.html) emitted by the
- [things](architecture-services-things.html) and [policies](architecture-services-policies.html) services when entities
- are changed there
-* executing search queries against the search index in order to find out which `Things` (which `thingId`s) match a
- given search
+{% include callout.html content="**TL;DR**: The Things-Search service listens for Thing and Policy change events, updates a search-optimized MongoDB collection, and processes RQL search queries to find matching Things." type="primary" %}
-## Model
+## Overview
-The things-search service has no model (entity) by its own, but uses the model of [things](architecture-services-things.html)
-and [policies](architecture-services-policies.html) services.
+The Things-Search service has two main responsibilities:
-It however contains a model which can transform an RQL
-search query into a Java domain model which is defined here:
+1. **Index maintenance**: Track changes to Things, Features, and Policies by consuming [events](basic-signals-event.html) from the [Things](architecture-services-things.html) and [Policies](architecture-services-policies.html) services, and keep a search-optimized index up to date.
+2. **Query execution**: Accept search queries and return matching Thing IDs from the index.
-* [rql parser ast](https://github.com/eclipse-ditto/ditto/tree/master/rql/model/src/main/java/org/eclipse/ditto/rql/model/predicates/ast)
+## How it works
-## Signals
+### Model
-Other services can communicate with the things-search service via:
+The Things-Search service does not define its own entity model. It uses the models from the [Things](architecture-services-things.html) and [Policies](architecture-services-policies.html) services.
+It does include a parser that transforms [RQL](basic-rql.html) search queries into a Java domain model:
-* [commands](https://github.com/eclipse-ditto/ditto/tree/master/thingsearch/model/src/main/java/org/eclipse/ditto/thingsearch/model/signals/commands):
- containing commands and command responses which are processed by this service
+* [RQL parser AST](https://github.com/eclipse-ditto/ditto/tree/master/rql/model/src/main/java/org/eclipse/ditto/rql/model/predicates/ast)
-## Persistence
+### Signals
-The Things-Search service maintains its own persistence in which it stores `Things` in an optimized way in order to
-provide a full search on arbitrary `Thing` data.
+Other services communicate with the Things-Search service via:
-Things-Search creates the following MongoDB collections:
+* [Commands](https://github.com/eclipse-ditto/ditto/tree/master/thingsearch/model/src/main/java/org/eclipse/ditto/thingsearch/model/signals/commands): Search commands and their responses
-* `search`: The search index.
-* `searchSync`: A single-document capped collection containing the instant until which `Thing` events are
-indexed for sure.
+### Persistence
+The Things-Search service maintains its own MongoDB collections optimized for full-text and attribute-based search:
+
+| Collection | Purpose |
+|------------|---------|
+| `search` | The search index containing Thing data in a search-optimized format |
+| `searchSync` | A single-document capped collection that records the instant until which Thing events are indexed |
+
+## Further reading
+
+* [Search concept](basic-search.html)
+* [Architecture Overview](architecture-overview.html)
+* [Things Service](architecture-services-things.html)
+* [Policies Service](architecture-services-policies.html)
diff --git a/documentation/src/main/resources/pages/ditto/architecture-services-things.md b/documentation/src/main/resources/pages/ditto/architecture-services-things.md
index 8687e892258..7d40d43383b 100644
--- a/documentation/src/main/resources/pages/ditto/architecture-services-things.md
+++ b/documentation/src/main/resources/pages/ditto/architecture-services-things.md
@@ -5,30 +5,40 @@ tags: [architecture]
permalink: architecture-services-things.html
---
-The "things" service takes care of persisting [Things](basic-thing.html) and [Features](basic-feature.html).
+The Things service persists and enforces authorization for [Things](basic-thing.html) and [Features](basic-feature.html), which represent your digital twins.
-## Model
+{% include callout.html content="**TL;DR**: The Things service owns all Thing and Feature entities, persists them via event sourcing in MongoDB, and enforces authorization using the Policy referenced by each Thing's `policyId`." type="primary" %}
-The model of the things service is defined around the entities `Thing` and `Feature`:
+## Overview
+
+The Things service manages the full lifecycle of [Thing](basic-thing.html) and [Feature](basic-feature.html) entities. It handles creation, modification, retrieval, and deletion, and it enforces access control on every operation.
+
+## How it works
+
+### Model
+
+The service is built around two entities, `Thing` and `Feature`:
* [Thing model](https://github.com/eclipse-ditto/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model)
-## Signals
+### Signals
+
+Other services communicate with the Things service through two signal types:
+
+* [ThingCommands](https://github.com/eclipse-ditto/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model/signals/commands/ThingCommand.java): Commands that the service processes (create, modify, retrieve, delete)
+* [ThingEvents](https://github.com/eclipse-ditto/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model/signals/events/ThingEvent.java): Events emitted when Thing or Feature entities change
-Other services can communicate with the things service via:
+### Persistence
-* [ThingCommands](https://github.com/eclipse-ditto/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model/signals/commands/ThingCommand.java):
- implementing classes provide commands which are processed by this service
-* [ThingEvents](https://github.com/eclipse-ditto/ditto/tree/master/things/model/src/main/java/org/eclipse/ditto/things/model/signals/events/ThingEvent.java):
- implementing classes represent events which are emitted when entities managed by this service were modified
+The Things service uses [Pekko persistence](https://pekko.apache.org/docs/pekko/current/persistence.html?language=java) with [event sourcing](basic-signals.html#overview) to persist changes and restore [things](basic-thing.html).
-## Persistence
+### Enforcement
-The things service uses [Pekko persistence](https://pekko.apache.org/docs/pekko/current/persistence.html?language=java) and
-with that [Event sourcing](basic-signals.html#architectural-style) in order to persist changes to
-and restore persisted [things](basic-thing.html).
+The service authorizes all [thing signals](#signals) using the [Policy](basic-policy.html) referenced by the Thing's `policyId`. You must have the appropriate permissions granted in the referenced policy to perform operations on a Thing.
-## Enforcement
+## Further reading
-The things service enforces/authorizes [thing signals](#signals) by the via `policyId` referenced
-[policy](basic-policy.html).
+* [Thing concept](basic-thing.html)
+* [Feature concept](basic-feature.html)
+* [Architecture Overview](architecture-overview.html)
+* [Policies Service](architecture-services-policies.html)
diff --git a/documentation/src/main/resources/pages/ditto/basic-acknowledgements.md b/documentation/src/main/resources/pages/ditto/basic-acknowledgements.md
index f84fff39dc4..326f285ef35 100644
--- a/documentation/src/main/resources/pages/ditto/basic-acknowledgements.md
+++ b/documentation/src/main/resources/pages/ditto/basic-acknowledgements.md
@@ -5,101 +5,78 @@ tags: [model]
permalink: basic-acknowledgements.html
---
-Acknowledgements are a concept in Ditto used to indicate that a
-[supported signal](#supported-signal-types) was successfully received or processed by either an
-internal Ditto functionality or an external subscriber of that signal.
+Acknowledgements let you confirm that a signal was successfully received or processed, either by Ditto internally or by an external subscriber.
-Acknowledgements can be seen as (potentially multiple) responses to a single signal like for example a twin command.
-This means that Ditto collects all the [requested acknowledgements](#requesting-acks) until the signal is
-successfully processed within a specified timeout interval.
+{% include callout.html content="**TL;DR**: Request acknowledgements (via the `requested-acks` header) to get confirmation that a command was persisted, forwarded, or processed by subscribers. Use them to achieve 'at least once' delivery guarantees (QoS 1)." type="primary" %}
-## Usage scenario examples
-Acknowledgements are useful for accomplishing the following example tasks:
+## Overview
-* Postpone a response to an API request (e.g. block an HTTP request) until one or more specific actions were performed
- in Ditto (e.g. a modification was successfully persisted).
-* Postpone a response until an external subscriber connected to Ditto reports that it successfully processed an
- [event](basic-signals-event.html) which e.g. resulted by a persistence change of Ditto.
-* Provide a QoS (quality of service) guarantee of "at least once" when processing
- messages in an end-to-end manner by e.g. technically acknowledging/settling a processed signal from a message broker
- (e.g. [AMQP 1.0](connectivity-protocol-bindings-amqp10.html) or [MQTT](connectivity-protocol-bindings-mqtt.html)) only
- after it was successfully applied to Ditto and potentially also to 3rd parties.
+Acknowledgements act as (potentially multiple) responses to a single signal such as a twin command. Ditto collects all [requested acknowledgements](#requesting-acks) until the signal is successfully processed or a timeout expires.
+## Usage scenarios
+
+Acknowledgements help you accomplish tasks like:
+
+* **Block until persisted**: Postpone an API response until a modification is persisted in Ditto.
+* **Block until consumed**: Postpone a response until an external subscriber confirms it processed the resulting event.
+* **At-least-once delivery**: Technically acknowledge a message from a broker (AMQP, MQTT) only after Ditto has processed it and optionally forwarded it to third parties.
## Supported signal types
-For the following signal types it is possible to define that certain acknowledgements are *requested*.
-* [Commands](basic-signals-command.html) *modifying the state of a twin* (Twin commands),
-* Live commands,
-* Live messages.
+You can request acknowledgements for:
+
+* [Commands](basic-signals-command.html) that modify twin state (twin commands)
+* Live commands
+* Live messages
## Acknowledgement labels
-A common (matching) label links acknowledgement requests and their corresponding acknowledgements.
-Ditto already uses some labels for its built-in acknowledgement requests.
-Those labels may not be used when sending back a custom acknowledgement.
+
+A label links an acknowledgement request to its corresponding acknowledgement. Ditto uses some labels internally; you cannot use those labels for custom acknowledgements.
### Built-in acknowledgement labels
-Ditto provides built-in acknowledgement requests that are automatically fulfilled on certain actions within the Ditto
-cluster:
-* **twin-persisted**: For acknowledgement requests of twin modifying commands.
-It is fulfilled when a modifying command has successfully updated the digital twin in Ditto's persistence.
-It is ignored for commands in the live channel.
-* **live-response**: For acknowledgement requests of live commands and live messages.
-It is fulfilled when a subscriber of the live command or message sends a corresponding response.
-It is ignored for commands in the twin channel.
-* **search-persisted**: For acknowledgement requests of twin modifying commands.
-It is fulfilled when a modifying command has successfully updated the search index of the digital twin.
-It is ignored for commands in the live channel.
+
+Ditto automatically fulfills these labels:
+
+* **twin-persisted**: Fulfilled when a modifying command successfully updates the digital twin in Ditto's persistence. Ignored for live channel commands.
+* **live-response**: Fulfilled when a subscriber of a live command or message sends a corresponding response. Ignored for twin channel commands.
+* **search-persisted**: Fulfilled when a modifying command successfully updates the search index. Ignored for live channel commands.
### Custom acknowledgement labels
-In addition to the [built-in](#built-in-acknowledgement-labels) acknowledgement requests,
-a supported signal can contain custom acknowledgement requests.
-A subscriber of such a signal can detect a requested acknowledgement via the `"requested-acks"` header.
-If the subscriber is in charge of handling a requested acknowledgement it
-[issues an acknowledgement](#issuing-acknowledgements).
+In addition to built-in labels, you can include custom acknowledgement requests in supported signals. A subscriber detects requested labels via the `"requested-acks"` header and [issues an acknowledgement](#issuing-acknowledgements) for the labels it handles.
+
+## Acknowledgement structure
-## Acknowledgements (ACKs)
-A single acknowledgement contains the following information:
-* Acknowledgement label (one of the requested labels of the [ack requests](#requesting-acks))
-* Header fields
- * mandatory: **correlation-id** the same correlation ID as the one of the signal which requested the acknowledgement
- * optional: additional header fields
-* Status code (HTTP status code semantic) defining whether an acknowledgement was successful or not
-* Optional payload as JSON
+A single acknowledgement contains:
-The [Ditto Protocol specification](protocol-specification-acks.html) describes in detail what is contained.
-An example of how acknowledgements in Ditto Protocol look like can be found at the
-[acknowledgement examples](protocol-examples.html#acknowledgements-acks) section.
+* **Label**: one of the requested labels
+* **Headers**: must include `correlation-id` matching the original signal; may include additional headers
+* **Status code**: HTTP-semantics status code indicating success or failure
+* **Payload** (optional): JSON data
+The [Ditto Protocol specification](protocol-specification-acks.html) describes the format in detail. See the [acknowledgement examples](protocol-examples.html#acknowledgements-acks) for sample messages.
-## Requesting ACKs
-With every supported signal there is the option to request acknowledgements.
-Acknowledgement requests are expressed as protocol specific header fields of signals.
-The following sections explain the various ways of requesting acknowledgements.
+## Requesting ACKs
-[Events](basic-signals-event.html) emitted by Ditto will include the custom acknowledgement requests in the
-`"requested-acks"` header.
+You request acknowledgements via headers on supported signals.
+
+[Events](basic-signals-event.html) emitted by Ditto include the custom acknowledgement requests in the `"requested-acks"` header.
### Requesting ACKs via HTTP
-Either specify the following HTTP header fields:
- * **requested-acks**: a comma separated list of [acknowledgement labels](#acknowledgement-labels).
- Example: `requested-acks: twin-persisted,some-connection-id:my-custom-ack`.
- * **timeout**: an optional time interval (in ms, s or m) to define how long the HTTP request should wait for
- acknowledgements and block.
- Default and maximum value: `60s`.
- Examples: `timeout: 42s`, `timeout: 250ms`, `timeout: 1m`.
-
-Or specify the header fields as query parameters to the HTTP params, e.g.:
+
+Set these HTTP headers:
+
+* **requested-acks**: comma-separated list of [acknowledgement labels](#acknowledgement-labels). Example: `requested-acks: twin-persisted,some-connection-id:my-custom-ack`
+* **timeout**: how long to block (default and max: `60s`). Examples: `timeout: 42s`, `timeout: 250ms`, `timeout: 1m`
+
+Or use query parameters:
+
```
PUT /api/2/things/org.eclipse.ditto:thing-1?requested-acks=twin-persisted,my-custom-ack&timeout=42s
```
-The response of an HTTP request, which requested several acknowledgements, will differ from the response to an HTTP
-request without acknowledgement requests.
+**Example response** -- all acknowledgements successful (overall status `200`):
-Example response when 2 acknowledgements were requested and were successful. The overall HTTP status code will be
-`200` (OK) in this case:
```json
{
"twin-persisted": {
@@ -117,9 +94,7 @@ Example response when 2 acknowledgements were requested and were successful. The
},
"my-custom-ack": {
"status": 200,
- "payload": {
- "outcome": "green"
- },
+ "payload": { "outcome": "green" },
"headers": {
"version": 2,
"correlation-id": "db878735-4957-4fd9-92dc-6f09bb12a093"
@@ -128,8 +103,8 @@ Example response when 2 acknowledgements were requested and were successful. The
}
```
-Example response when 2 acknowledgements were requested and one lead to a timeout. The overall HTTP status code will be
-`424` (Dependency failed) in this case:
+**Example response** -- one acknowledgement timed out (overall status `424`):
+
```json
{
"twin-persisted": {
@@ -153,293 +128,199 @@ Example response when 2 acknowledgements were requested and one lead to a timeou
"message": "The acknowledgement request reached the specified timeout of 42,000ms.",
"description": "Try increasing the timeout and make sure that the requested acknowledgement is sent back in time."
},
- "headers": {
- "version": 2,
- "correlation-id": "db878735-4957-4fd9-92dc-6f09bb12a093"
- }
+ "headers": { "correlation-id": "db878735-4957-4fd9-92dc-6f09bb12a093" }
}
}
```
### Requesting ACKs via WebSocket
-Together with a received Ditto [command](basic-signals-command.html) in [Ditto Protocol](protocol-specification.html),
-`"requested-acks"` (as JsonArray of strings) and `"timeout"` headers in the
-[Ditto Protocol headers](protocol-specification.html#headers) can be included in order to request acknowledgements via
-WebSocket.
-The response will be an (aggregating) [acknowledgements](protocol-specification-acks.html#acknowledgements-aggregating)
-message.
+Include `"requested-acks"` (JSON array of strings) and `"timeout"` in the [Ditto Protocol headers](protocol-specification.html#headers). The response is an [aggregated acknowledgements](protocol-specification-acks.html#acknowledgements-aggregated) message.
### Requesting ACKs via connections
-Acknowledgements for Ditto managed [connection sources](basic-connections.html#sources) can be requested in 2 ways:
-* specifically for each consumed supported signal as part of
- the [Ditto Protocol headers](protocol-specification.html#headers) `"requested-acks"` (as JsonArray of strings)
-* by configuring the managed connection source to
- [request acknowledgements for all consumed supported signals](basic-connections.html#source-acknowledgement-requests).
-
-#### Requesting ACKs via Ditto Protocol message
-Together with a received Ditto [command](basic-signals-command.html) in [Ditto Protocol](protocol-specification.html),
-`"requested-acks"` (as JsonArray of strings) and `"timeout"` headers in the
-[Ditto Protocol headers](protocol-specification.html#headers) can be included in order to request acknowledgements
-via established connections consuming messages from [sources](basic-connections.html#sources).
-
-The response will be an (aggregating) [acknowledgements](protocol-specification-acks.html#acknowledgements-aggregating)
-message.
-
-#### Requesting ACKs via connection source configuration
-[Connection sources](basic-connections.html#sources) can be
-[configured to add specific acknowledgement requests](basic-connections.html#source-acknowledgement-requests) for each
-consumed message of the underlying physical connection (e.g. to a message broker).
-
-This can be used in order to ensure that e.g. all messages consumed from a single source should be processed in an
-"at least once" mode (QoS 1).
-E.g. if configured that the [built-in](#built-in-acknowledgement-labels) `twin-persisted` acknowledgement is requested,
-a received twin-modifying command will only be technically acknowledged to the connection channel if Ditto successfully
-applied and persisted the command.
-
-{% include note.html content="These requested acknowledgements will be appended after payload mapping is applied.
+
+You can request acknowledgements in two ways:
+
+1. **Per signal**: Set `"requested-acks"` in the [Ditto Protocol headers](protocol-specification.html#headers) of each consumed signal.
+2. **Per source**: [Configure the connection source](basic-connections.html#source-acknowledgement-requests) to add acknowledgement requests to all consumed signals.
+
+{% include note.html content="These requested acknowledgements will be appended after payload mapping is applied.
This means, that in case you decided to split your message into multiple messages, all of these messages will request the same acknowledgements.
If this is not what you want to achieve, have a look at [how to add acknowledgement requests during payload mapping](#requesting-acks-via-ditto-protocol-message-in-payload-mapping)." %}
#### Requesting ACKs via Ditto Protocol message in payload mapping
-During inbound payload mapping, you can create one or more Ditto Protocol messages.
-
-If you configured your connection source to add requested acknowledgements to your commands, this will cause all
-produced messages to request the same acknowledgements.
-If you however want to add requested acknowledgements only to some of those created messages, you need to set the
-`"requested-acks"` header (as described in
-[Requesting ACKs via Ditto Protocol message](#requesting-acks-via-ditto-protocol-message) section) during payload
-mapping for those commands you like to request an acknowledgement.
+During inbound payload mapping, you can set `"requested-acks"` headers on individual messages rather than applying the source-level configuration uniformly.
## Issuing acknowledgements
-Acknowledgements are issued by subscribers of events generated by twin-modifying commands, or by subscribers of
-live commands and live messages. In order to issue a single acknowledgement, a
-[Ditto Protocol acknowledgement](protocol-specification-acks.html#acknowledgement) message has to be built and sent
-back, using the same `"correlation-id"` in the [protocol headers](protocol-specification.html#headers) as contained
-in the received twin event, live command or live message.
-The labels of issued acknowledgements are globally unique for each subscriber. Before a subscriber is allowed to
-send acknowledgements, it must declare the labels of acknowledgements it sends. Any declared label taken by another
-subscriber causes an appropriate error for each channel that may issue acknowledgements.
+Subscribers of twin events, live commands, or live messages issue acknowledgements by sending a [Ditto Protocol acknowledgement](protocol-specification-acks.html#acknowledgement) message with the same `"correlation-id"` as the received signal.
-### Semantics of status codes for issued acknowledgements
+Each subscriber must declare the labels of acknowledgements it sends. Any label already declared by another subscriber causes an error.
-The `status` code of issued acknowledgements have certain semantics when it comes to whether Ditto shall e.g. try to
-retry processing a consumed message (when it was consumed from a message broker which persisted the message).
+### Status code semantics
-Application specific code invoking a Ditto API and requesting acknowledgements should also adapt the semantics in order
-to correctly perform retries:
+| Status code | Description | Should retry | Reasoning |
+|---|---|---|---|
+| `2xx` | Successfully processed | No | Received and processed. |
+| `4xx` | Request error | No | Bad request or auth failure -- retrying yields the same result. |
+| `408` | Request timeout | Yes | Timeout -- retrying may succeed. |
+| `424` | Mixed status codes | Yes | Contains a timeout or server error. |
+| `5xx` | Server error | Yes | Temporary backend error. |
-| Status code | Description | Should be retried | Reasoning |
-| --- | --- | --- | --- |
-| `2xx` | Successfully processed / ACKed | ❌ | Successfully received + processed. |
-| `4xx` | Request was erroneous | ❌ | Request could not be understood or authentication failed, so retrying would result in the same outcome. |
-| `408` | Request timeout | ✔ | HTTP semantic for request timeout, retrying does make sense. |
-| `424` | Mixed status codes, with at least one request error | ✔ | There could be a timeout or server error in the mixed status codes, so retrying does make sense. |
-| `5xx` | Server error | ✔ | The processing backend encountered a temporary error, retrying could solve the issue. |
+### Issuing ACKs via HTTP
-### Issuing ACKs via Ditto's HTTP API
-It is not possible to issue acknowledgements via Ditto's HTTP API, because it is impossible to subscribe for twin events,
-live commands or live messages via HTTP.
+You cannot issue acknowledgements via Ditto's HTTP API. HTTP does not support subscribing to
+twin events, live commands, or live messages -- use WebSocket or managed connections instead.
### Issuing ACKs via WebSocket
-Create and send the [Ditto Protocol acknowledgement](protocol-specification-acks.html#acknowledgement) message over an
-established WebSocket in response to a twin event, live command or live message that contains a `"requested-acks"`
-header.
-Only acknowledgements with declared labels are accepted. To declare acknowledgement labels, set them as the value
-of the query parameter `declared-acks` as comma-separated list:
+Send the [acknowledgement message](protocol-specification-acks.html#acknowledgement) over the WebSocket. Declare labels via the `declared-acks` query parameter:
+
```
GET /ws/2?declared-acks=some-connection-id:ack-label-1,my:ack-label-2
```
-The websocket will be closed right after it has been opened if any other another subscriber already declared the same label.
-This means that it is not possible to establish a second websocket connection with the same declared acknowledgements
-before closing the first one.
+The WebSocket closes immediately if another subscriber already declared the same label.
-{% include warning.html content="Therefore, it is not recommended relying on the websocket API for high
- availability scenarios."
+{% include warning.html content="Therefore, it is not recommended relying on the websocket API for high
+ availability scenarios."
%}
### Issuing ACKs via connections
-Requested acknowledgements for Ditto managed [connection targets](basic-connections.html#targets) can be issued in 2
-ways:
-
-* specifically for each published twin event, live command or live message by sending a
- [Ditto Protocol acknowledgement](protocol-specification-acks.html#acknowledgement) back,
- via a source of the same connection;
-* by configuring the managed connection target to automatically
- [issue acknowledgements](basic-connections.html#target-issued-acknowledgement-label) for all published twin events,
- live commands and live messages that request them.
-
-Acknowledgements sent via a source must
-[have their labels declared](basic-connections.html#source-declared-acknowledgement-labels)
-in the field `declaredAcks` as a JSON array.
-The labels of target-issued acknowledgements are declared automatically.
-Acknowledgement labels of a connection must be prefixed by the connection ID or the `{%raw%}{{connection:id}}{%endraw%}`
-placeholder followed by a colon, for example `{%raw%}{{connection:id}}:my-custom-ack{%endraw%}`.
-
-If some source-declared or target-issued acknowledgement labels are taken by a websocket subscriber,
-all acknowledgements sent by the connection are rejected with error until the websocket is closed.
-
-#### Issuing ACKs via Ditto Protocol acknowledgement message
-Create and send the [Ditto Protocol acknowledgement](protocol-specification-acks.html#acknowledgement) message over a
-source of the connection, in response to an event, live command or live message which contained an `"requested-acks"`
-header.
-
-#### Issuing ACKs via connection target configuration
-[Connection targets](basic-connections.html#targets) can be configured to
-[issue certain acknowledgements automatically](basic-connections.html#target-issued-acknowledgement-label)
-for each twin event, live command or live message published to the underlying physical connection (e.g. to a message
-broker).
-
-This can be used in order to automatically issue technical acknowledgements once an event, live command or live message
-was published to an HTTP endpoint or into a message broker. When this target guarantees having processed the event
-via its protocol (see the
-[defined semantics of the status codes](#semantics-of-status-codes-for-issued-acknowledgements)), a successful
-acknowledgement is created and returned to the requester.
+You can issue acknowledgements in two ways:
+
+1. **Via source**: Send a [Ditto Protocol acknowledgement](protocol-specification-acks.html#acknowledgement) message through a source of the same connection. Labels must be [declared](basic-connections.html#source-declared-acknowledgement-labels) in the `declaredAcks` field as a JSON array.
+2. **Via target configuration**: [Configure the target](basic-connections.html#target-issued-acknowledgement-label) to automatically issue acknowledgements for each twin event, live command, or live message published to the underlying channel (e.g., a message broker). When the target confirms delivery via its transport protocol (see [status code semantics](#status-code-semantics)), Ditto creates a successful acknowledgement and returns it to the requester. Target-issued labels are declared automatically.
+
+Acknowledgement labels for connections must be prefixed by the connection ID or the `{%raw%}{{connection:id}}{%endraw%}` placeholder followed by a colon, e.g., `{%raw%}{{connection:id}}:my-custom-ack{%endraw%}`.
+
+If a source-declared or target-issued label conflicts with a label declared by a WebSocket subscriber, all acknowledgements from the connection are rejected until the WebSocket is closed.
## Quality of Service
-### QoS 0 - at most once
-By default, Ditto processes all messages/commands processed in an "at most once" (or QoS 0) semantic.
-For many of the use cases in the IoT, QoS 0 is sufficient, e.g. when processing telemetry data of a sensor:
-if one sensor value is not applied to the digital twin there will soon follow the next sensor reading, and the twin will
-be eventually up to date again.
-
-### QoS 1 - at least once
-However, there are IoT use cases where it is of upmost importance that a signal is processed "at least once" (or QoS 1),
-e.g. in order to guarantee that it was persisted in the digital twin or that an [event](basic-signals-event.html)
-consumer connected to Ditto did successfully receive a [notification](basic-changenotifications.html) which resulted
-from a [command](basic-signals-command.html) which Ditto received and processed.
-
-The "acknowledgements" concept documented on this page provides means by which supported signals Ditto consumes and
-processes are treated with an "at least once" (or QoS 1) semantic.
-
-[Create/modify commands](protocol-specification-things-create-or-modify.html) will technically be acknowledged on the
-sent channel (e.g. HTTP or WebSocket or any [connection type](basic-connections.html#connection-types)) when it was a
-success.
-If it could not be applied successfully, the signal will be negatively acknowledged.
-The [status code of the acknowledgement](protocol-specification-acks.html#combined-status-code) reflects the failure.
-
-Based on the used channel, the acknowledgement will be translated to the capabilities of the command or live message
-channel, e.g. for HTTP an HTTP response will be sent with the outcome as HTTP status (`2xx` for a successful
-acknowledgement, and `4xx` for a non-successful one) together with additional details as HTTP response.
-
-### Assure QoS until persisted in Ditto - twin-persisted
-In order to ensure that a [create/modify command](protocol-specification-things-create-or-modify.html) resulted in a
-successful update of twin in Ditto's managed database, [request the acknowledgement](#requesting-acks) for
-the [built-in "twin-persisted"](#built-in-acknowledgement-labels) acknowledgement label.
-
-### Assure QoS until processing of a live command/message by a subscriber - live-response
-In order to ensure that a live command or live message is processed by a subscriber, set `response-required` to true.
-This way the [built-in "live-response" acknowledgement label](#built-in-acknowledgement-labels) is [automatically requested](#requesting-acks) for the live command/message.
-This acknowledgement request is fulfilled when the subscriber sends a live response or message response.
-
-### Assure QoS until processing of a twin event or live command/message by subscribers - custom label
-In order to ensure that a [create/modify command](protocol-specification-things-create-or-modify.html) resulted in an
-event which was consumed by an application integrating with Ditto, or that a live command or live message is consumed
-without any live or message response, [request the acknowledgement](#requesting-acks) for a
-[custom acknowledgement label](#custom-acknowledgement-labels).
+### QoS 0 -- at most once
+
+By default, Ditto processes signals with "at most once" semantics. For many IoT use cases (e.g., telemetry data), this is sufficient -- if one sensor reading is lost, the next one updates the twin.
+
+### QoS 1 -- at least once
+
+For critical signals, use acknowledgements to achieve "at least once" delivery. Modify commands
+are technically acknowledged on the transport channel (HTTP, WebSocket, or any
+[connection type](basic-connections.html#supported-connection-types)) only when processing succeeds. On failure,
+the signal is negatively acknowledged (e.g., AMQP NACK, MQTT no-PUBACK), triggering redelivery
+by the broker.
+
+The [acknowledgement status code](protocol-specification-acks.html#combined-status-code) reflects the
+outcome. For HTTP, this translates to `2xx` for success and `4xx` for failure.
+
+#### Ensure persistence -- twin-persisted
+
+Request the [built-in `twin-persisted`](#built-in-acknowledgement-labels) label to confirm that a
+modify command successfully updated the digital twin in Ditto's database.
+
+#### Ensure live processing -- live-response
+
+Set `response-required: true` on live commands or live messages. This automatically adds
+`live-response` to `requested-acks`. The acknowledgement is fulfilled when a subscriber sends a
+live response or message response.
+
+#### Ensure subscriber consumption -- custom labels
+
+Request a [custom acknowledgement label](#custom-acknowledgement-labels) to confirm that an
+external subscriber processed the resulting event or live command. The subscriber must
+[issue the acknowledgement](#issuing-acknowledgements) via WebSocket or a connection source.
## Weak Acknowledgements (WACKs)
-Since there are scenarios where the subscriber of events or live messages has defined an RQL filter or is not allowed to receive an event by a policy, it is not always possible that an acknowledgement can be provided.
-To avoid that a command fails because of a missing acknowledgement for those reasons, we introduced weak acknowledgements.
+When a subscriber has an RQL filter that excludes an event, or when a policy prevents delivery,
+Ditto cannot obtain a real acknowledgement. To prevent commands from failing due to missing acks
+in these cases, Ditto issues **weak acknowledgements** automatically.
-These weak acknowledgements are issued automatically by ditto, in case a message or an event is filtered by a subscriber which declared to provide one or more of the requested acknowledgements for the command.
-A weak acknowledgement can be identified by checking the header with value `ditto-weak-ack`.
-Weak acknowledgements have this header set to `true`.
+Weak acknowledgements are issued when a subscriber that declared one or more of the requested
+acknowledgement labels filters out the event or message (e.g., via an RQL filter on a connection
+target or a policy restriction).
+
+Identify weak acknowledgements by checking the `ditto-weak-ack` header:
+
+```json
+{
+ "my-connection:my-ack": {
+ "status": 200,
+ "headers": {
+ "correlation-id": "...",
+ "ditto-weak-ack": true
+ }
+ }
+}
+```
-These weak acknowledgements do not cause redelivery of messages consumed by a Connection.
+Weak acknowledgements do **not** cause redelivery of messages consumed by a connection.
## Interaction between headers
-Three headers control how Ditto responds to a command: `response-required`, `requested-acks`, `timeout`.
-* `response-required`: `true` or `false`.
- It governs whether the user gets a (detailed) reply.
- In case of a live message or a live command it also has impact on the `requested-acks`:
- * If `response-required` is `true`, the acknowledgement label `live-response` will be added to `requested-acks` if not
- present and `requested-acks` was not explicitly set to an empty JSON array.
- * If it is `false`, the acknowledgement label `live-response` will be removed from `requested-acks` if present.
-* `requested-acks`: JSON array of acknowledgement requests.
- It determines the content of the response and transport-layer message settlement.
-* `timeout`: Duration.
- It governs how long Ditto waits for responses and acknowledgements.
-
-It is considered a client error if `timeout` is set to `0s` while `response-required` is `true` or `requested-acks` is
-nonempty.
+
+Three headers control how Ditto responds: `response-required`, `requested-acks`, and `timeout`.
+
+* **response-required**: `true` or `false`. Controls whether the caller gets a detailed reply. For live messages/commands, it also affects `requested-acks`:
+ * `true`: adds `live-response` to `requested-acks` if not present (unless `requested-acks` was explicitly set to empty).
+ * `false`: removes `live-response` from `requested-acks`.
+* **requested-acks**: JSON array of acknowledgement labels. Determines response content and transport-layer settlement.
+* **timeout**: Duration. How long Ditto waits for responses and acknowledgements.
+
+It is a client error to set `timeout` to `0s` while `response-required` is `true` or `requested-acks` is nonempty.
### Default header values
-Ditto set each of the three headers `response-required`, `requested-acks`, `timeout` to a default value according to any
-values of the other two headers set by the user.
-The default values depend only on headers set by the user; they do not depend on each other.
-Setting the default header values this way never produces any combination considered a client error unless the headers
-set by the user already cause a client error.
-| Header | Default value | Default value if all three headers are not set |
-| --- | --- | --- |
-| response-required | `false` if `timeout` is zero or `requested-acks` is empty, `true` otherwise | `true` |
-| requested-acks | `empty` if `timeout` is zero or `response-required` is `false`, the channel's default acknowledgement request otherwise |`["twin-persisted"]` for TWIN channel, `["live-response"]` for LIVE channel |
-| timeout | `60s` | `60s` |
+Ditto sets defaults for each header based on the other headers:
-The following sections show how each Ditto API interprets the three headers.
+| Header | Default value | Default (all unset) |
+|---|---|---|
+| response-required | `false` if timeout is zero or requested-acks is empty; `true` otherwise | `true` |
+| requested-acks | empty if timeout is zero or response-required is false; channel default otherwise | `["twin-persisted"]` (TWIN) / `["live-response"]` (LIVE) |
+| timeout | `60s` | `60s` |
### HTTP
-Since an HTTP response always follows an HTTP request, the header `response-required` is interpreted as whether
-the user wants a *detailed* response.
-If it is set to `false`, the HTTP response consists of status line and headers without body, or with a minimal body
-containing other status codes.
-If acknowledgements are requested, the HTTP response is delayed until all requested acknowledgements are received.
-Generally, if a request cannot be answered within the defined timeout, the HTTP response has status code 408.
-A response containing successful acknowledgements (2xx) and at least one failed acknowledgement (4xx) has status code
-424 (failed dependency).
-In this case the status codes of all acknowledgements should be check to determine the one which caused the failure.
-
-| API | response-required | requested-acks | timeout | Outcome |
-| --- | --- | --- | --- | --- |
-| HTTP | false | empty | zero | 202 Accepted immediately |
-| HTTP | false | empty | non-zero | 202 Accepted immediately |
-| HTTP | false | non-empty | zero | 400 Bad Request: timeout may not be zero if acknowledgements are requested |
-| HTTP | false | non-empty | non-zero | 202 Accepted after receiving the requested acknowledgements |
-| HTTP | true | empty | zero | 400 Bad Request: timeout may not be zero if response is required |
-| HTTP | true | empty | non-zero | Response |
-| HTTP | true | non-empty | zero | 400 Bad Request: timeout may not be zero if response is required |
-| HTTP | true | non-empty | non-zero | Aggregated response and acknowledgements |
+
+| response-required | requested-acks | timeout | Outcome |
+|---|---|---|---|
+| false | empty | zero | 202 Accepted immediately |
+| false | empty | non-zero | 202 Accepted immediately |
+| false | non-empty | zero | 400 Bad Request |
+| false | non-empty | non-zero | 202 Accepted after receiving acks |
+| true | empty | zero | 400 Bad Request |
+| true | empty | non-zero | Response |
+| true | non-empty | zero | 400 Bad Request |
+| true | non-empty | non-zero | Aggregated response and acks |
### WebSocket
-In the absence of client errors, a response is sent for a command if and only if `response-required` is set to `true`.
-Ditto supports no transport-layer message settlement for WebSocket; acknowledgements are only received as text frames.
-Consequently, it is considered a client error to have non-empty `requested-acks` while `response-required` is set to
-`false`.
-
-| API | response-required | requested-acks | timeout | Outcome |
-| --- | --- | --- | --- | --- |
-| WebSocket | false | empty | zero | No response |
-| WebSocket | false | empty | non-zero | No response |
-| WebSocket | false | non-empty | zero | Error: timeout may not be zero if acknowledgements are requested |
-| WebSocket | false | non-empty | non-zero | Error: WebSocket cannot send acknowledgements without a response |
-| WebSocket | true | empty | zero | Error: timeout may not be zero if response is required |
-| WebSocket | true | empty | non-zero | Response |
-| WebSocket | true | non-empty | zero | Error: timeout may not be zero if response is required |
-| WebSocket | true | non-empty | non-zero | Aggregated response and acknowledgements |
+
+| response-required | requested-acks | timeout | Outcome |
+|---|---|---|---|
+| false | empty | zero | No response |
+| false | empty | non-zero | No response |
+| false | non-empty | zero | Error |
+| false | non-empty | non-zero | Error: cannot send acks without response |
+| true | empty | zero | Error |
+| true | empty | non-zero | Response |
+| true | non-empty | zero | Error |
+| true | non-empty | non-zero | Aggregated response and acks |
### Connectivity
-For any incoming supported signal through a connection source, the header `response-required` determines whether a
-response message is published at the reply-target of the source.
-The header `requested-acks` determines the transport-layer message settlement and the content of any response message
-published at the reply-target.
-Examples of transport-layer message settlement mechanisms are AMQP 0.9.1 consumer acknowledgement mode, AMQP 1.0
-disposition frames, and MQTT PUBACK/PUBREC/PUBREL messages for incoming PUBLISH with QoS 1 or 2.
-
-| API | response-required | requested-acks | timeout | Outcome |
-| --- | --- | --- | --- | --- |
-| Connectivity | false | empty | zero | Nothing published at reply-target; message settled immediately |
-| Connectivity | false | empty | non-zero | Nothing published at reply-target; message settled immediately |
-| Connectivity | false | non-empty | zero | Error published at reply-target: timeout may not be zero if acknowledgements are requested; message settled negatively |
-| Connectivity | false | non-empty | non-zero | Nothing published at reply-target; message settled after receiving the requested acknowledgements |
-| Connectivity | true | empty | zero | Error published at reply-target: timeout may not be zero when response is required; message settled negatively |
-| Connectivity | true | empty | non-zero | Response published at reply-target; message settled immediately |
-| Connectivity | true | non-empty | zero | Error published at reply-target: timeout may not be zero if response is required; message settled negatively |
-| Connectivity | true | non-empty | non-zero | Aggregated response and acknowledgements published at reply-target; message settled after receiving the requested acknowledgements |
+
+| response-required | requested-acks | timeout | Outcome |
+|---|---|---|---|
+| false | empty | zero | Nothing published; settled immediately |
+| false | empty | non-zero | Nothing published; settled immediately |
+| false | non-empty | zero | Error; settled negatively |
+| false | non-empty | non-zero | Nothing published; settled after acks |
+| true | empty | zero | Error; settled negatively |
+| true | empty | non-zero | Response published; settled immediately |
+| true | non-empty | zero | Error; settled negatively |
+| true | non-empty | non-zero | Aggregated response and acks published; settled after acks |
+
+## Further reading
+
+- [Acknowledgement protocol specification](protocol-specification-acks.html) -- Ditto Protocol format for acks
+- [Protocol examples](protocol-examples.html#acknowledgements-acks) -- sample ack messages
+- [Connections](basic-connections.html) -- configuring ack requests and issuance on connections
diff --git a/documentation/src/main/resources/pages/ditto/basic-apis.md b/documentation/src/main/resources/pages/ditto/basic-apis.md
index 9d6fbc655e9..ec4c1ab2bfb 100644
--- a/documentation/src/main/resources/pages/ditto/basic-apis.md
+++ b/documentation/src/main/resources/pages/ditto/basic-apis.md
@@ -5,48 +5,4 @@ tags: [model]
permalink: basic-apis.html
---
-Ditto provides two ways to interact with:
-
-* A [REST-like HTTP API](httpapi-overview.html) with a sophisticated resource layout that allows to create, read,
- update and delete Things and the Thing's data.
-* A JSON-based [WebSocket API](httpapi-protocol-bindings-websocket.html) implementing the
- [Ditto Protocol](protocol-overview.html).
-
-
-## HTTP API or WebSocket?
-
-The two ways are **almost equally powerful** and allow the same operations to work with the Thing's data, send
-messages to Things and receive messages from Things.
-
-* The lightweight REST-like HTTP API can be used
- * on less powerful devices lacking a Java runtime or supporting other (scripting) languages like JavaScript, Python, C/C++,
- * and for developing Web-based user interfaces.
-* The WebSocket API proves useful for
- * gathering data streams from devices or massive data from another message broker,
- * real-time device monitoring,
- * event-driven Web applications,
- * full duplex communication scenarios, etc.
-
-
-## Comparison by feature
-
-| Feature | Ditto Protocol over WebSocket | REST-like HTTP API |
-|---------|--------------------------------|---------------------------|
-| Things management | ✓ | ✓ |
-| Features management | ✓ | ✓ |
-| Search Things | ✓ | ✓ |
-| Count Things | no | ✓ |
-| Messages | ✓ | ✓ |
-| Change notifications | ✓ | ✓ (SSEs) |
-| Access control via Policy | ✓ | ✓ (v2 only) |
-
-
-## Further aspects in which the interfaces differ
-
-| Criteria | Ditto Protocol over WebSocket | REST-like HTTP API |
-|---------------------|---------------------------------|---------------------------|
-| Programming language | Almost any web-oriented programming language, e.g. Java, JavaScript, .NET | Almost any programming language, e.g. Java, JavaScript, NodeJS, .NET, Python, C/C++ |
-| Connection paradigm | Connection-oriented with an always open and persistent connection with only one-time handshake overhead for lowest latency and highest throughput | Connectionless protocol with lower permanent resource allocation on sporadic transactions |
-| Channel security | WSS: WebSocket over Transport Layer Security | HTTPS: HTTP over Transport Layer Security |
-| Message exchange pattern | Non-blocking request - asynchronous response | Blocking request - response |
-| Authentication mechanism | User authentication using: HTTP BASIC Authentication, JSON Web Token (JWT) issued OpenID connect provider | User authentication using: HTTP BASIC Authentication, JSON Web Token (JWT) issued OpenID connect provider |
+For a comprehensive overview of all Ditto APIs and interfaces, see the [API Overview](httpapi-overview.html).
diff --git a/documentation/src/main/resources/pages/ditto/basic-auth-checkpermissions.md b/documentation/src/main/resources/pages/ditto/basic-auth-checkpermissions.md
index f64e0e84d72..493c5819419 100644
--- a/documentation/src/main/resources/pages/ditto/basic-auth-checkpermissions.md
+++ b/documentation/src/main/resources/pages/ditto/basic-auth-checkpermissions.md
@@ -1,48 +1,102 @@
---
-title: Checking Permissions for Resources
-keywords: permissions, authorization, resources, policy, checkPermissions
-tags: [model]
-permalink: basic-auth-checkpermissions.html
+title: Checking Permissions for Resources
+keywords: permissions, authorization, resources, policy, checkPermissions
+tags: [model]
+permalink: basic-auth-checkpermissions.html
---
-The `/checkPermissions` endpoint allows clients to validate permissions for specified entities on various resources, verifying access rights as defined in Ditto's policies.
+The `/checkPermissions` endpoint lets you verify whether the current user has specific permissions
+on specific resources -- without modifying any data.
-## Overview
+{% include callout.html content="**TL;DR**: POST a JSON object to `/checkPermissions` listing the resources and
+permissions you want to check. Ditto returns `true` or `false` for each one." type="primary" %}
-The `/checkPermissions` endpoint is part of Ditto's HTTP API, enhancing its policy-based authorization system by enabling permission validation checks on resources without modifying them.
-This functionality is valuable for UI-driven applications, where permissions checks can determine whether certain UI elements should be displayed or disabled based on the user’s access rights.
+## When to use this
-## Request Structure
+Permission checks are useful for:
-Submit a `POST` request with a JSON payload specifying entities, resources, and permissions:
+* **UI applications** -- determine whether to show or disable buttons based on the user's access
+ rights
+* **Pre-flight checks** -- verify that a batch operation will succeed before starting it
+* **Debugging** -- confirm that a Policy grants the expected permissions
+
+## How it works
+
+Send a `POST` request to the `/checkPermissions` endpoint with a JSON body. Each key in the JSON
+object names a check, and each value specifies the resource, entity, and required permissions:
```json
{
- "entity_name": {
- "resource": "thing:/features/lamp/properties/on",
- "entityId": "org.eclipse.ditto:some-thing-1",
- "hasPermissions": ["READ"]
- },
- "another_entity": {
- "resource": "message:/features/lamp/inbox/messages/toggle",
- "entityId": "org.eclipse.ditto:some-thing-2",
- "hasPermissions": ["WRITE"]
- }
+ "canReadLampState": {
+ "resource": "thing:/features/lamp/properties/on",
+ "entityId": "org.eclipse.ditto:some-thing-1",
+ "hasPermissions": ["READ"]
+ },
+ "canToggleLamp": {
+ "resource": "message:/features/lamp/inbox/messages/toggle",
+ "entityId": "org.eclipse.ditto:some-thing-1",
+ "hasPermissions": ["WRITE"]
+ },
+ "canEditPolicy": {
+ "resource": "policy:/",
+ "entityId": "org.eclipse.ditto:some-policy-1",
+ "hasPermissions": ["READ", "WRITE"]
+ }
}
```
-## Fields
-- entity_name: Identifier for the entity performing the action.
-- resource: Path of the target resource, starting with thing:, message:, or policy: followed by a valid resource path.
-- entityId: Unique identifier for the entity, such as a thingId or policyId, depending on the resource.
-- hasPermissions: Array of required permissions, such as READ or WRITE.
-## Response Structure
-The response indicates permission status for each entity and resource, returning a JSON object mapping entities to true (authorized) or false (unauthorized) values.
+### Request fields
+
+| Field | Description |
+|-------|-------------|
+| *(key)* | A name you choose to identify this check in the response |
+| `resource` | The resource path to check. Starts with `thing:`, `message:`, or `policy:` followed by a valid resource path. |
+| `entityId` | The ID of the entity (Thing ID or Policy ID, depending on the resource type) |
+| `hasPermissions` | An array of permissions to check: `READ`, `WRITE`, and/or `EXECUTE` |
+
+## Response
+
+Ditto returns a JSON object mapping each check name to `true` (authorized) or `false`
+(not authorized):
```json
{
- "entity_name": true,
- "another_entity": false
+ "canReadLampState": true,
+ "canToggleLamp": true,
+ "canEditPolicy": false
}
```
-This endpoint is especially useful for applications requiring quick permission validation for multiple entities across various resources.
\ No newline at end of file
+
+## Example
+
+Check whether the authenticated user can read a Thing's temperature and send a reset message:
+
+```bash
+curl -u ditto:ditto -X POST -H 'Content-Type: application/json' -d '{
+ "readTemp": {
+ "resource": "thing:/features/temperature/properties/value",
+ "entityId": "com.example:sensor-1",
+ "hasPermissions": ["READ"]
+ },
+ "sendReset": {
+ "resource": "message:/inbox/messages/reset",
+ "entityId": "com.example:sensor-1",
+ "hasPermissions": ["WRITE"]
+ }
+}' 'http://localhost:8080/api/2/checkPermissions'
+```
+
+Response:
+
+```json
+{
+ "readTemp": true,
+ "sendReset": false
+}
+```
+
+## Further reading
+
+* [Authentication & Authorization](basic-auth.html) -- how Ditto authenticates requests
+* [Policies](basic-policy.html) -- define the permissions that this endpoint checks
+* [HTTP API reference](http-api-doc.html) -- full API documentation
diff --git a/documentation/src/main/resources/pages/ditto/basic-auth.md b/documentation/src/main/resources/pages/ditto/basic-auth.md
index 70ed406806e..d61f71c2fc1 100644
--- a/documentation/src/main/resources/pages/ditto/basic-auth.md
+++ b/documentation/src/main/resources/pages/ditto/basic-auth.md
@@ -1,71 +1,92 @@
---
-title: Authentication and authorization
+title: Authentication & Authorization
keywords: auth, authentication, authorization, policies, policy, sso, single sign on
tags: [model]
permalink: basic-auth.html
---
-You can integrate your solutions with Ditto
+Ditto protects every API request with authentication (verifying identity) and authorization
+(checking permissions).
-* via the [HTTP API](http-api-doc.html) or
-* via WebSocket.
+{% include callout.html content="**TL;DR**: Ditto authenticates requests via pre-authentication (for example, nginx
+basic auth) or JWT tokens from OpenID Connect providers. Authorization is enforced through
+[Policies](basic-policy.html) that map authenticated subjects to fine-grained permissions." type="primary" %}
-On all APIs Ditto protects functionality and data by using
+## Authentication
-* **Authentication** to make sure the requester is the one he/she claims to be,
-* **Authorization** to make sure the requester is allowed to see, use or change the information he wants to access.
+Every request to Ditto's [HTTP API](http-api-doc.html) or WebSocket API must carry valid
+credentials. Ditto supports two authentication mechanisms:
-## Authentication
+### Pre-authentication
+
+An HTTP reverse proxy (like nginx) in front of Ditto authenticates the user and passes the
+verified identity to Ditto. The default Docker deployment uses nginx with HTTP Basic Authentication.
+
+See the [pre-authentication configuration guide](operating-authentication.html#pre-authentication)
+for setup details.
+
+### JWT (OpenID Connect)
-User authentication at the HTTP API
+Ditto accepts JWT tokens
+from Google and other OpenID Connect providers. You configure trusted providers in the Ditto
+installation, and Ditto validates the token signature and claims on each request.
-A user who calls the HTTP API can be authenticated using two mechanisms:
+See the [OpenID Connect configuration guide](operating-authentication.html#openid-connect) for
+setup details.
-* Pre-authentication by an HTTP reverse proxy in front of Ditto, e.g. doing HTTP BASIC Authentication by providing
- username and password as [documented in the installation/operation guide](installation-operating.html#pre-authentication).
-* A JWT issued by Google or other
- OpenID Connect providers as [documented in the installation/operation guide](installation-operating.html#openid-connect).
+## Authenticated subjects
-### Authenticated subjects
+Every request processed by Ditto carries one or more **authenticated subjects**. A subject
+identifies the requester and takes the form `:`:
-Every request to one of Ditto's API is done in scope of already authenticated subjects.
-This authentication may be provided via nginx (like mentioned [above](#authentication)), a
-JWT or in a connection via the
-configured `authorizationContext` in scope of the connection's [authorization](basic-connections.html#authorization).
+| Example | Source |
+|---------|--------|
+| `nginx:ditto` | nginx pre-authentication |
+| `google:1234567890` | Google JWT |
+| `my-keycloak:user-uuid` | Custom OpenID Connect provider |
-For each of the possibilities of authenticating subjects, the [command](basic-signals-command.html) or
-[message](basic-messages.html) processed by Ditto will contain one or more of the "authenticated subjects" which e.g.
-might be user IDs.
+These subjects are matched against [Policy](basic-policy.html) entries to determine what the
+requester can read, write, or execute.
+For connections, the subjects come from the connection's configured
+[authorization context](basic-connections.html#authorization).
-### Single sign-on (SSO)
+## Single sign-on (SSO)
-By configuring an arbitrary OpenID Connect provider (as mentioned above) it is possible for Ditto to participate in SSO
-for the following browser based requests:
-* [HTTP API](httpapi-overview.html) invocations
- * sending along a JWT token as `Authorization` header with `Bearer` value
-* Establishing a [WebSocket](httpapi-protocol-bindings-websocket.html) connection for bidirectional communication with
- Ditto via [Ditto Protocol](protocol-overview.html) JSON messages
- * sending along a JWT token as `Authorization` header with `Bearer` value (recommended)
- * sending along a JWT token as query parameter `access_token` (use only if the websocket client does not
- support setting http headers e.g. plain WebSocket API of browsers)
-* Opening a [Server sent event](httpapi-sse.html) connection in order to receive change notifications of twins in the
+By configuring an OpenID Connect provider, Ditto participates in single sign-on flows for
+browser-based applications:
+
+* **HTTP API** -- send the JWT as an `Authorization: Bearer ` header
+* **WebSocket** -- send the JWT as an `Authorization: Bearer ` header (recommended), or as
+ the `access_token` query parameter (use only if your WebSocket client does not support custom
+ headers, for example the plain browser WebSocket API)
+* **Server Sent Events** -- pass `withCredentials: true` when creating the `EventSource` in the
browser
- * passing the `withCredentials: true` option when creating the SSE in the browser
## Authorization
-Authorization is implemented with a Policy
-(in API version 2).
+Once Ditto identifies the authenticated subjects, it checks them against the
+Policy
+attached to the target resource. The Policy determines whether the subjects have the required
+permissions (`READ`, `WRITE`, or `EXECUTE`) on the requested resource path.
+
+See [Policies](basic-policy.html) for the full authorization model.
+
+### Authorization context in DevOps commands
+
+When using [DevOps commands](operating-devops.html), you pass an
+`authorizationContext` that must contain a subject known to Ditto's authentication layer.
-Please find details at [Policies](basic-policy.html).
+For the Docker quickstart deployment, use `nginx:ditto` -- this makes DevOps commands execute
+with the same identity as HTTP requests from the `ditto` user:
-### Authorization Context in DevOps Commands
+```text
+nginx:ditto
+```
-An `authorizationContext` which is passed to [DevOps Commands](installation-operating.html#devops-commands) needs
-to be a subject known to Ditto's authentication. In the simplest case, it's `nginx:{username}` where `{username}` is a user
-that is configured for basic auth in the included nginx's `nginx.htpasswd` file (where the `nginx:` prefix comes from).
+## Further reading
-If you are using the provided docker quickstart example from [Getting Started](installation-running.html) you
-can simply use `nginx:ditto`, then the commands that are passed from the connection are executed as if they
-were issued via HTTP from the user `ditto`.
+* [Policies](basic-policy.html) -- define fine-grained access control
+* [Checking Permissions](basic-auth-checkpermissions.html) -- validate permissions without
+ modifying data
+* [Installation & Operation](installation-operating.html) -- configure authentication providers
diff --git a/documentation/src/main/resources/pages/ditto/basic-changenotifications.md b/documentation/src/main/resources/pages/ditto/basic-changenotifications.md
index 0a410343694..c5a7dbac22a 100644
--- a/documentation/src/main/resources/pages/ditto/basic-changenotifications.md
+++ b/documentation/src/main/resources/pages/ditto/basic-changenotifications.md
@@ -1,90 +1,89 @@
---
-title: Change notifications
+title: Change Notifications
keywords: change, event, feature, notification, thing, filtering, rql, push, subscribe, consume, enrich, extra
tags: [model, rql]
permalink: basic-changenotifications.html
---
-[Signals](basic-signals.html) already described what an [Event](basic-signals-event.html) in Ditto is.
-Events are emitted after an entity (either a **digital twin** or an actual device) was changed.
+Change notifications deliver [events](basic-signals-event.html) to your application whenever a
+digital twin or device state changes.
-At the Ditto API there are different ways for getting notified of such events:
-* Via the [WebSocket API](httpapi-protocol-bindings-websocket.html) a WebSocket client gets all Events the authenticated subject
- (e.g. a user) is [authorized](basic-auth.html) to receive as [Ditto Protocol](protocol-overview.html) messages.
-* Via [HTTP SSEs](httpapi-sse.html) a consumer of the SSE `EventSource` gets all Events the authenticated subject
- (e.g. a user) is [authorized](basic-auth.html) to receive directly in the format of the changed entity
- (e.g. as [Thing JSON](basic-thing.html#model-specification) format).
-* Via an established [connection](basic-connections.html) in the [connectivity](connectivity-overview.html) service
+{% include callout.html content="**TL;DR**: Subscribe to change notifications via WebSocket, Server Sent Events (SSE),
+or connections. Filter by namespace or RQL expression to receive only the events you care about." type="primary" %}
+## How to receive change notifications
-## Filtering
+Ditto publishes events through three channels. Each delivers events to authenticated subjects that
+have the required [authorization](basic-auth.html):
-In order to not get all of the events an [authenticated subject](basic-auth.html#authenticated-subjects)
-(e.g. a user added in nginx) is allowed to see, but to filter for specific criteria,
-events may be filtered on the Ditto backend side before they are sent to an event receiver.
+| Channel | Format | Use case |
+|---------|--------|----------|
+| [WebSocket API](httpapi-protocol-bindings-websocket.html) | [Ditto Protocol](protocol-overview.html) messages | Bidirectional communication from browser or backend clients |
+| [HTTP SSE](httpapi-sse.html) | Changed entity JSON (for example, [Thing JSON](basic-thing.html#model-specification)) | Lightweight, read-only streaming in browsers |
+| [Connections](basic-connections.html) | Configurable via [connectivity](connectivity-overview.html) | Server-to-server integration with message brokers |
-The above mentioned different APIs provide their own mechanisms on how to define such filters, but they all share the
-common functionality of based on which information events may be filtered.
+## Filtering
-{% include note.html content="All filters are specified in a URL query format, therefore their values should be URL
- encoded before sending them to the backend. The equal (=) and the ampersand (&) character must be encoded in any RQL
- filter!" %}
+You can filter events on the Ditto backend before they reach your application. Each API provides
+its own mechanism for specifying filters, but all support the same filter types.
-### By namespaces
+{% include note.html content="All filters are specified in URL query format, so their values should be URL-encoded.
+The equal sign (=) and ampersand (&) must be encoded in any RQL filter." %}
-Filtering may be done based on a namespace name. Each Ditto [Thing](basic-thing.html) has an ID containing a namespace
-(see also the conventions for a [Thing ID](basic-thing.html#thing-id)).
+### Filter by namespace
-By providing the `namespaces` filter, a comma separated list of which namespaces to include in the result, only Things
-in namespaces of interest are considered and thus only events of these Things are emitted at the API.
+Provide a comma-separated list of namespaces to receive events only from Things in those
+namespaces:
-For example, one would only subscribe for events occurring in 2 specific namespaces by defining:
-```
+```text
namespaces=org.eclipse.ditto.one,org.eclipse.ditto.two
```
-### By RQL expression
+### Filter by RQL expression
-If filtering by namespaces is not sufficient, Ditto also allows to provide an [RQL expression](basic-rql.html)
-specifying:
+For more granular control, use an [RQL expression](basic-rql.html) to filter based on:
-* a Thing payload based condition determining which events should be emitted and which should not
-* a filter based on the fields of the [Ditto Protocol](protocol-specification.html) message which should be filtered,
- e.g.:
- * using the `topic:action` placeholder as query property, filtering for lifecycle events (`created`, `deleted`)
- and other filter options on the [Ditto Protocol topic](protocol-specification-topic.html) can be done
- * using the `resource:path` placeholder as query property, filtering based on the affected
- [Ditto Protocol path](protocol-specification.html#path) of a Ditto Protocol message can be done
- * for all supported placeholders, please refer to the
- [placeholders documentation](basic-placeholders.html#scope-rql-expressions-when-filtering-for-ditto-protocol-messages)
+* **Thing data** -- filter on the modified values in the event payload
+* **Ditto Protocol fields** -- filter on message metadata using
+ [placeholders](basic-placeholders.html#scope-rql-expressions-when-filtering-for-ditto-protocol-messages):
+ * `topic:action` -- filter for lifecycle events (`created`, `deleted`)
+ * `resource:path` -- filter by the affected [resource path](protocol-specification.html#path)
-{% include note.html content="This filter is *by default* applied on the modified data of a Thing.
- Data which was not changed will only be considered when it was
- [enriched via \"extraFields\"](basic-enrichment.html)." %}
+{% include note.html content="The RQL filter applies to the *modified* data by default. Unchanged data is only
+considered when it has been [enriched via extraFields](basic-enrichment.html)." %}
-This provides the opportunity to formulate filters like the following:
+### Examples
-#### Examples
+Only emit events when `count` changes to a value greater than 42:
-Only emit events when attribute "count" was changed to a value greater than 42:
-```
+```text
filter=gt(attributes/count,42)
```
-Only emit events for Things starting with myThing when a feature "lamp" was modified:
-```
+Only emit events for Things starting with "myThing" when the "lamp" feature changes:
+
+```text
filter=and(like(thingId,"org.eclipse.ditto:myThing*"),exists(features/lamp))
```
-Only emit events when the attribute "manufacturer" was changed to starting with "ACME & Sons".
-The `&` must be escaped in that case:
-```
+Only emit events when `manufacturer` starts with "ACME & Sons" (note the encoded `&`):
+
+```text
filter=like(attributes/manufacturer,"ACME %26 Sons*")
```
Only emit events for Thing creation and deletion:
-```
+
+```text
filter=and(in(topic:action,'created','deleted'),eq(resource:path,'/'))
```
-You get the idea of how mighty this becomes by utilizing Ditto's [RQL expressions](basic-rql.html).
+See the full [RQL expression reference](basic-rql.html) for the complete query language.
+
+## Further reading
+
+* [Signal Enrichment](basic-enrichment.html) -- add extra context (like attributes) to events
+* [Signals & Communication Pattern](basic-signals.html) -- understand the signal types
+* [WebSocket Protocol Binding](httpapi-protocol-bindings-websocket.html) -- subscribe via WebSocket
+* [Server Sent Events](httpapi-sse.html) -- subscribe via SSE
+* [Connections](basic-connections.html) -- subscribe via managed connections
diff --git a/documentation/src/main/resources/pages/ditto/basic-conditional-requests.md b/documentation/src/main/resources/pages/ditto/basic-conditional-requests.md
index 3a49e99f210..bbfab54bdaa 100644
--- a/documentation/src/main/resources/pages/ditto/basic-conditional-requests.md
+++ b/documentation/src/main/resources/pages/ditto/basic-conditional-requests.md
@@ -5,41 +5,41 @@ tags: [protocol, http, rql]
permalink: basic-conditional-requests.html
---
-Ditto already supports [Conditional Requests](httpapi-concepts.html#conditional-requests) as defined in [RFC-7232](https://tools.ietf.org/html/rfc7232)
-where the `If-Match` and `If-None-Match` headers can be used to specify if a request should be applied or not.
-With the `condition` header it is possible to specify a condition based on the state of the actual thing.
-It is possible to combine both headers within one request. If you use both headers keep in mind that the ETag header is evaluated first.
+Conditional requests let you apply commands to Things only when specific conditions about the Thing's current state are met.
+
+{% include callout.html content="**TL;DR**: Use the `condition` header with an RQL expression to make updates conditional on the Thing's current state. You can also use `If-Match`/`If-None-Match` headers for ETag-based conditions, and `live-channel-condition` for automatic twin/live switching." type="primary" %}
+
+## Overview
+
+Ditto supports [conditional requests as defined in RFC-7232](https://tools.ietf.org/html/rfc7232) using `If-Match` and `If-None-Match` headers. Additionally, the `condition` header lets you specify conditions based on the current state of the persisted twin.
+
+You can combine both header types in one request. When you do, Ditto evaluates the ETag header first.
## Defining conditions
-Ditto supports retrieving, modifying, deleting and sending messages to/from things based on specific conditions of the
-current persisted twin state.
-For example, if you want to update the value of an attribute, but only if the current attribute value is not already 42,
-you can specify a condition:
+You define conditions using [RQL expressions](basic-rql.html). A condition specifies that Ditto should apply the request only if the expression evaluates to `true` against the current twin state.
-```
+For example, to update an attribute only if the current value is not already 42:
+
+```text
PUT /api/2/things/org.eclipse.ditto:foo1/attributes/value?condition=ne(attributes/value,42)
42
```
-Conditions are based on [RQL expressions](basic-rql.html) and define that a request should be applied to a thing
-only if the condition is met. It is possible to use any field in your thing to define a condition.
-E.g. you can use a timestamp in case you only want to change the state of the thing, if the provided value
-is newer than in the last state of the thing.
+You can reference any field in the Thing to build a condition. This is useful for timestamp-based updates where you only want to apply a change if the incoming value is newer than the stored one.
-* If the condition specified in the request is fulfilled, the thing will be updated and an event will be emitted.
-* If the condition specified in the request is not fulfilled, the thing is not modified, and no [event/change notification](basic-changenotifications.html) is emitted.
+- If the condition is met, Ditto updates the Thing and emits an event.
+- If the condition is not met, Ditto does not modify the Thing and emits no [event/change notification](basic-changenotifications.html).
-Conditional requests are supported by HTTP API, WebSocket, Ditto protocol and Ditto Java Client.
+Conditional requests work with HTTP API, WebSocket, Ditto Protocol, and the Ditto Java Client.
### Permissions for conditions
-READ permission is necessary on the resource specified in the condition, otherwise, the request will fail.
+You need READ permission on the resource referenced in the condition. Otherwise, the request fails.
## Examples
-In this part, we will show how to use conditional updates via HTTP API, Ditto protocol, and Ditto Java Client.
-The below examples assume that we have the following thing state:
+The following examples assume this Thing state:
```json
{
@@ -60,36 +60,29 @@ The below examples assume that we have the following thing state:
}
```
-In our example we want to update the **temperature** value, but only if the current value is newer than the already stored one.
-To express this condition, we use the _lastModified_ field in the temperature feature.
-
-In the following sections, we will show how request the conditional update via HTTP API, Ditto protocol,
-and Ditto Java Client which is based on the WebSocket protocol.
+The goal: update the temperature value only if the incoming value is newer than the stored one, using the `lastModified` field.
### HTTP API
-Using the HTTP API it is possible to specify the condition via query parameter
+Specify the condition as a query parameter:
-```
+```bash
curl -X PUT -H 'Content-Type: application/json' /api/2/things/org.eclipse.ditto:fancy-thing/features/temperature/properties/value?condition=gt(features/temperature/properties/lastModified,'2021-08-10T15:10:02.592Z') -d 19.26
```
-or via HTTP header
+Or as an HTTP header:
-```
+```bash
curl -X PUT -H 'Content-Type: application/json' -H 'condition: gt(features/temperature/properties/lastModified,"2021-08-10T15:10:02.592Z")' /api/2/things/org.eclipse.ditto:fancy-thing/features/temperature/properties/value -d 19.26
```
-### Ditto protocol
-
-The Ditto protocol supports also conditional updates.
-This is an example how to do a conditional update via [Ditto Protocol](protocol-specification.html) message:
+### Ditto Protocol
```json
{
"topic": "org.eclipse.ditto/fancy-thing/things/twin/commands/modify",
"headers": {
- "condition": "gt(features/temperature/properties/lastModified,2021-08-10T15:10:02.592Z)"
+ "condition": "gt(features/temperature/properties/lastModified,2021-08-10T15:10:02.592Z)"
},
"path": "/features/temperature/properties/value",
"value": 19.26
@@ -98,13 +91,10 @@ This is an example how to do a conditional update via [Ditto Protocol](protocol-
### Ditto Java Client
-The third option to use conditional updates is the ditto-client.
-The following code snippet demonstrates how to achieve this.
-
```java
final Option option =
Options.condition("gt(features/temperature/properties/lastModified,\"2021-08-10T15:10:02.592Z\")")
-
+
client.twin().forFeature(ThingId.of("org.eclipse.ditto:fancy-thing"), "temperature")
.putProperty("value", 42, option)
.whenComplete((ununsed, throwable) -> {
@@ -118,78 +108,51 @@ client.twin().forFeature(ThingId.of("org.eclipse.ditto:fancy-thing"), "temperatu
## Live channel condition
-Ditto also supports retrieving thing data with an automatic approach for switching between
-[twin](protocol-twinlive.html#twin) and [live](protocol-twinlive.html#live) channel.
+Ditto supports automatic switching between [twin](protocol-twinlive.html#twin-channel) and [live](protocol-twinlive.html#live-channel) channels based on a condition.
-Conditions are defined with RQL as described [before](#defining-conditions).
-If a condition is matched, the Thing data is retrieved from the device itself.
+Define the condition with RQL. If it matches, Ditto retrieves data from the device itself:
-Example: retrieve data from the device itself if a certain attribute is configured at the twin:
-```
+```text
GET .../things/{thingId}?live-channel-condition=eq(attributes/useLiveChannel,true)
```
-Example: retrieve data from the device itself if the last modification timestamp is behind a specified timestamp:
-```
+```text
GET .../things/{thingId}?live-channel-condition=lt(_modified,"2021-12-24T12:23:42Z")
```
-### Live channel condition headers
+### Timeout strategy
-Additionally, a strategy to handle timeouts for retrieving live thing data can be specified with the header
-`live-channel-timeout-strategy`.
-The header value holds a **strategy** what to do in case a timeout (can also be specified as header) was encountered.
+The `live-channel-timeout-strategy` header controls what happens when the device does not respond within the timeout:
-* If the value cannot be retrieved live from the device itself during the specified timeout, the request will
- `fail` (which is the default strategy if not specified otherwise) with a status code 408.
-* Alternatively, if `use-twin` was defined as the `live-channel-timeout-strategy` strategy, the request will fall back
- to the persisted twin and return the latest value stored in the digital twin.
+- `fail` (default): return status code `408`
+- `use-twin`: fall back to the persisted twin data
+### Response headers
-### Live channel condition response headers
+The response includes two headers indicating which channel was used:
-The response includes two additional headers to indicate which channel was used to retrieve the thing data:
+- `live-channel-condition-matched`: `true` or `false`
+- `channel`: `twin` or `live`
-* `live-channel-condition-matched` – value could be `true` or `false` and states whether the passed live-channel-condition was a match or not
-* `channel` – value could be `twin` or `live` and defines which channel was the origin of the returned data.
-
-In line with the procedure described above on how to retrieve live data directly from a device, a new type of
-pre-configured payload mapping, namely [UpdateTwinWithLiveResponse](connectivity-mapping.html#updatetwinwithliveresponse-mapper)
-was introduced.
-
-Upon activation, the digital twin stored in Eclipse Ditto will implicitly be updated with the latest data from the
-_live response_ sent by the device.
+You can use the [UpdateTwinWithLiveResponse](connectivity-mapping.html#updatetwinwithliveresponse-mapper) payload mapper to automatically update the digital twin with live response data.
## Path-specific conditions
-In addition to global conditions, Ditto supports since version 3.8.0 path-specific conditions for
-[merge operations](protocol-specification-things-merge.html) using the `merge-thing-patch-conditions` header.
-This feature allows you to apply different [RQL conditions](basic-rql.html) to different parts of a merge patch,
-enabling fine-grained control over which parts of the patch are applied based on the current state of the Thing.
-
-The `merge-thing-patch-conditions` header contains a JSON object where each key represents a JSON pointer path and each
-value is an RQL expression that must evaluate to `true` for that path to be included in the merge.
-
-The provided key (JSON pointer path) is relative to the modification `path` of the merge command.
-When e.g. providing a command which merges at path `/`, provided key to target a `value` property in a `temperature`
-feature would have the format `features/temperature/properties/value`.
-Issuing a merge command to merge at the feature level `features/temperature` would require the key to be `properties/value`
-instead.
+Since Ditto 3.8.0, you can apply different conditions to different parts of a [merge operation](protocol-specification-things.html#merge-commands) using the `merge-thing-patch-conditions` header.
-* If a path-specific condition is fulfilled, that part of the merge patch will be applied.
-* If a path-specific condition is not fulfilled, that part of the merge patch will be skipped.
-* Paths without conditions will always be applied.
+The header contains a JSON object where each key is a JSON pointer path (relative to the merge command's `path`) and each value is an RQL expression. Paths without conditions are always applied.
-Path-specific conditions are supported by HTTP API, WebSocket, Ditto protocol and Ditto Java Client.
+- If a path-specific condition is met, that part of the merge patch is applied.
+- If a path-specific condition is not met, that part is skipped.
+- Paths without conditions are always applied.
### Permissions for path-specific conditions
-READ permission is necessary on all resources referenced in the path-specific conditions, otherwise, the request will fail.
+You need READ permission on all resources referenced in the conditions.
### Examples
-This part shows how to use path-specific conditions via HTTP API, Ditto protocol, and Ditto Java Client.
-The below examples assume that there is a thing with the following thing state:
+Given this Thing state:
```json
{
@@ -227,9 +190,7 @@ The below examples assume that there is a thing with the following thing state:
#### HTTP API
-Using the HTTP API it is possible to specify path-specific conditions via HTTP header `merge-thing-patch-conditions`:
-
-```shell
+```bash
curl -X PATCH -H 'Content-Type: application/merge-patch+json' \
-H 'merge-thing-patch-conditions: {"features/temperature/properties/value": "gt(features/temperature/properties/value,20)", "features/humidity/properties/value": "lt(features/humidity/properties/value,80)"}' \
http://localhost:8080/api/2/things/org.eclipse.ditto:fancy-thing \
@@ -237,24 +198,21 @@ curl -X PATCH -H 'Content-Type: application/merge-patch+json' \
```
In this example:
-- `temperature` will only be updated if the current temperature is greater than 20 (condition fails, so temperature won't be updated)
-- `humidity` will only be updated if the current humidity is less than 80 (condition fails, so humidity won't be updated)
-- `status` will always be updated (no condition specified)
+- `temperature` is not updated (15 is not > 20)
+- `humidity` is not updated (90 is not < 80)
+- `status` is always updated (no condition)
-The same call could also be done on `/features` path with adjusted condition keys.
-Notice that the conditions are still relative to the root of the thing, only the key paths are adjusted accordingly:
+The same call can also target the `/features` path with adjusted condition keys.
+Notice that conditions are still relative to the root of the Thing, only the key paths are adjusted:
-```shell
+```bash
curl -X PATCH -H 'Content-Type: application/merge-patch+json' \
-H 'merge-thing-patch-conditions: {"temperature/properties/value": "gt(features/temperature/properties/value,20)", "humidity/properties/value": "lt(features/humidity/properties/value,80)"}' \
http://localhost:8080/api/2/things/org.eclipse.ditto:fancy-thing/features \
-d '{"temperature": {"properties": {"value": 25}}, "humidity": {"properties": {"value": 60}}, "status": {"properties": {"state": "updated"}}}'
```
-#### Ditto protocol
-
-The Ditto protocol supports also path-specific conditions for merge operations.
-This is an example how to do a conditional merge via [Ditto Protocol](protocol-specification.html) message:
+#### Ditto Protocol
```json
{
@@ -268,21 +226,9 @@ This is an example how to do a conditional merge via [Ditto Protocol](protocol-s
"lastMaintenance": "2023-01-20T15:00:00Z"
},
"features": {
- "temperature": {
- "properties": {
- "value": 25
- }
- },
- "humidity": {
- "properties": {
- "value": 60
- }
- },
- "status": {
- "properties": {
- "state": "updated"
- }
- }
+ "temperature": { "properties": { "value": 25 } },
+ "humidity": { "properties": { "value": 60 } },
+ "status": { "properties": { "state": "updated" } }
}
}
}
@@ -290,9 +236,6 @@ This is an example how to do a conditional merge via [Ditto Protocol](protocol-s
#### Ditto Java Client
-The third option to use path-specific conditions is the [ditto-client](client-sdk-java.html).
-The following code snippet demonstrates how to achieve this.
-
```java
Map cond = new HashMap<>();
cond.put("features/temperature/properties/value", "gt(features/temperature/properties/value,20)");
@@ -308,19 +251,13 @@ client.twin().forId(ThingId.of("org.eclipse.ditto:fancy-thing"))
.set("features", JsonObject.newBuilder()
.set("temperature", JsonObject.newBuilder()
.set("properties", JsonObject.newBuilder()
- .set("value", 25)
- .build())
- .build())
+ .set("value", 25).build()).build())
.set("humidity", JsonObject.newBuilder()
.set("properties", JsonObject.newBuilder()
- .set("value", 60)
- .build())
- .build())
+ .set("value", 60).build()).build())
.set("status", JsonObject.newBuilder()
.set("properties", JsonObject.newBuilder()
- .set("state", "updated")
- .build())
- .build())
+ .set("state", "updated").build()).build())
.build())
.build(), condOption)
.whenComplete((unused, throwable) -> {
@@ -332,63 +269,68 @@ client.twin().forId(ThingId.of("org.eclipse.ditto:fancy-thing"))
});
```
-### Configuration for path-specific conditions
-
-When using path-specific conditions, it's possible that all parts of a merge payload are filtered out, resulting in
-empty JSON objects being persisted to the database.
-Ditto provides a configuration option to avoid persisting events in case that only empty objects are left after applying
-the conditions.
-
-#### Empty object removal
+### Empty object removal configuration
**Configuration option:** `MERGE_REMOVE_EMPTY_OBJECTS_AFTER_PATCH_CONDITION_FILTERING`
-**Default behavior:** `false` (empty objects are preserved for backward compatibility)
+**Default**: `false` (empty objects preserved for backward compatibility).
-**When enabled:** Empty objects created by patch condition filtering are removed recursively, preventing unnecessary
-database operations and saving database I/O.
+When enabled, Ditto recursively removes empty JSON objects created by condition filtering. This
+prevents unnecessary database operations when all parts of a merge patch are filtered out.
-**Example scenario - Complete filtering:**
+**Example scenario** -- given this Thing state:
```json
-// Current Thing state
{
"features": {
- "temp": {"properties": {"value": 15}}, // Current: 15
- "hum": {"properties": {"value": 70}} // Current: 70
+ "temp": { "properties": { "value": 15 } },
+ "hum": { "properties": { "value": 70 } }
}
}
+```
-// Merge payload
+Merge payload:
+
+```json
{
"features": {
- "temp": {"properties": {"value": 25}},
- "hum": {"properties": {"value": 60}}
+ "temp": { "properties": { "value": 25 } },
+ "hum": { "properties": { "value": 60 } }
}
}
+```
+
+Patch conditions (both evaluate to `false`):
-// Patch conditions (both will evaluate to false)
+```json
{
- "features/temp/properties/value": "gt(features/temp/properties/value,30)", // 15 > 30 = false
- "features/hum/properties/value": "lt(features/hum/properties/value,50)" // 70 < 50 = false
+ "features/temp/properties/value": "gt(features/temp/properties/value,30)",
+ "features/hum/properties/value": "lt(features/hum/properties/value,50)"
}
+```
+
+**Result with configuration disabled** (default) -- empty objects preserved, database operation
+still occurs:
-// Result with configuration disabled (default)
+```json
{
"features": {
- "temp": {"properties": {}}, // Empty object preserved
- "hum": {"properties": {}} // Empty object preserved
+ "temp": { "properties": {} },
+ "hum": { "properties": {} }
}
}
-// → Database operation still occurs, empty objects stored
+```
-// Result with configuration enabled
-{} // Completely empty payload
-// → Database operation skipped entirely, no storage, no event emission, no new revision
+**Result with configuration enabled** -- completely empty payload, database operation skipped
+entirely (no storage, no event emission, no new revision):
+
+```json
+{}
```
-For configuration details, see [Merge operations configuration](installation-operating.html#merge-operations-configuration).
+For configuration details, see [Merge operations configuration](operating-configuration.html#merge-operations-configuration).
-## Further reading on RQL expressions
+## Further reading
-See [RQL expressions](basic-rql.html).
+- [RQL expressions](basic-rql.html) -- query language reference
+- [Things specification](protocol-specification-things.html) -- Thing commands including merge
diff --git a/documentation/src/main/resources/pages/ditto/basic-connections.md b/documentation/src/main/resources/pages/ditto/basic-connections.md
index daedb7f27c7..493d1b73ab6 100644
--- a/documentation/src/main/resources/pages/ditto/basic-connections.md
+++ b/documentation/src/main/resources/pages/ditto/basic-connections.md
@@ -1,41 +1,39 @@
---
-title: Connections
-keywords: connection, connectivity, mapping, connection, integration, placeholder, qos, at least once, delivery, guarantee
+title: Connections Overview
+keywords: connection, connectivity, mapping, integration, placeholder, qos, at least once, delivery, guarantee
tags: [connectivity]
permalink: basic-connections.html
---
-## Connection model
+Connections let you integrate Ditto with external messaging systems so that devices can exchange data with their digital twins through protocols like AMQP, MQTT, HTTP, and Kafka.
- {%
- include note.html content="To get started with connections right away, consult the
- [Manage connections](connectivity-manage-connections.html) page. "
- %}
+{% include callout.html content="**TL;DR**: A connection is a managed communication channel between Ditto and an external system. You configure sources to consume inbound messages and targets to publish outbound messages, with authorization, enforcement, and payload mapping applied automatically." type="primary" %}
-You can integrate your Ditto instance with external messaging services such as
-[Eclipse Hono](https://eclipse.org/hono/), a [RabbitMQ](https://www.rabbitmq.com/) broker or an
-[Apache Kafka](https://kafka.apache.org/) broker via custom "connections".
+## Overview
-Additionally, you may invoke foreign HTTP endpoints by using the
-[HTTP connection type](connectivity-protocol-bindings-http.html).
+You integrate your Ditto instance with external messaging services -- such as
+[Eclipse Hono](https://eclipse.org/hono/), [RabbitMQ](https://www.rabbitmq.com/),
+[Apache Kafka](https://kafka.apache.org/), or any HTTP endpoint -- by creating connections.
-A connection represents a communication channel for the exchange of messages between any service and Ditto.
-It requires a transport protocol, which is used to transmit [Ditto Protocol](protocol-overview.html) messages.
-Ditto supports one-way and two-way communication over connections. This enables consumer/producer scenarios
-as well as fully-fledged command and response use cases. Nevertheless, those options might be limited by
-the transport protocol or the other endpoint's capabilities.
-
-All connections are configured and supervised via Ditto's
-[Connectivity service](architecture-services-connectivity.html). The following model defines the connection itself:
+A connection represents a communication channel that uses a transport protocol to transmit
+[Ditto Protocol](protocol-overview.html) messages. Ditto supports one-way and two-way communication,
+enabling consumer/producer scenarios as well as full command-and-response workflows.
-{% include docson.html schema="jsonschema/connection.json" %}
+All connections are configured and supervised by Ditto's
+[Connectivity service](architecture-services-connectivity.html).
+
+To create and manage connections, use the [HTTP API](connectivity-manage-connections.html) or
+[DevOps piggyback commands](connectivity-manage-connections-piggyback.html).
+## Connection model
+
+The following schema defines the connection model:
-### Connection types
+{% include docson.html schema="jsonschema/connection.json" %}
-The top design priority of this model is to be as generic as possible, while still allowing protocol specific
-customizations and tweaks. This enables the implementations of different customizable connection types, and support
-for custom payload formats. Currently, the following connection types are supported:
+## Supported connection types
+
+Ditto supports these connection types:
* [AMQP 0.9.1](connectivity-protocol-bindings-amqp091.html)
* [AMQP 1.0](connectivity-protocol-bindings-amqp10.html)
@@ -44,144 +42,105 @@ for custom payload formats. Currently, the following connection types are suppor
* [HTTP 1.1](connectivity-protocol-bindings-http.html)
* [Kafka 2.x](connectivity-protocol-bindings-kafka2.html)
-The `sources` and `targets` address formats depends on the `connectionType` and has therefore `connectionType`
-specific limitations. Those are documented with the corresponding protocol bindings.
+The format of `sources` and `targets` addresses depends on the `connectionType` and is documented
+in each protocol binding page.
+
+## Sources
-### Sources
+Sources consume messages **from** external systems. Inbound messages can be:
-Sources are used to connect to message brokers / external systems in order to consume messages **from them**.
+* [Commands](basic-signals-command.html)
+* [Messages](basic-messages.html)
+* [Live commands/responses/events](protocol-twinlive.html)
+* [Acknowledgements](protocol-specification-acks.html)
-Source messages can be of the following type:
-* [commands](basic-signals-command.html)
-* [messages](basic-messages.html)
-* [live commands/responses/events](protocol-twinlive.html)
-* [acknowledgements](protocol-specification-acks.html)
+A source contains:
-Sources contain:
-* several addresses (depending on the [connection type](#connection-types) those are interpreted differently,
- e.g. as queues, topics, etc.),
-* a consumer count defining how many consumers should be attached to each source address,
-* an authorization context (see [authorization](#authorization)) specifying which
- [authorization subject](basic-policy.html#subjects) is used to authorize messages from the source,
-* enforcement information that allows filtering the messages that are consumed in this source,
-* [acknowledgement requests](basic-acknowledgements.html#requesting-acks) this source requires in order
- to ensure QoS 1 ("at least once") processing of consumed messages before technically acknowledging them to the channel,
-* declared labels of [acknowledgements](protocol-specification-acks.html) the source is allowed to send,
-* [header mapping](connectivity-header-mapping.html) for mapping headers of source messages to internal headers, and
-* a reply-target to configure publication of any responses of incoming commands.
+* **addresses** -- interpreted as queues, topics, etc. depending on the [connection type](#supported-connection-types)
+* **consumerCount** -- how many consumers attach to each address
+* **authorizationContext** -- [authorization subjects](basic-policy.html#subjects) used to authorize inbound messages (see [Authorization](#authorization))
+* **enforcement** -- filters to verify that a device only modifies its own digital twin
+* **acknowledgementRequests** -- controls [QoS 1 processing](#source-acknowledgement-requests)
+* **declaredAcks** -- labels of [acknowledgements](protocol-specification-acks.html) this source may send
+* **headerMapping** -- maps external headers to internal headers (see [Header mapping](connectivity-header-mapping.html))
+* **replyTarget** -- where to publish responses to incoming commands
-#### Source enforcement
+### Source enforcement
-Messages received from external systems are mapped to Ditto internal format, either by applying some custom mapping or
-the default mapping for [Ditto Protocol](protocol-overview.html) messages.
+By default, Ditto does not verify whether the device identity in an inbound message matches the
+targeted thing. You can add enforcement to ensure a device only modifies its own digital twin.
-During this mapping the digital twin of the device is determined i.e.
-which thing is accessed or modified as a result of the message. By default, no sanity check is done if this target
-thing corresponds to the device that originally sent the message. In some use cases this might be valid, but
-in other scenarios you might want to enforce that a device only sends data to its digital twin.
-Note that this could also be achieved by assigning a specific policy to each device and use [placeholders](#placeholders)
-in the authorization subject, but this can get cumbersome to maintain for a large number of devices.
+Enforcement requires that the external system provides a verified device identity (for example, in a
+message header).
-With an enforcement, you can use a single policy for all devices
-and still make sure that a device only modifies its associated digital twin. Enforcement is only feasible if the message
-contains the verified identity of the sending device (e.g. in a message header). This verification has to be done by the
-external system e.g. by properly authenticating the devices and providing the identity in the messages sent to Ditto.
+The enforcement configuration has two fields:
-The enforcement configuration consists of two fields:
-* `input`: Defines where device identity is extracted.
-* `filters`: Defines the filters that are matched against the input. At least one filter must match the input value,
-otherwise the message is rejected.
+* `input` -- where the device identity is extracted from
+* `filters` -- patterns matched against the input; at least one must match or the message is rejected
-The following placeholders are available for the `input` field:
+**Placeholders for `input`:**
| Placeholder | Description | Example |
|-----------|-------|---------------|
-| `{%raw%}{{ header: }}{%endraw%}` | Any header from the message received via the source (case-insensitive). | `{%raw%}{{header:device_id }}{%endraw%}` |
-| `{%raw%}{{ source:address }}{%endraw%}` | The address on which the message was received. | devices/sensors/temperature1 |
-
+| `{%raw%}{{ header: }}{%endraw%}` | Any header from the received message (case-insensitive) | `{%raw%}{{header:device_id }}{%endraw%}` |
+| `{%raw%}{{ source:address }}{%endraw%}` | The address the message was received on | devices/sensors/temperature1 |
-The following placeholders are available for the `filters` field:
+**Placeholders for `filters`:**
| Placeholder | Description | Example |
|-----------|-------|---------------|
-| `{%raw%}{{ thing:id }}{%endraw%}` | Full ID composed of ''namespace'' + '':'' as a separator + ''name'' | eclipse.ditto:thing-42 |
-| `{%raw%}{{ thing:namespace }}{%endraw%}` | Namespace (i.e. first part of an ID) | eclipse.ditto |
-| `{%raw%}{{ thing:name }}{%endraw%}` | Name (i.e. second part of an ID ) | thing-42 |
+| `{%raw%}{{ thing:id }}{%endraw%}` | Full ID (namespace + name) | eclipse.ditto:thing-42 |
+| `{%raw%}{{ thing:namespace }}{%endraw%}` | Namespace (first part of ID) | eclipse.ditto |
+| `{%raw%}{{ thing:name }}{%endraw%}` | Name (second part of ID) | thing-42 |
+
+**Example:** Device `sensor:temperature1` provides its identity in a `device_id` header. To enforce
+that it can only write to its own twin:
-Assuming a device `sensor:temperature1` pushes its telemetry data to Ditto which is stored in a thing named
-`sensor:temperature1`. The device identity is provided in a header field `device_id`. To enforce that the device can
-only send data to the Thing `sensor:temperature1` the following enforcement configuration can be used:
```json
{
- "addresses": [ "telemetry/hono_tenant" ],
+ "addresses": ["telemetry/hono_tenant"],
"authorizationContext": ["ditto:inbound-auth-subject"],
"enforcement": {
"input": "{%raw%}{{ header:device_id }}{%endraw%}",
- "filters": [ "{%raw%}{{ thing:id }}{%endraw%}" ]
+ "filters": ["{%raw%}{{ thing:id }}{%endraw%}"]
}
}
```
-Note: This example assumes that there is a valid user named `ditto:inbound-auth-subject` in Ditto.
-If you want to use a user for the basic auth (from the [HTTP API](connectivity-protocol-bindings-http.html)) use
-the prefix `nginx:`, e.g. `nginx:ditto`.
-See [Basic Authentication](basic-auth.html#authorization-context-in-devops-commands) for more information.
-
-#### Source acknowledgement requests
+{% include note.html content="If you use basic auth from the HTTP API, prefix authorization subjects with `nginx:` (e.g., `nginx:ditto`). See [Basic Authentication](basic-auth.html#authorization-context-in-devops-commands)." %}
-A source can configure, that for each incoming message additional
-[acknowledgement requests](basic-acknowledgements.html#requesting-acks) are added.
+### Source acknowledgement requests
-That is desirable whenever incoming messages should be processed with a higher "quality of service" than the default,
-which is "at most once" (or QoS 0).
+To process inbound messages with "at least once" (QoS 1) semantics instead of the default "at most
+once" (QoS 0), configure `acknowledgementRequests/includes` to request the
+["twin-persisted"](basic-acknowledgements.html#built-in-acknowledgement-labels) acknowledgement.
+The message is then technically acknowledged only after the twin is successfully persisted.
-In order to process messages from sources with an "at least once" (or QoS 1) semantic, configure the source's
-`"acknowledgementRequests/includes"` to add the
-["twin-persisted"](basic-acknowledgements.html#built-in-acknowledgement-labels) acknowledgement request, which will
-cause that a consumed message over this source will technically be acknowledged, it the twin was
-successfully updated/persisted by Ditto.
+The optional `filter` field uses an [fn:filter()](basic-placeholders.html#function-library) expression
+to control when acknowledgements are requested:
-How the technical acknowledgment is done is specific for the used [connection type](#connection-types) and documented
-in scope of that connection type.
-
-In addition to the `"includes"` defining which acknowledgements to request for each incoming message, the optional
-`"filter"` holds an [fn:filter()](basic-placeholders.html#function-library) function defining when to request
-acknowledgements at all for an incoming message. This filter is applied on both acknowledgements: those
-[requested in the message](basic-acknowledgements.html#requesting-acks-via-ditto-protocol-message) and the ones requested
-via the configured `"includes"` array.
-
-The JSON for a source with acknowledgement requests could look like this. The `"filter"` in the example causes that
-acknowledgements are only requested if the "qos" header was either not present or does not equal `0`:
```json
{
- "addresses": [
- ""
- ],
+ "addresses": [""],
"authorizationContext": ["ditto:inbound-auth-subject"],
"headerMapping": {
"qos": "{%raw%}{{ header:qos }}{%endraw%}"
},
"acknowledgementRequests": {
- "includes": [
- "twin-persisted",
- "receiver-connection-id:my-custom-ack"
- ],
+ "includes": ["twin-persisted", "{%raw%}{{connection:id}}{%endraw%}:my-custom-ack"],
"filter": "fn:filter(header:qos,'ne','0')"
}
}
```
-#### Source declared acknowledgement labels
+### Source declared acknowledgement labels
+
+Acknowledgements sent via a source must have their labels declared in the `declaredAcks` array.
+Labels must be prefixed by the connection ID (or `{%raw%}{{connection:id}}{%endraw%}`) followed by a colon:
-The acknowledgements sent via a source must have their labels declared in the field `declardAcks` as a JSON array.
-If the label of an acknowledgement is not in the `declaredAcks` array, then the acknowledgement is rejected with
-an error. The declared labels must be prefixed by the connection ID followed by a colon or the
-`{%raw%}{{connection:id}}{%endraw%}` placeholder followed by a colon. For example:
```json
{
- "addresses": [
- ""
- ],
+ "addresses": [""],
"authorizationContext": ["ditto:inbound-auth-subject"],
"declaredAcks": [
"{%raw%}{{connection:id}}{%endraw%}:my-custom-ack"
@@ -189,19 +148,14 @@ an error. The declared labels must be prefixed by the connection ID followed by
}
```
-#### Source header mapping
+### Source header mapping
-For incoming messages, an optional [header mapping](connectivity-header-mapping.html) may be applied.
-Mapped headers are added to the headers of the Ditto protocol message obtained by payload mapping.
-The default [Ditto payload mapper](connectivity-mapping.html#ditto-mapper) does not retain any external header;
-in this case all Ditto protocol headers come from the header mapping.
+You can apply an optional [header mapping](connectivity-header-mapping.html) to inbound messages.
+Mapped headers are added to the Ditto protocol message produced by payload mapping:
-The JSON for a source with header mapping could look like this:
```json
{
- "addresses": [
- ""
- ],
+ "addresses": [""],
"authorizationContext": ["ditto:inbound-auth-subject"],
"headerMapping": {
"correlation-id": "{%raw%}{{ header:message-id }}{%endraw%}",
@@ -210,87 +164,45 @@ The JSON for a source with header mapping could look like this:
}
```
-#### Source reply target
+### Source reply target
+
+A source may define a reply target to publish responses to incoming commands. The reply target's
+address and header mapping are defined within the reply target, while its payload mapping is
+inherited from the parent source.
-A source may define a reply target to publish the responses of incoming commands.
-For a reply target, the address and header mapping are defined in itself, whereas its payload mapping is inherited
-from the parent source, because a payload mapping definition specifies the transformation for both: incoming and outgoing
-messages.
+To publish responses at the address from the incoming command's `reply-to` header, configure source
+header mapping and reply target together. If an incoming command lacks the `reply-to` header, no
+response is published:
-For example, to publish responses at the target address equal to the `reply-to` header of incoming commands,
-define source header mapping and reply target as follows. If an incoming command does not have the `reply-to` header,
-then its response is dropped.
```json
{
"headerMapping": {
"reply-to": "{%raw%}{{ header:reply-to }}{%endraw%}"
},
- "replyTarget": {
- "enabled": true,
- "address": "{%raw%}{{ header:reply-to }}{%endraw%}"
- }
-}
-```
-
-The reply target may contain its own header mapping (`"headerMapping"`) in order to map response headers.
-
-In addition, the reply target contains the expected response types (`"expectedResponseTypes"`) which should be
-published to the reply target.
-The following reply targets are available to choose from:
-* **response**: Send back successful responses (e.g. responses after a Thing was successfully modified,
- but also responses for [query commands](basic-signals-command.html#query-commands)).
- Includes positive [acknowledgements](protocol-specification-acks.html#acknowledgements-aggregating).
-* **error**: Send back error responses (e.g. thing not modifiable due to lacking permissions)
-* **nack**: If negative [acknowledgement](protocol-specification-acks.html#acknowledgements-aggregating) responses should be delivered.
-
-This is an example `"replyTarget"` containing both header mapping and expected response types:
-```json
-{
"replyTarget": {
"enabled": true,
"address": "{%raw%}{{ header:reply-to }}{%endraw%}",
"headerMapping": {
"correlation-id": "{%raw%}{{ header:correlation-id }}{%endraw%}"
},
- "expectedResponseTypes": [
- "response",
- "error",
- "nack"
- ]
+ "expectedResponseTypes": ["response", "error", "nack"]
}
}
```
+The `expectedResponseTypes` control which responses are published:
-Additionally,
-sources can be configured with special response diversion headers to redirect responses to other connections.
-
-#### Response diversion configuration
-
-Sources can configure [response diversion](connectivity-response-diversion.html)
-to redirect responses to different connections instead of the configured reply target.
-This is done through special header mapping keys:
-
-```json
-{
- "headerMapping": {
- "divert-response-to-connection": "target-connection-id",
- "divert-expected-response-types": "response,error,nack"
- }
-}
-```
+* **response** -- successful responses and positive acknowledgements
+* **error** -- error responses
+* **nack** -- negative acknowledgements
-Where:
-- `divert-response-to-connection`: Target connection ID for diversion
-- `divert-expected-response-types`: Comma-separated list of response types to divert
+### Response diversion
-This enables advanced multi-protocol workflows and response routing scenarios.
-See the [Response diversion](connectivity-response-diversion.html) documentation for detailed configuration examples
-and use cases.
+Sources can redirect responses to different connections instead of the configured reply target using
+special header mapping keys. See [Response diversion](connectivity-response-diversion.html) for details.
-## Sources examples with response diversion
+**Static response diversion** -- redirect all responses to a fixed connection:
-### Static response diversion
```json
{
"addresses": ["commands/sensor"],
@@ -306,29 +218,31 @@ and use cases.
}
```
-### Dynamic response diversion
+Where:
+- `divert-response-to-connection`: Target connection ID for diversion
+- `divert-expected-response-types`: Comma-separated list of response types to divert
+
+**Dynamic response diversion** -- route based on message content using a JavaScript payload mapper:
+
```json
{
"addresses": ["commands/+"],
"authorizationContext": ["ditto:device-commands"],
"headerMapping": {
- "divert-expected-response-types": "response,error"
+ "divert-expected-response-types": "response,error"
},
"payloadMapping": ["response-router"]
}
```
-With a corresponding JavaScript mapper that sets the target connection dynamically based on message content or headers.
```javascript
function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {
- // ... mapping logic ...
-
- let dittoHeaders = {
+ var parsedPayload = JSON.parse(textPayload);
+ var dittoHeaders = {
"correlation-id": headers["correlation-id"],
"divert-response-to": determineTargetConnection(headers, parsedPayload),
"divert-expected-response-types": "response,error"
};
-
return Ditto.buildDittoProtocolMsg(
namespace, name, group, channel, criterion, action, path,
dittoHeaders, value
@@ -346,48 +260,41 @@ function determineTargetConnection(headers, payload) {
}
```
+## Targets
-### Targets
-
-Targets are used to connect to messages brokers / external systems in order to publish messages **to them**.
+Targets publish messages **to** external systems. Outbound messages can be:
-Target messages can be of the following type:
-* [Thing messages](basic-messages.html)
* [Thing events](basic-signals-event.html)
-* [Thing live commands/responses/events](protocol-twinlive.html)
+* [Thing messages](basic-messages.html)
+* [Live commands/responses/events](protocol-twinlive.html)
* [Policy announcements](protocol-specification-policies-announcement.html)
* [Connection announcements](protocol-specification-connections-announcement.html)
-Targets contain:
-* one address (that is interpreted differently depending on the [connection type](#connection-types), e.g. as queue, topic, etc.),
-* [topics](#target-topics-and-filtering) that will be sent to the target,
-* an authorization context (see [authorization](#authorization)) specifying which
- [authorization subject](basic-policy.html#subjects) is used to authorize messages to the target, and
-* [header mapping](connectivity-header-mapping.html) to compute external headers from Ditto protocol headers.
-
+A target contains:
-#### Target topics and filtering
+* **address** -- interpreted as a queue, topic, etc. depending on the [connection type](#supported-connection-types)
+* **topics** -- which [message types](#target-topics-and-filtering) to publish
+* **authorizationContext** -- [authorization subjects](basic-policy.html#subjects) that must have READ permission
+* **headerMapping** -- maps Ditto protocol headers to external headers
-Which types of messages should be published to the target address, can be defined via configuration.
+### Target topics and filtering
-In order to only consume specific events like described in [change notifications](basic-changenotifications.html), the
-following parameters can additionally be provided when specifying the `topics` of a target:
+You define which message types to publish via the `topics` array. You can filter by
+[namespaces](basic-changenotifications.html#filter-by-namespace) and
+[RQL expressions](basic-changenotifications.html#filter-by-rql-expression):
-| Description | Topic | [Filter by namespaces](basic-changenotifications.html#by-namespaces) | [Filter by RQL expression](basic-changenotifications.html#by-rql-expression) |
-|-------------|-----------------|------------------|-----------|
-| Subscribe for [Thing events/change notifications](basic-changenotifications.html) | `_/_/things/twin/events` | ✔ | ✔ |
-| Subscribe for [Thing messages](basic-messages.html) | `_/_/things/live/messages` | ✔ | ✔ |
-| Subscribe for [Thing live commands](protocol-twinlive.html) | `_/_/things/live/commands` | ✔ | ❌ |
-| Subscribe for [Thing live events](protocol-twinlive.html) | `_/_/things/live/events` | ✔ | ✔ |
-| Subscribe for [Policy announcements](protocol-specification-policies-announcement.html) | `_/_/policies/announcements` | ✔ | ❌ |
-| Subscribe for [Connection announcements](protocol-specification-connections-announcement.html) | `_/_/connections/announcements` | ❌ | ❌ |
+| Topic | Namespace filter | RQL filter |
+|-------|:---:|:---:|
+| `_/_/things/twin/events` | ✔ | ✔ |
+| `_/_/things/live/messages` | ✔ | ✔ |
+| `_/_/things/live/commands` | ✔ | ❌ |
+| `_/_/things/live/events` | ✔ | ✔ |
+| `_/_/policies/announcements` | ✔ | ❌ |
+| `_/_/connections/announcements` | ❌ | ❌ |
-The parameters are specified similar to HTTP query parameters, the first one separated with a `?` and all following ones
-with `&`. You need to URL-encode the filter values before using them in a configuration.
+Filter parameters use HTTP query parameter syntax (`?` for the first, `&` for subsequent). URL-encode
+filter values before using them:
-For example, this way the connection session would register for all events in the namespace `org.eclipse.ditto` and which
-would match an attribute "counter" to be greater than 42. Additionally, it would subscribe to messages in the namespace
-`org.eclipse.ditto`:
```json
{
"address": "",
@@ -396,25 +303,28 @@ would match an attribute "counter" to be greater than 42. Additionally, it would
"_/_/things/twin/events?extraFields=attributes/placement&filter=gt(attributes/placement,'Kitchen')",
"_/_/things/live/messages?namespaces=org.eclipse.ditto"
],
- "authorizationContext": ["ditto:outbound-auth-subject", "..."]
+ "authorizationContext": ["ditto:outbound-auth-subject"]
}
```
-#### Target topics and enrichment
+### Target topics and enrichment
-When extra fields should be added to outgoing messages on a connection, an `extraFields` parameter can be added
-to the topic. This is supported for all topics:
+You can add extra fields to outgoing messages with the `extraFields` parameter.
+See [signal enrichment](basic-enrichment.html) for details.
-| Description | Topic | [Enrich by extra fields](basic-enrichment.html) |
-|-------------|-----------------|------------------|
-| Subscribe for [Thing events/change notifications](basic-changenotifications.html) | `_/_/things/twin/events` | ✔ |
-| Subscribe for [Thing messages](basic-messages.html) | `_/_/things/live/messages` | ✔ |
-| Subscribe for [Thing live commands](protocol-twinlive.html) | `_/_/things/live/commands` | ✔ |
-| Subscribe for [Thing live events](protocol-twinlive.html) | `_/_/things/live/events` | ✔ |
-| Subscribe for [Policy announcements](protocol-specification-policies-announcement.html) | `_/_/policies/announcements` | ❌ |
-| Subscribe for [Connection announcements](protocol-specification-connections-announcement.html) | `_/_/connections/announcements` | ❌ |
+Not all topics support enrichment:
+
+| Topic | Extra fields |
+|-------|:---:|
+| `_/_/things/twin/events` | ✔ |
+| `_/_/things/live/messages` | ✔ |
+| `_/_/things/live/commands` | ✔ |
+| `_/_/things/live/events` | ✔ |
+| `_/_/policies/announcements` | ❌ |
+| `_/_/connections/announcements` | ❌ |
Example:
+
```json
{
"address": "",
@@ -422,55 +332,34 @@ Example:
"_/_/things/twin/events?extraFields=attributes/placement",
"_/_/things/live/messages?extraFields=features/ConnectionStatus"
],
- "authorizationContext": ["ditto:outbound-auth-subject", "..."]
+ "authorizationContext": ["ditto:outbound-auth-subject"]
}
```
-#### Target issued acknowledgement label
+### Target issued acknowledgement label
-A target can be configured to automatically [issue acknowledgements](basic-acknowledgements.html#issuing-acknowledgements)
-for each published/emitted message, once the underlying channel confirmed
-that the message was successfully received.
+A target can automatically [issue an acknowledgement](basic-acknowledgements.html#issuing-acknowledgements)
+once the channel confirms successful delivery. The label must be prefixed by the connection ID
+or `{%raw%}{{connection:id}}{%endraw%}`:
-That is desirable whenever outgoing messages (e.g. [events](basic-signals-event.html)) are handled in scope of a command
-sent with an "at least once" (QoS 1) semantic in order to only acknowledge that command, if the event was successfully
-forwarded into another system.
-
-For more details on that topic, please refer to the [acknowledgements](basic-acknowledgements.html) section.
-
-Whether an outgoing message is treated as successfully sent or not is specific for the used
-[connection type](#connection-types) and documented in scope of that connection type.
-
-The issued acknowledgement label must be prefixed by the connection ID followed by a colon or the
-`{%raw%}{{connection:id}}{%endraw%}` placeholder followed by a colon.
-The JSON for a target with issued acknowledgement labels could look like this:
```json
{
"address": "",
- "topics": [
- "_/_/things/twin/events"
- ],
- "authorizationContext": ["ditto:inbound-auth-subject"],
+ "topics": ["_/_/things/twin/events"],
+ "authorizationContext": ["ditto:outbound-auth-subject"],
"issuedAcknowledgementLabel": "{%raw%}{{connection:id}}{%endraw%}:my-custom-ack"
}
```
-#### Target header mapping
+### Target header mapping
-For outgoing messages, an optional [header mapping](connectivity-header-mapping.html) may be applied.
-Mapped headers are added to the external headers.
-The default [Ditto payload mapper](connectivity-mapping.html#ditto-mapper) does not define any external header;
-in this case, all external headers come from the header mapping.
+You can apply an optional [header mapping](connectivity-header-mapping.html) to outgoing messages:
-The JSON for a target with header mapping could like this:
```json
{
"address": "",
- "topics": [
- "_/_/things/twin/events",
- "_/_/things/live/messages?namespaces=org.eclipse.ditto"
- ],
- "authorizationContext": ["ditto:inbound-auth-subject"],
+ "topics": ["_/_/things/twin/events"],
+ "authorizationContext": ["ditto:outbound-auth-subject"],
"headerMapping": {
"message-id": "{%raw%}{{ header:correlation-id }}{%endraw%}",
"content-type": "{%raw%}{{ header:content-type }}{%endraw%}",
@@ -480,107 +369,83 @@ The JSON for a target with header mapping could like this:
}
```
+## Authorization
-### Authorization
-
-A connection is initiated by the connectivity service. This obviates the need for client authorization, because
-Ditto becomes the client in this case. Nevertheless, to access resources within Ditto, the connection must know on
-whose behalf it is acting. This is controlled via the configured `authorizationContext`, which holds a list of
-self-assigned authorization subjects. Before a connection can access a Ditto resource, one of its
-`authorizationSubject`s must be granted the access rights by the authorization mechanism of a
+Ditto initiates connections as a client, so no client authorization is needed from the external
+system. However, to access Ditto resources, each connection must specify an `authorizationContext`
+with self-assigned authorization subjects. These subjects must be granted access through
[Policies](basic-policy.html).
-A connection target can only send data for things to which it has READ rights, as data flows from a thing to a target.
-A connection source can only receive data for things to which it has WRITE rights, as data flows from a source to a thing.
-
-### Specific configuration
-
-Some [connection types](#connection-types) require specific configuration, which is not supported for other connection types.
-Those are put into the `specificConfig` field.
-
-### Payload Mapping
-
-For more information on mapping message payloads see the corresponding [Payload Mapping Documentation](connectivity-mapping.html).
+* A **target** can only send data for things to which it has **READ** rights
+* A **source** can only receive data for things to which it has **WRITE** rights
## Placeholders
-The configuration of a connection allows to use placeholders at certain places. This allows more fine-grained control
-over how messages are consumed or where they are published to. The general syntax of a placeholder is
-`{% raw %}{{ placeholder }}{% endraw %}`. Have a look at the [placeholders concept](basic-placeholders.html) for
-more details on that.
+Connection configurations support placeholders with the syntax
+`{% raw %}{{ placeholder }}{% endraw %}`. See the [placeholders concept](basic-placeholders.html)
+for full details.
### Placeholder for source authorization subjects
-Processing the messages received via a source using the _same fixed authorization subject_ may not be
-suitable for every scenario. For example, if you want to declare fine-grained write permissions per device, this would
-not be possible with a fixed global subject. For this use case, we have introduced placeholder substitution for
-authorization subjects of source addresses that are resolved when processing messages from a source.
-Of course, this requires the sender of the
-message to provide necessary information about the original issuer of the message.
+You can use header placeholders in source authorization subjects to apply per-device permissions:
- {%
- include important.html content="Only use this kind of placeholder if you trust the source of the message. The value from the header is used as the **authorized subject**." additionalStyle=""
- %}
-
-You can access any header value of the incoming message by using a placeholder like `{% raw %}{{ header:name }}{% endraw %}`.
+{%
+ include important.html content="Only use this kind of placeholder if you trust the source of the message. The value from the header is used as the **authorized subject**." additionalStyle=""
+%}
-Example:
-
-Assuming the messages received from the source _telemetry_ contain a `device_id` header (e.g. _sensor-123_),
-you may configure your source's authorization subject as follows:
```json
- {
- "id": "auth-subject-placeholder-example",
- "sources": [
- {
- "addresses": [ "telemetry" ],
- "authorizationContext": ["device:{% raw %}{{ header:device_id }}{% endraw %}"]
- }
- ]
- }
+{
+ "id": "auth-subject-placeholder-example",
+ "sources": [{
+ "addresses": ["telemetry"],
+ "authorizationContext": ["device:{% raw %}{{ header:device_id }}{% endraw %}"]
+ }]
+}
```
-The placeholder is then replaced by the value from the message headers and the message is forwarded and processed under
-the subject _device:sensor-123_.
-In case the header cannot be resolved or the header contains unexpected characters, an exception is thrown, which is sent
-back to the sender as an error message, if a valid _reply-to_ header was provided, otherwise the message is dropped.
### Placeholder for target addresses
-Another use case for placeholders may be to publish twin events or live commands and events to a target address
-containing thing-specific information e.g. you can distribute things from different namespaces to different target addresses.
-You can use the placeholders `{% raw %}{{ thing:id }}{% endraw %}`, `{% raw %}{{ thing:namespace }}{% endraw %}`
-and `{% raw %}{{ thing:name }}{% endraw %}` in the target address for this purpose.
-For a thing with the ID _org.eclipse.ditto:device-123_ these placeholders would be resolved as follows:
+You can use thing placeholders in target addresses to route messages by namespace or device:
| Placeholder | Description | Resolved value |
|--------|------------|------------|
-| `thing:id` | Full ID composed of _namespace_ `:` (as a separator), and _name_ | _org.eclipse.ditto:device-123_ |
-| `thing:namespace` | Namespace (i.e. first part of an ID) | _org.eclipse.ditto_ |
-| `thing:name` | Name (i.e. second part of an ID ) | _device-123_ |
+| `thing:id` | Full ID (namespace:name) | `org.eclipse.ditto:device-123` |
+| `thing:namespace` | Namespace | `org.eclipse.ditto` |
+| `thing:name` | Name | `device-123` |
-Additionally to the placeholders mentioned above, all documented
-[connection placeholders](basic-placeholders.html#scope-connections) may be
-used in target addresses. However, if any placeholder in the target address fails to resolve, then the message will be
-dropped.
+All [connection placeholders](basic-placeholders.html#scope-connections) are also available.
+If any placeholder fails to resolve, the message is dropped.
-Example:
-
-Sending live commands and events to a target address that contains the thing's namespace.
```json
- {
- "id": "target-placeholder-example",
- "targets": [
- {
- "addresses": [ "live/{% raw %}{{ thing:namespace }}{% endraw %}" ],
- "authorizationContext": ["ditto:auth-subject"],
- "topics": [ "_/_/things/live/events", "_/_/things/live/commands" ]
- }
- ]
- }
+{
+ "id": "target-placeholder-example",
+ "targets": [{
+ "addresses": ["live/{% raw %}{{ thing:namespace }}{% endraw %}"],
+ "authorizationContext": ["ditto:auth-subject"],
+ "topics": ["_/_/things/live/events", "_/_/things/live/commands"]
+ }]
+}
```
+## Specific configuration
+
+Some connection types require protocol-specific settings in the `specificConfig` field. See each
+protocol binding page for details.
+
+## Payload mapping
+
+You can transform message payloads between external formats and Ditto Protocol using
+[payload mapping](connectivity-mapping.html).
+
## SSH tunneling
-Ditto supports tunneling a connection by establishing an SSH tunnel and using it to connect to the actual endpoint.
+Ditto supports tunneling connections through SSH. See [SSH tunneling](connectivity-ssh-tunneling.html)
+for setup instructions.
+
+## Further reading
-See [SSH tunneling](connectivity-ssh-tunneling.html) on how to setup and configure SSH tunneling with Ditto.
+* [Managing connections](connectivity-manage-connections.html) -- create, modify, and monitor connections
+* [Payload mapping](connectivity-mapping.html) -- transform message payloads
+* [Header mapping](connectivity-header-mapping.html) -- map external headers
+* [TLS certificates](connectivity-tls-certificates.html) -- secure connections with TLS
+* [Acknowledgements](basic-acknowledgements.html) -- configure delivery guarantees
diff --git a/documentation/src/main/resources/pages/ditto/basic-enrichment.md b/documentation/src/main/resources/pages/ditto/basic-enrichment.md
index 62ebe144ecf..9d050adb2ca 100644
--- a/documentation/src/main/resources/pages/ditto/basic-enrichment.md
+++ b/documentation/src/main/resources/pages/ditto/basic-enrichment.md
@@ -1,38 +1,49 @@
---
-title: Signal enrichment
+title: Signal Enrichment
keywords: change, event, enrich, extra, enrichment, fields, extraFields
tags: [protocol]
permalink: basic-enrichment.html
---
-[Signals](basic-signals.html) which are emitted to subscribers via [WebSocket API](httpapi-protocol-bindings-websocket.html),
-[HTTP SSEs](httpapi-sse.html) or established [connections](basic-connections.html) may be enriched
-by `extraFields` to also be included in the sent message.
+Signal enrichment lets you attach additional Thing data to events and messages when they are
+delivered to subscribers, so you get the context you need without making extra API calls.
-[Events](basic-signals-event.html), for example, only contain the actually changed data by default, so when they are
-subscribed to via one of the APIs listed above, the data they contain may be as sparse as:
-"temperature value was changed to 23.4 for the thing with ID xx".
+{% include callout.html content="**TL;DR**: Use `extraFields` when subscribing to events or messages to include
+additional Thing data (like attributes or related properties) in each delivered signal." type="primary" %}
-Often it is helpful to additionally include some extra fields as context to be included when subscribing
-(e.g. via WebSocket or a connection). For example in order to include static metadata stored in the `attributes`.
+## Why use enrichment?
-Therefore, it is possible to define `extraFields` to include when subscribing for:
-* [events/change notifications](basic-changenotifications.html)
-* [messages](basic-messages.html)
-* [live commands](protocol-twinlive.html)
-* [live events](protocol-twinlive.html)
+[Events](basic-signals-event.html) contain only the data that changed. A temperature update event
+might tell you: "the temperature of Thing X changed to 23.4". But your application might also
+need to know *where* that sensor is located or *what unit* the reading uses.
+
+Without enrichment, you would need a separate API call to look up that context. With enrichment,
+you define `extraFields` once when you subscribe, and Ditto includes those fields automatically
+in every delivered signal.
+
+You can enrich:
+
+* [Events / change notifications](basic-changenotifications.html)
+* [Messages](basic-messages.html)
+* [Live commands](protocol-twinlive.html)
+* [Live events](protocol-twinlive.html)
+
+## How to specify extraFields
+
+The `extraFields` parameter uses the same syntax as [field selectors](httpapi-concepts.html#field-selectors)
+for partial Thing retrieval. How you pass it depends on the API:
-How the `extraFields` are specified is depending on the API, please find the specific API information here:
* [WebSocket enrichment](httpapi-protocol-bindings-websocket.html#enrichment)
-* [SSE field enrichment](httpapi-sse.html#field-enrichment)
+* [SSE field enrichment](httpapi-sse.html#by-field-enrichment)
* [Connection target enrichment](basic-connections.html#target-topics-and-enrichment)
-The `extra` data is added to the [extra field in Ditto Protocol messages](protocol-specification.html#extra) being an
-JSON object containing all selected fields.
+The enriched data appears in the [extra field](protocol-specification.html#extra) of the Ditto
+Protocol message as a JSON object.
## Example
-For example a Thing could look like this:
+Consider a Thing with this structure:
+
```json
{
"thingId": "org.eclipse.ditto:fancy-thing",
@@ -44,7 +55,7 @@ For example a Thing could look like this:
"temperature": {
"properties": {
"value": 23.42,
- "unit": "Celcius"
+ "unit": "Celsius"
}
},
"humidity": {
@@ -57,8 +68,8 @@ For example a Thing could look like this:
}
```
-Now whenever its temperature is modified you normally only get the following information in the event
-(this is a [Ditto Protocol](protocol-specification.html) message):
+Without enrichment, a temperature change event contains only the changed value:
+
```json
{
"topic": "org.eclipse.ditto/fancy-thing/things/twin/events/modified",
@@ -69,17 +80,16 @@ Now whenever its temperature is modified you normally only get the following inf
}
```
-What you could want is to:
-* additionally add the `attributes`
-* additionally add the `unit` value of the temperature
+### Adding extra context
-In that case you would define to include `extraFields`
-(syntax is the same as for retrieving partial things with [field selector](httpapi-concepts.html#with-field-selector)):
-```
+To include the Thing's location and the temperature unit, subscribe with:
+
+```text
extraFields=attributes,features/temperature/properties/unit
```
-In that case, each emitted Ditto Protocol event would include an `extra` section containing the selected data:
+Each event now includes an `extra` section with the requested data:
+
```json
{
"topic": "org.eclipse.ditto/fancy-thing/things/twin/events/modified",
@@ -89,12 +99,12 @@ In that case, each emitted Ditto Protocol event would include an `extra` section
"revision": 34,
"extra": {
"attributes": {
- "location": "kitchen"
+ "location": "Kitchen"
},
"features": {
"temperature": {
"properties": {
- "unit": "Celcius"
+ "unit": "Celsius"
}
}
}
@@ -102,28 +112,45 @@ In that case, each emitted Ditto Protocol event would include an `extra` section
}
```
-It is possible to use the wildcard operator '*' as feature ID and add a property of multiple features
-(syntax is the same as for [field selector with wildcard](httpapi-concepts.html#field-selector-with-wildcard)).
-This would add the property 'unit' of all features:
-```
+### Using wildcards
+
+Use the wildcard `*` as a feature ID to include a property from all features:
+
+```text
extraFields=features/*/properties/unit
```
-If you however want to see a property only for the features changed within this event you could make use of placeholders.
-The following example would enrich the unit of all features that have changed within this event:
-```
+### Using placeholders
+
+Use placeholders to enrich only with data from the feature that triggered the event:
+
+```text
{%raw%}extraFields=features/{{feature:id}}/properties/unit{%endraw%}
```
-{% include note.html content="Please note that 'deleted' events cannot be enriched with the deleted values." %}
+{% include note.html content="Deleted events cannot be enriched with the deleted values." %}
-Please have a look at available placeholders for the use case:
-* [Signal enrichment for Websocket](basic-placeholders.html#scope-websocket-signal-enrichment)
+For the full list of available placeholders, see:
+* [Signal enrichment for WebSocket](basic-placeholders.html#scope-websocket-signal-enrichment)
* [Signal enrichment for SSE](basic-placeholders.html#scope-sse-signal-enrichment)
* [Signal enrichment for connections](basic-placeholders.html#scope-connections)
-## Enrich and filter
+## Enrichment with filtering
+
+You can combine `extraFields` with [event filtering](basic-changenotifications.html#filtering).
+Enriched data becomes available to the filter expression, so you can filter based on values that
+are not part of the change itself.
+
+For example, only receive temperature events for Things in the kitchen:
+
+```text
+extraFields=attributes/location&filter=eq(attributes/location,"Kitchen")
+```
+
+## Further reading
-In combination with [event filtering](basic-changenotifications.html#filtering) enriched data can also be used to
-filter. For example, when selecting `extraFields=attributes/location`, an additional `filter` may define to only
-emit events for a certain location: `extraFields=attributes/location&filter=eq(attributes/location,"kitchen")`.
+* [Change Notifications](basic-changenotifications.html) -- subscribe to events
+* [HTTP API concepts: field selectors](httpapi-concepts.html#field-selectors) -- syntax for
+ selecting fields
+* [Ditto Protocol extra field](protocol-specification.html#extra) -- where enriched data appears
+ in the protocol message
diff --git a/documentation/src/main/resources/pages/ditto/basic-errors.md b/documentation/src/main/resources/pages/ditto/basic-errors.md
index ce86ac8315a..67928af880e 100644
--- a/documentation/src/main/resources/pages/ditto/basic-errors.md
+++ b/documentation/src/main/resources/pages/ditto/basic-errors.md
@@ -5,57 +5,60 @@ tags: [model]
permalink: basic-errors.html
---
-Errors are datatypes containing information about occurred failures which were either
-cause by the user or appeared in the server.
+Errors are structured responses that describe failures caused by client mistakes or server problems.
-## Error model specification
+{% include callout.html content="**TL;DR**: Every error includes an HTTP `status` code, an `error` code string, a human-readable `message`, and an optional `description` with resolution hints. Use the `status` code for programmatic handling -- the `error` code string may change without notice." type="primary" %}
+
+## Overview
+
+When a Ditto operation fails, Ditto returns an error response containing details about what went wrong and how you might fix it.
+
+## Error model
{% include docson.html schema="jsonschema/error.json" %}
### Status
-The "status" uses HTTP status codes semantics (see [RFC 7231](https://tools.ietf.org/html/rfc7231#section-6))
-to indicate whether a specific command has been successfully completed, or not.
+The `status` field uses HTTP status code semantics (see [RFC 7231](https://tools.ietf.org/html/rfc7231#section-6)) to indicate whether a command succeeded or failed.
-These "status" codes can be seen as API/contract which will be always the same for a specific error.
-Use the "status" in order to identify an error, as the additional "error" and "description" might change
-without prior notice.
+The `status` code is part of Ditto's stable API. Use it to identify and handle errors programmatically.
-### Error
+### Error code
-A Ditto error contains an "error" code which is a string identifier that uniquely identifies the error.
+The `error` field contains a string identifier that uniquely identifies the error type (e.g., `things:attribute.notfound`).
-These error codes Ditto provides in addition to the HTTP **status** code are not to be considered as API and must
-therefore not be relied on.
-They might change without prior notice.
+These error codes are **not** part of the stable API and may change without notice. Do not rely on them for programmatic error handling.
-Ditto itself uses the following prefixes for its error codes:
+Ditto uses these prefixes for its error codes:
-* `things:` - for errors related to [things](basic-thing.html)
-* `policies:` - for errors related to [policies](basic-policy.html)
-* `things-search:` - for errors related to the [things search](basic-search.html)
-* `acknowledgement:` - for errors related to [acknowledgements](basic-acknowledgements.html)
-* `messages:` - for errors related to [messages](basic-messages.html)
-* `placeholder:` - for errors related to [placeholders](basic-placeholders.html)
-* `jwt:` - for errors related to JWT based [authentication](basic-auth.html)
-* `gateway:` - for errors produced by the (HTTP/WS) [gateway](architecture-services-gateway.html) service
-* `connectivity:` - for errors produced by the [connectivity](architecture-services-connectivity.html) service
+| Prefix | Domain |
+|---|---|
+| `things:` | [Things](basic-thing.html) |
+| `policies:` | [Policies](basic-policy.html) |
+| `things-search:` | [Things search](basic-search.html) |
+| `acknowledgement:` | [Acknowledgements](basic-acknowledgements.html) |
+| `messages:` | [Messages](basic-messages.html) |
+| `placeholder:` | [Placeholders](basic-placeholders.html) |
+| `jwt:` | JWT-based [authentication](basic-auth.html) |
+| `gateway:` | [Gateway](architecture-services-gateway.html) service |
+| `connectivity:` | [Connectivity](architecture-services-connectivity.html) service |
### Message
-The error "message" contains a short message describing the encountered problem in plain english text.
+The `message` field contains a short, human-readable description of the problem.
### Description
-The optional error "description" describes in more detail how the error could be resolved.
+The optional `description` field provides more detail on how to resolve the error.
### Href
-The optional href contains a link to Ditto documentation or external resources in order to help to resolve the error.
-
+The optional `href` field links to Ditto documentation or external resources that help resolve the error.
## Examples
+A "not found" error:
+
```json
{
"status": 404,
@@ -65,6 +68,8 @@ The optional href contains a link to Ditto documentation or external resources i
}
```
+An "invalid ID" error:
+
```json
{
"status": 400,
@@ -75,3 +80,7 @@ The optional href contains a link to Ditto documentation or external resources i
}
```
+## Further reading
+
+- [Protocol errors](protocol-specification-errors.html) -- error format in Ditto Protocol messages
+- [Protocol specification](protocol-specification.html) -- the full message format reference
diff --git a/documentation/src/main/resources/pages/ditto/basic-feature.md b/documentation/src/main/resources/pages/ditto/basic-feature.md
index b87beea9ed1..c32805bc116 100644
--- a/documentation/src/main/resources/pages/ditto/basic-feature.md
+++ b/documentation/src/main/resources/pages/ditto/basic-feature.md
@@ -1,89 +1,139 @@
---
-title: Feature
+title: Features
keywords: definition, properties, desiredProperties, entity, feature, functionblock, informationmodel, model
tags: [model]
permalink: basic-feature.html
---
-A Feature is used to manage all data and functionality of a Thing that can be clustered in an outlined technical
-context.
+A Feature groups related state data and capabilities of a [Thing](basic-thing.html) under a named
+identifier -- for example, a "temperature" feature on a weather station or a "lamp" feature on a
+smart light.
-For different contexts or aspects of a Thing different Features can be used which are all belonging to the same Thing
-and do not exist without this Thing.
+{% include callout.html content="**TL;DR**: A Feature has an ID, properties (current state), optional desired
+properties (target state), and an optional definition that links it to a type model." type="primary" %}
-## Model specification
+## How Features work
-The feature model in API version 2:
+Each Feature within a Thing represents one functional aspect of the device. A smart thermostat
+might have these Features:
-{% include docson.html schema="jsonschema/feature_v2.json" %}
+* `temperature` -- current and target temperature readings
+* `humidity` -- current humidity level
+* `schedule` -- heating schedule configuration
+
+You access each Feature independently through the API. For example, to read the current temperature:
+
+```bash
+curl -u ditto:ditto -X GET \
+ 'http://localhost:8080/api/2/things/com.example:thermostat-1/features/temperature/properties/value'
+```
+## Model specification
+
+{% include docson.html schema="jsonschema/feature_v2.json" %}
## Feature ID
-Within a Thing each Feature is identified by a unique string - the so-called Feature ID.
-A Feature ID often needs to be set in the path of an HTTP request. Due to this fact we strongly recommend using a
-restricted set of characters (e.g. those for [Uniform Resource Identifiers (URI)](https://www.ietf.org/rfc/rfc3986.txt)).
+Every Feature within a Thing has a unique string identifier. Since Feature IDs often appear in HTTP
+request paths, use characters that are safe in
+[URIs](https://www.ietf.org/rfc/rfc3986.txt) -- letters, digits, hyphens, and underscores.
-The Feature ID may not be the wildcard operator `*` because it has a special meaning and can lead to unexpected
-results when retrieving or searching for things/features.
+The Feature ID must not be the wildcard operator `*`, because it has special meaning in queries
+and can produce unexpected results.
## Feature properties
-The **data** related to Features is managed in form of a **list of properties**. These properties can be categorized,
-e.g. to manage the status, the configuration or any fault information.
-Feature properties are represented as one JSON object.
-
-Each property itself can be either a simple/scalar value or a complex object; allowed is any JSON value.
+Properties hold the Feature's current state data as a JSON object. Each property can be a scalar
+value or a nested object -- any valid JSON value works:
+
+```json
+{
+ "properties": {
+ "value": 23.5,
+ "unit": "Celsius",
+ "location": {
+ "latitude": 48.1351,
+ "longitude": 11.5820
+ }
+ }
+}
+```
## Feature desired properties
-Desired properties represent the desired state of the properties. They are a tool to represent the desired target state
-of the properties.
-The **desiredProperties** related to Features are managed in form of a **list of properties**. These desired properties
-can be categorized, e.g. to manage the status, the configuration or any fault information.
-Feature desired properties are represented as one JSON object.
+Desired properties represent the **target state** you want the device to reach. They have the same
+structure as regular properties but describe what *should be* rather than what *is*.
-Each desired property itself can be either a simple/scalar value or a complex object; allowed is any JSON value.
+For example, you might set a desired temperature of 22.0 while the current temperature reads 19.5:
-Please note however, that besides persisting the desired properties, and indexing the fields for search requests, filtering
-etc. for the time being, Ditto does not implement their further processing. Such functionality will come with future releases.
+```json
+{
+ "properties": {
+ "value": 19.5
+ },
+ "desiredProperties": {
+ "value": 22.0
+ }
+}
+```
-## Feature definition
+Ditto persists desired properties and indexes them for search queries. The logic for reconciling
+desired state with actual state is up to your application or device firmware.
-Ditto supports specifying a definition for a feature in order to document how a feature's state is structured
-(in [properties](#feature-properties)), and which behavior/capabilities
-([messages related to features](basic-messages.html)) can be expected from such a feature.
+## Feature definition
-A feature's definition is a list of definition identifiers containing
-* either valid HTTP(s) URLs (e.g. in order to define that the Feature is described by [WoT (Web of Things) Thing Models](basic-wot-integration.html#thing-model-describing-a-ditto-feature))
-* or a *namespace*, *name* and *version* separated by colons: `::`
- * in that case the "knowledge" where to look up the definition must be managed somewhere else
+A definition links the Feature to a type model that documents the Feature's structure and
+capabilities. You specify definitions as a list containing:
-A definition can be seen as a type for features. The [properties](#feature-properties) of a
-feature containing a definition identifier `"org.eclipse.ditto:lamp:1.0.0"` can be expected to follow the structure
-described in the `lamp` type of namespace `org.eclipse.ditto` semantically versioned with version `1.0.0`.
+* **HTTP(s) URLs** -- for example, a [WoT (Web of Things) Thing Model](basic-wot-integration.html#thing-model-describing-a-ditto-feature)
+* **Identifier strings** in the format `::` -- for example,
+ `org.eclipse.ditto:lamp:1.0.0`
-{% include note.html content="Ditto does not contain a type system on its own and does not specify how to describe types.
- You may e.g. use the official [W3C Web of Things (WoT)](#the-link-to-w3c-wot-web-of-things) standard
- to describe data structures and supported messages of Ditto features." %}
+A definition acts as a type annotation. If a Feature has the definition `org.eclipse.ditto:lamp:1.0.0`,
+you can expect its properties to follow the structure described by version `1.0.0` of the "lamp" type
+in the `org.eclipse.ditto` namespace.
-Currently, Ditto **does not** ensure that the `properties` or `desiredProperties` of a feature or its supported messages
-must follow the type defined in the definition.
-This can be ensured e.g. before sending a property to Ditto or before sending a message.
+{% include note.html content="Ditto does not contain a built-in type system and does not validate that properties
+ match their definition. You can use the [W3C Web of Things (WoT)](#the-link-to-w3c-wot-web-of-things) standard
+ to describe data structures and supported messages for Ditto features." %}
### The link to W3C WoT (Web of Things)
-If a [feature definition](#feature-definition) has the form of an HTTP(s) URL, this URL pointing to a resource may be
-interpreted as the link to a [W3C WoT (Web of Things)](https://www.w3.org/TR/wot-thing-description11/)
-[Thing Model](https://www.w3.org/TR/wot-thing-description11/#thing-model) in [JSON-LD](https://www.w3.org/TR/json-ld11/)
-format.
-
-For a detailed explanation how WoT and its concepts link to Ditto, please consult the
-[dedicated WoT integration documentation](basic-wot-integration.html).
-
-
-# Example
+When a [feature definition](#feature-definition) is an HTTP(s) URL, that URL can point to a
+[W3C WoT Thing Model](https://www.w3.org/TR/wot-thing-description11/#thing-model) in
+[JSON-LD](https://www.w3.org/TR/json-ld11/) format.
+
+For a detailed explanation of how WoT concepts map to Ditto, see the
+[WoT integration documentation](basic-wot-integration.html).
+
+## Example
+
+A Thing with two Features -- one for a heating element, one for a switch:
+
+```json
+{
+ "heating-no1": {
+ "properties": {
+ "connected": true,
+ "temperature": 21.5
+ },
+ "desiredProperties": {
+ "temperature": 23.0
+ }
+ },
+ "switchable": {
+ "definition": [ "org.eclipse.ditto:Switcher:1.0.0" ],
+ "properties": {
+ "on": true,
+ "lastToggled": "2017-11-15T18:21Z"
+ }
+ }
+}
+```
+
+## Further reading
-Please inspect the following separate pages for the WoT integration and an example:
* [WoT (Web of Things) integration](basic-wot-integration.html)
* [WoT (Web of Things) example](basic-wot-integration-example.html)
+* [Things](basic-thing.html) -- the parent entity that contains Features
+* [Messages](basic-messages.html) -- send commands to specific Features
diff --git a/documentation/src/main/resources/pages/ditto/basic-history.md b/documentation/src/main/resources/pages/ditto/basic-history.md
index 8b6a019f7ce..3c0b8e01a20 100644
--- a/documentation/src/main/resources/pages/ditto/basic-history.md
+++ b/documentation/src/main/resources/pages/ditto/basic-history.md
@@ -5,171 +5,137 @@ tags: [history]
permalink: basic-history.html
---
-Starting with **Eclipse Ditto 3.2.0**, APIs for retrieving the history of the following entities is provided:
-* [things](basic-thing.html)
-* [policies](basic-policy.html)
-* [connections](basic-connections.html)
+Ditto provides APIs for retrieving the history of Things, Policies, and Connections, letting you inspect past states and stream modification events.
-The capabilities of these APIs are the following:
+{% include callout.html content="**TL;DR**: You can retrieve any entity at a specific revision or timestamp, and stream historical modification events for Things and Policies. Configure `history-retention-duration` to control how long historical data is kept." type="primary" %}
-| Entity | [Retrieving entity at a specific revision or timestamp](#retrieving-entity-from-history) | [Streaming modification events of an entity specifying from/to revision/timestamp](#streaming-historical-events-of-entity) |
-|------------|------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------|
-| Thing | ✓ | ✓ |
-| Policy | ✓ | ✓ |
-| Connection | ✓ | no |
+## Overview
+
+Since Eclipse Ditto 3.2.0, you can access the history of:
+
+* [Things](basic-thing.html)
+* [Policies](basic-policy.html)
+* [Connections](basic-connections.html)
+
+| Entity | Retrieve at revision/timestamp | Stream historical events |
+|---|---|---|
+| Thing | Yes | Yes |
+| Policy | Yes | Yes |
+| Connection | Yes | No |
{% include note.html content="Ditto's history API capabilities are not comparable with the features of a time series database.
E.g. no aggregations on or compactions of the historical data can be done." %}
+## Retrieving an entity from history
-## Retrieving entity from history
-
-Provides:
-* Finding out the state of an entity (thing, policy, connection) at a given:
- * revision number
- * timestamp
-* Retrieving "historical headers" persisted together with a modification (see [configuring historical headers to persist](#configuring-historical-headers-to-persist))
+You can retrieve the state of an entity (Thing, Policy, Connection) at a given revision number or timestamp. You can also retrieve "historical headers" persisted alongside a modification (see [configuring historical headers](#configuring-historical-headers-to-persist)).
-Target use cases:
-* Compare changes to entity (e.g. a connection) to a former state
- * In order to solve potential errors in e.g. policy or connection configuration
-* **Audit log**: Find out who (which subject) did a change to an entity
- * E.g. in order to find out who changed a policy/connection
- * Configure [which headers to persist as historical headers](#configuring-historical-headers-to-persist) to e.g. include the subject which did a modification
+Use cases:
+* Compare an entity's current state to a former state for debugging
+* **Audit log**: Find out which subject made a specific change
-### Retrieve entity at specific revision
+### Retrieve at a specific revision
-Retrieving an entity at an (historical) revision, set the header `at-historical-revision` to a `long` number for all
-"retrieve" commands of persisted state.
+Set the `at-historical-revision` header to a revision number on any retrieve command.
-Example for the HTTP API:
```bash
-# Access a thing:
+# Retrieve a Thing at revision 1:
curl -u ditto:ditto 'http://localhost:8080/api/2/things/org.eclipse.ditto:thing-1' \
--header 'at-historical-revision: 1'
-
-# Access a policy:
+
+# Retrieve a Policy at revision 1:
curl -u ditto:ditto 'http://localhost:8080/api/2/policies/org.eclipse.ditto:policy-1' \
--header 'at-historical-revision: 1'
-
-# Access a connection:
+
+# Retrieve a Connection at revision 1:
curl -u devops:foobar 'http://localhost:8080/api/2/connections/some-connection-1' \
--header 'at-historical-revision: 1'
```
-The same functionality is available via a [header of a Ditto Protocol](protocol-specification.html#headers) message.
+This functionality is also available via [Ditto Protocol headers](protocol-specification.html#headers).
-If [historical headers](#configuring-historical-headers-to-persist) were configured to be persisted, they can be found
-in the response header named `historical-headers`.
+If [historical headers](#configuring-historical-headers-to-persist) are configured, they appear in the response header `historical-headers`.
-### Retrieve entity at specific timestamp
+### Retrieve at a specific timestamp
-Retrieving an entity at an (historical) timestamp, set the header `at-historical-timestamp` to an ISO-8601 formatted
-`string` for all "retrieve" commands of persisted state.
+Set the `at-historical-timestamp` header to an ISO-8601 timestamp.
-Example for the HTTP API:
```bash
-# Access a thing:
+# Retrieve a Thing at a specific time:
curl -u ditto:ditto 'http://localhost:8080/api/2/things/org.eclipse.ditto:thing-1' \
--header 'at-historical-timestamp: 2022-10-24T03:11:15Z'
-
-# Access a policy:
+
+# Retrieve a Policy at a specific time:
curl -u ditto:ditto 'http://localhost:8080/api/2/policies/org.eclipse.ditto:policy-1' \
--header 'at-historical-timestamp: 2022-10-24T06:11:15Z'
-
-# Access a connection:
+
+# Retrieve a Connection at a specific time:
curl -u devops:foobar 'http://localhost:8080/api/2/connections/some-connection-1' \
--header 'at-historical-timestamp: 2022-10-24T07:11Z'
```
-The same functionality is available via a [header of a Ditto Protocol](protocol-specification.html#headers) message.
+## Streaming historical events
-If [historical headers](#configuring-historical-headers-to-persist) were configured to be persisted, they can be found
-in the response header named `historical-headers`.
+You can stream a sequence of modification events for a specific Thing or Policy. Specify the range by revision numbers or timestamps.
+Use cases:
+* Inspect how an entity changed over time
+* Display historical values on a chart
-## Streaming historical events of entity
+### Streaming via SSE
-Provides:
-* A stream of changes to a specific thing or policy, based on specified:
- * entity ID
- * start revision number (and optional stop revision number)
- * start timestamp (and optional stop timestamp)
-* Retrieving "historical headers" persisted together with a modification (see [configuring historical headers to persist](#configuring-historical-headers-to-persist))
+The [SSE (Server Sent Event) API](httpapi-sse.html) is the simplest way to stream historical events. It is available **for Things only**.
-Target use cases:
-* Inspect the changes of an entity over time
- * E.g. displaying a value on a chart with that way
+Use these query parameters:
-### Streaming historical events via SSE
+**Revision-based:**
+* `from-historical-revision`: Starting revision. Use negative values for relative offsets from the current revision.
+* `to-historical-revision`: Optional end revision. Use `0` for latest, or negative for relative offsets.
-The easiest way to stream historical events is the [SSE (Server Sent Event) API](httpapi-sse.html).
-This API is however **only available for things** (not for policies).
+**Timestamp-based:**
+* `from-historical-timestamp`: Starting timestamp (ISO-8601).
+* `to-historical-timestamp`: Optional end timestamp.
-Use the following query parameters in order to specify the start/stop revision/timestamp.
+Each historical event is normalized to the Thing JSON representation.
-Either use the revision based parameters:
-* `from-historical-revision`: Specifies the revision number to start streaming historical modification events from.
- May also be negative in order to specify to get the last `n` revisions relative to the **most recent revision** (`_revision` of the thing).
-* `to-historical-revision`: Optionally specifies the revision number to stop streaming at (if omitted, it streams events until the current state of the entity).
- May also be 0 or negative in order to specify to get either the latest (`0`) or the `n`th most recent revision.
-
-Alternatively, use the timestamp based parameters:
-* `from-historical-timestamp`: specifies the timestamp to start streaming historical modification events from
-* `to-historical-timestamp`: optionally specifies the timestamp to stop streaming at (if omitted, it streams events until the current state of the entity)
-
-The messages sent over the SSE are the same as for the [SSE (Server Sent Event) API](httpapi-sse.html), each historical
-modification event is "normalized" to the Thing JSON representation.
-
-Examples:
```bash
-# stream complete history starting from earliest available revision of a thing:
+# Stream complete history from earliest available revision:
curl --http2 -u ditto:ditto -H 'Accept:text/event-stream' -N \
http://localhost:8080/api/2/things/org.eclipse.ditto:thing-2?from-historical-revision=0&fields=thingId,attributes,features,_revision,_modified
-# stream specific history range of a thing based on revisions:
+# Stream a specific revision range:
curl --http2 -u ditto:ditto -H 'Accept:text/event-stream' -N \
http://localhost:8080/api/2/things/org.eclipse.ditto:thing-2?from-historical-revision=23&to-historical-revision=42&fields=thingId,attributes,features,_revision,_modified
-# stream specific history range of a thing based on timestamps:
+# Stream a specific timestamp range:
curl --http2 -u ditto:ditto -H 'Accept:text/event-stream' -N \
http://localhost:8080/api/2/things/org.eclipse.ditto:thing-2?from-historical-timestamp=2022-10-24T11:44:36Z&to-historical-timestamp=2022-10-24T11:44:37Z&fields=thingId,attributes,features,_revision,_modified
-# stream specific history range, additionally selecting _context in "fields" which contains the historical headers:
+# Include historical headers via the _context field:
curl --http2 -u ditto:ditto -H 'Accept:text/event-stream' -N \
http://localhost:8080/api/2/things/org.eclipse.ditto:thing-2?from-historical-revision=0&fields=thingId,attributes,features,_revision,_modified,_context
```
-#### Filtering streamed historical events for things via SEE
-
-When streaming historical events for [things](basic-thing.html), an optional `filter` in form of an
-[RQL](basic-rql.html) may be declared in order to only receive thing events matching the defined query.
+#### Filtering streamed events via SSE
-This can e.g. be useful to only stream events in which a certain feature or a certain property/attribute was included.
+Add a `filter` parameter with an [RQL](basic-rql.html) expression to only receive events matching the query:
-In addition to the parameters selecting from/to revision or timestamp, the following parameter can be defined:
-* `filter`: specifies the [RQL](basic-rql.html) filter which events to return in the stream must match
-
-Examples:
```bash
-# stream complete history starting from earliest available revision of a thing, but only those where a feature "bamboo" was modified:
+# Only events where a "bamboo" feature was modified:
curl --http2 -u ditto:ditto -H 'Accept:text/event-stream' -N \
http://localhost:8080/api/2/things/org.eclipse.ditto:thing-2?from-historical-revision=0&fields=thingId,attributes,features,_revision,_modified&filter=exists(features/bamboo)
-# stream specific history range of a thing based on timestamps, filtering for temperature values of a sensor being greater than 50:
+# Only events where temperature exceeded 50:
curl --http2 -u ditto:ditto -H 'Accept:text/event-stream' -N \
http://localhost:8080/api/2/things/org.eclipse.ditto:thing-2?from-historical-timestamp=2022-10-24T11:44:36Z&to-historical-timestamp=2022-10-24T11:44:37Z&fields=thingId,attributes,features,_revision,_modified&filter=gt(features/temperature/properties/value,50)
```
-### Streaming historical events via Ditto Protocol
+### Streaming via Ditto Protocol
-Please inspect the [protocol specification of DittoProtocol messages for streaming persisted events](protocol-specification-streaming-subscription.html)
-to find out how to stream historical (persisted) events via DittoProtocol.
-Using the DittoProtocol, historical events can be streamed either via WebSocket or connections.
+Use the [streaming subscription protocol](protocol-specification-streaming-subscription.html) to stream historical events via WebSocket or connections.
-Example protocol interaction for retrieving the persisted events of a thing:
+**Step 1**: Subscribe for persisted events:
-**First:** Subscribe for the persisted events of a thing
```json
{
"topic": "org.eclipse.ditto/thing-2/things/twin/streaming/subscribeForPersistedEvents",
@@ -182,42 +148,34 @@ Example protocol interaction for retrieving the persisted events of a thing:
}
```
-Alternatively to `fromHistoricalRevision` and `toHistoricalRevision`, also a timestamp based range may be used:
-`fromHistoricalTimestamp` and `toHistoricalTimestamp`.
-The "to" can be omitted in order to receive all events up to the current revision or timestamp.
+You can also use `fromHistoricalTimestamp` and `toHistoricalTimestamp`. Omit the "to" parameter to stream up to the current state.
+
+Ditto responds with a `created` event containing the subscription ID:
-As a result, the following `created` event is received as response:
```json
{
"topic": "org.eclipse.ditto/thing-2/things/twin/streaming/created",
"path": "/",
- "headers": {},
- "value": {
- "subscriptionId": "0"
- }
+ "value": { "subscriptionId": "0" }
}
```
-**Second:** Once the streaming subscription is confirmed to be created, request demand (of how many events to get streamed),
-referencing the `subscriptionId`:
+**Step 2**: Request demand:
+
```json
{
"topic": "org.eclipse.ditto/thing-2/things/twin/streaming/request",
"path": "/",
- "headers": {},
- "value": {
- "subscriptionId": "0",
- "demand": 25
- }
+ "value": { "subscriptionId": "0", "demand": 25 }
}
```
-The backend will start sending the requested persisted events as `next` messages:
+Ditto sends `next` events containing the historical events:
+
```json
{
"topic": "org.eclipse.ditto/thing-2/things/twin/streaming/next",
"path": "/",
- "headers": {},
"value": {
"subscriptionId": "0",
"item": {
@@ -227,31 +185,27 @@ The backend will start sending the requested persisted events as `next` messages
}
```
-It will do so either until all existing events were sent, in that case a `complete` event is sent:
+It continues until all existing events are sent, at which point it sends a `complete` event:
+
```json
{
"topic": "org.eclipse.ditto/thing-2/things/twin/streaming/complete",
"path": "/",
- "headers": {},
- "value": {
- "subscriptionId": "0"
- }
+ "value": { "subscriptionId": "0" }
}
```
-Or it will stop after the `demand` was fulfilled, waiting for the requester to claim more demand with a new `request`
-message.
+If the requested `demand` is fulfilled before all events are sent, the backend pauses and waits
+for the requester to claim more demand with a new `request` message.
-#### Filtering streamed historical events for things via Ditto Protocol
+#### Filtering via Ditto Protocol
-The `filter` for streaming historical thing events may also be specified via Ditto Protocol.
+Include a `filter` in the subscribe command's value:
-Example protocol message for subscribing for the persisted events of a thing with a `filter`:
```json
{
"topic": "org.eclipse.ditto/thing-2/things/twin/streaming/subscribeForPersistedEvents",
"path": "/",
- "headers": {},
"value": {
"fromHistoricalRevision": 1,
"toHistoricalRevision": 10,
@@ -260,35 +214,32 @@ Example protocol message for subscribing for the persisted events of a thing wit
}
```
-
## Configuring historical headers to persist
-In the configuration of the services (things, policies, connectivity) there is a section where to configure the historical
-headers to persist, for example this is the section for policies:
+Configure which Ditto headers to persist alongside events in the service configuration (things, policies, connectivity):
```hocon
event {
- # define the DittoHeaders to persist when persisting events to the journal
- # those can e.g. be retrieved as additional "audit log" information when accessing a historical policy revision
historical-headers-to-persist = [
- #"ditto-originator" # who (user-subject/connection-pre-auth-subject) issued the event
+ #"ditto-originator"
#"correlation-id"
]
historical-headers-to-persist = ${?POLICY_EVENT_HISTORICAL_HEADERS_TO_PERSIST}
}
```
-By default, no headers are persisted as historical headers, but it could e.g. make sense to persist the `ditto-originator`
-in order to provide "audit log" functionality in order to find out who (which subject) changed a policy at which time.
+By default, no headers are persisted. Persisting `ditto-originator` enables audit-log functionality (tracking who made each change).
## Cleanup retention time configuration
-In order to be able to access the history of entities, their journal database entries must not be cleaned up too quickly.
+To access entity history, journal entries must not be cleaned up too quickly.
+
+By default, Ditto enables [background cleanup](operating-devops.html#managing-background-cleanup) to remove stale data from MongoDB. If you use history capabilities, either:
+
+* Disable cleanup entirely (which increases database storage usage)
+* Configure `history-retention-duration` to keep history for a specific duration before cleanup
-By default, Ditto enables the [background cleanup](installation-operating.html#managing-background-cleanup) in order to
-delete "stale" (when not using the history feature) data from the MongoDB.
+## Further reading
-If Ditto shall be used with history capabilities, the cleanup has either
-* be disabled completely (which however could lead to a lot of used database storage)
-* or be configured with a `history-retention-duration` of a duration how long to keep "the history" before cleaning up
- snapshots and events
+- [Streaming subscription protocol](protocol-specification-streaming-subscription.html) -- reactive-streams protocol for historical events
+- [SSE API](httpapi-sse.html) -- server-sent events for streaming
diff --git a/documentation/src/main/resources/pages/ditto/basic-messages.md b/documentation/src/main/resources/pages/ditto/basic-messages.md
index 852c4245224..b0dc00c3327 100644
--- a/documentation/src/main/resources/pages/ditto/basic-messages.md
+++ b/documentation/src/main/resources/pages/ditto/basic-messages.md
@@ -5,168 +5,128 @@ tags: [model]
permalink: basic-messages.html
---
-Messages *do not affect* the state of a digital twin or an actual device.
-Therefore, Ditto does not handle messages like [commands](basic-signals-command.html): there are no responses which are
-produced by Ditto and no events which are emitted for messages.
+Messages let you send arbitrary payloads to or from a device through its digital twin. Unlike
+[commands](basic-signals-command.html), messages do not change the twin's state -- Ditto routes
+them without inspecting their content.
- {%
- include note.html content="Ditto has no knowledge of the payload of messages but merely routes messages between
- connected devices."
- %}
+{% include callout.html content="**TL;DR**: Messages are fire-and-forget payloads routed through Ditto to or from
+devices. Ditto does not interpret message content, does not guarantee delivery, and does not retain
+messages for offline devices." type="primary" %}
-Messages provide the possibility to send something **to** or **from** an actual device using an arbitrary subject/topic.
-They contain a custom payload with a custom content-type, so you can choose what content best
-fits your solution.
+## How messages work
-Expressed differently, Messages
-* **to** devices are operations which should trigger an action on a device (e.g. with a subject `turnOff`),
-* **from** devices are events/alarms which are emitted by devices (e.g. with a subject `smokeDetected`).
+You send a message **to** a device to trigger an action (for example, `turnOff`). A device sends
+a message **from** itself to report something (for example, `smokeDetected`). Ditto routes the
+message to all currently connected and authorized recipients.
{% include image.html file="pages/basic/ditto-messages.png" alt="Ditto Messages" caption="How Ditto acts as router for Messages" max-width=600 %}
+{% include note.html content="Ditto has no knowledge of the payload of messages but routes messages between
+connected devices." %}
-## Characteristics of Messages
-
-Eclipse Ditto is not a message broker and does not want to offer features a message broker does.
-
-It can be seen as a message router which:
-* accepts messages via 2 APIs ([HTTP](httpapi-messages.html) and
- [Ditto Protocol](protocol-specification-things-messages.html), e.g. via [WebSocket binding](httpapi-protocol-bindings-websocket.html))
-* checks for **currently connected** interested parties whether they may receive a specific Message
- (performs [authorization checks](basic-auth.html#authorization))
-* routes the Message and reply Messages in between connected clients
-
- {% include warning.html content="Ditto offers no message retention. If a device isn't connected when a Message should
- be routed, it will never receive the Message."
- %}
-
- {% include warning.html content="Ditto makes no statement about Message QoS. Messages are routed **at most once**."
- %}
-
- {% include warning.html content="Ditto does deliver messages only in \"fan out\" style,
- if the same credentials are connected twice, both connections will receive Messages if the credential is authorized
- to read a Message."
- %}
-
-
-## Elements
-
-Ditto messages always have to have at least these elements:
-* **Direction**: *to* / *from*,
-* **Thing ID**: the ID of the `Thing` (actual device) which should receive/send the message and
-* **Subject**: the custom subject/topic.
-
-Additionally, they can contain more information:
-* **Feature ID**: if a message should be addressed to the [Feature](basic-feature.html) of a Thing, this specifies
- its ID.
-* **content-type**: defines which content-type the optional payload has.
-* **correlation-id**: Ditto can route message responses back to the issuer of a message. Therefore, a correlation-id has
- to be present in the message.
+### Important characteristics
+Ditto is a message **router**, not a message **broker**:
-## Payload
+{% include warning.html content="Ditto offers no message retention. If a device is not connected when a message is
+ routed, it will never receive the message." %}
-A message can optionally contain a payload. As Ditto does neither have to understand the message nor its payload, the
-content-type and serialization is arbitrary.
+{% include warning.html content="Ditto makes no statement about message QoS. Messages are routed **at most once**." %}
+{% include warning.html content="Ditto delivers messages in a fan-out style. If the same credentials are connected
+ twice, both connections receive the message (assuming authorization permits it)." %}
-## APIs
+## Message elements
-Messages can be sent via:
-* the [WebSocket API](httpapi-protocol-bindings-websocket.html) as [Ditto Protocol](protocol-overview.html) messages,
-* the [HTTP API](httpapi-overview.html) either as "fire and forget" messages or, when expecting a response, in a
- blocking way at the [Messages HTTP API endpoint](http-api-doc.html#/Messages)
-* Ditto managed [connection sources](basic-connections.html#sources) when receiving messages in [Ditto Protocol](protocol-overview.html)
- via the source
+Every message requires these fields:
-Messages can be received via:
-* the [WebSocket API](httpapi-protocol-bindings-websocket.html) as [Ditto Protocol](protocol-overview.html) messages
-* the [Server Sent Event API](httpapi-sse.html#subscribe-for-messages-for-a-specific-thing)
-* Ditto managed [connection targets](basic-connections.html#target-topics-and-filtering) when "subscribing for Thing messages"
+| Field | Description |
+|-------|-------------|
+| **Direction** | `to` (toward the device) or `from` (from the device) |
+| **Thing ID** | The ID of the `Thing` that should receive or is sending the message |
+| **Subject** | A custom topic string (for example, `turnOff`, `smokeDetected`) |
+Optional fields:
-## Receiving Messages
+| Field | Description |
+|-------|-------------|
+| **Feature ID** | Target a specific [Feature](basic-feature.html) instead of the whole Thing |
+| **content-type** | MIME type of the payload (for example, `application/json`) |
+| **correlation-id** | Required if you want Ditto to route a response back to the sender |
-To be able to receive Messages for a Thing, you need to have `READ` access on that Thing.
-When a Message is sent to or from a Thing, **every** connected WebSocket, [SSE](httpapi-sse.html) or
-[connection target](basic-connections.html#targets) with the correct access rights will receive the Message.
+## Payload
-If there is more than one response to a message received by multiple consumers, only the
-first response will be routed back to the initial issuer of a Message.
+A message can carry any payload. Since Ditto does not interpret the content, you choose the
+serialization format and content-type that fits your solution.
-### Filtering when subscribing for messages
+## Sending and receiving messages
-In order to not receive all messages an [authenticated subject](basic-auth.html#authenticated-subjects) is entitled to
-receive when subscribing for messages, but to filter for specific criteria, messages may be filtered on the Ditto
-backend side before they are sent to a message receiver.
+### Sending
-#### Filtering messages by namespaces
+You can send messages through:
-Filtering messages may be done based on a namespace name. Each Ditto [Thing](basic-thing.html) has an ID containing a
-namespace (see also the conventions for a [Thing ID](basic-thing.html#thing-id)).
+* The [HTTP API](http-api-doc.html#/Messages) -- fire-and-forget or blocking (waits for a response)
+* The [WebSocket API](httpapi-protocol-bindings-websocket.html) -- as [Ditto Protocol](protocol-overview.html) messages
+* [Connection sources](basic-connections.html#sources) -- incoming Ditto Protocol messages via a managed connection
-By providing the `namespaces` filter, a comma separated list of which namespaces to include in the result, only Things
-in namespaces of interest are considered and thus only messages of these Things are published to the message receiver.
+Sending requires `WRITE` permission on the Thing (see [Permissions](#permissions)).
-For example, one would only subscribe for messages occurring in 2 specific namespaces by defining:
-```
-namespaces=org.eclipse.ditto.one,org.eclipse.ditto.two
-```
+### Receiving
-#### Filtering messages by RQL expression
+You can receive messages through:
-If filtering by namespaces is not sufficient, Ditto also allows to provide an [RQL expression](basic-rql.html)
-specifying:
+* The [WebSocket API](httpapi-protocol-bindings-websocket.html) -- as Ditto Protocol messages
+* [Server Sent Events (SSE)](httpapi-sse.html#thing-messages)
+* [Connection targets](basic-connections.html#target-topics-and-filtering) -- subscribing for "Thing messages"
-* an [enriched](basic-enrichment.html) Thing state based condition determining when messages should be delivered and when not
-* a filter based on the fields of the [Ditto Protocol](protocol-specification.html) message which should be filtered,
- e.g.:
- * using the `topic:subject` placeholder as query property, filtering for the message's subject
- and other filter options on the [Ditto Protocol topic](protocol-specification-topic.html) can be done
- * using the `resource:path` placeholder as query property, filtering based on the affected
- [Ditto Protocol path](protocol-specification.html#path) of a Ditto Protocol message can be done, e.g. targeting a
- specific message addressed to a certain feature ID
- * for all supported placeholders, please refer to the
- [placeholders documentation](basic-placeholders.html#scope-rql-expressions-when-filtering-for-ditto-protocol-messages)
+Receiving requires `READ` permission on the Thing. Every connected client with the correct
+permissions receives the message. If multiple consumers respond, only the first response is
+routed back to the original sender.
+### Filtering messages
-## Sending Messages
+When subscribing for messages, you can filter to receive only the ones you care about.
-If you want to send a Message to or from a Thing, you need `WRITE` permissions on that Thing.
-Every WebSocket or [connection target](basic-connections.html#targets) that is able to receive Messages for the
-Thing (`READ` permission), will receive your message.
+**By namespace:** Provide a comma-separated list of namespaces:
+```text
+namespaces=org.eclipse.ditto.one,org.eclipse.ditto.two
+```
-## Responding to Messages
+**By RQL expression:** Use an [RQL expression](basic-rql.html) with
+[enriched](basic-enrichment.html) Thing state or Ditto Protocol field
+[placeholders](basic-placeholders.html#scope-rql-expressions-when-filtering-for-ditto-protocol-messages):
-Since messages are stateless there is no *direct* response to a Message.
+* `topic:subject` -- filter by message subject
+* `resource:path` -- filter by the affected resource path
-For Ditto to be able to route the response of a Message back to the issuer, the
-correlation-ids need to match. E.g. when the sender uses correlation-id `random-aa98s`,
-any receiver can reply by using the same correlation-id `random-aa98s`.
+## Responding to messages
+Messages are stateless, so there is no built-in request-response mechanism. To route a response
+back to the sender, use the same `correlation-id` in the reply message.
## Permissions
-### API version 2
+To control message access, configure the `message:/` resource in the Thing's
+[Policy](basic-policy.html#message-resources):
-Permissions in API version 2 can be defined fine-grained. In order to be able to receive all Messages of a Thing,
-you need `READ` permission for the `message:/` resource in the used [Policy](basic-policy.html#message).
-There can however be specified that you may only receive specific Messages (with a defined subject), also
-expressed via [Policy entry](basic-policy.html#message).
-The same applies for being able to send Messages, here a `WRITE` permission is required either globally for
-all messages via the `message:/` resource or only for specific ones.
+* `READ` on `message:/` -- receive all messages for the Thing
+* `WRITE` on `message:/` -- send messages to the Thing
+* Fine-grained control -- grant access to specific subjects or features (for example,
+ `message:/features/lamp/inbox/messages/turnOff`)
-There is one sole exception, which are [Claim Messages](#claim-messages). You do
-not need the access rights for sending them.
+### Claim messages
+Claim messages are a special case for gaining initial access to a Thing:
-## Claim Messages
+* The subject is always `claim`
+* You do **not** need `WRITE` permission to send a claim message
+* Ditto forwards it to all subscribers registered for claim messages on that Thing
+* The receiver decides whether to grant access (for example, by updating the Policy)
-A Claim Message is used to gain access to a Thing. For this purpose a Claim Messages has two specifics:
-* the predefined message subject is always *claim*
-* you do not require `WRITE` permission to send a Claim Message
+## Further reading
-Apart from that the Claim Message is handled like a standard Message. It is forwarded to all Ditto Protocol bindings
-that registered for Claim Messages of the specific Thing. The decision whether to grant access (by setting permissions)
-is completely up to the receiver of the Claim Message e.g. after verifying the payload of the Message.
+* [Policies](basic-policy.html) -- configure message permissions
+* [Change Notifications](basic-changenotifications.html) -- subscribe to state change events
+* [Messages HTTP API](http-api-doc.html#/Messages) -- HTTP endpoint reference
+* [Ditto Protocol messages specification](protocol-specification-things-messages.html) -- wire format
diff --git a/documentation/src/main/resources/pages/ditto/basic-metadata.md b/documentation/src/main/resources/pages/ditto/basic-metadata.md
index 72af4c09aa3..001e1468f4d 100644
--- a/documentation/src/main/resources/pages/ditto/basic-metadata.md
+++ b/documentation/src/main/resources/pages/ditto/basic-metadata.md
@@ -1,200 +1,172 @@
---
-title: Thing Metadata
+title: Metadata
keywords: metadata, things, model, semantic
tags: [model]
permalink: basic-metadata.html
---
-A Thing in Ditto is also able to store metadata information, e.g. about single
-[feature properties](basic-feature.html#feature-properties), complete features and also [attributes](basic-thing.html#attributes)
-or other data stored in a digital twin ([thing](basic-thing.html)).
+Metadata lets you attach contextual information to any part of a Thing -- for example, recording
+when a value was last updated, who changed it, or what unit of measurement it uses.
-This metadata can contain additional information which shall not be treated as part of the twin's value, however may
-be useful to provide some context of the twin's data.
+{% include callout.html content="**TL;DR**: Metadata is extra information attached to Thing attributes and feature
+properties. You set it via the `put-metadata` header, read it via field selectors or the `get-metadata`
+header, and delete it via the `delete-metadata` header." type="primary" %}
-Metadata has not its own API but can only be modified, retrieved or deleted while modifying the state of a twin as a side effect.
-By default, metadata is not returned on API requests, but must be [asked for explicitly](#reading-metadata-information).
+## What is metadata for?
-An example is the timestamp when the current value of e.g. a feature property was updated for the last time.
-Or the metadata information of a feature property may contain information about its type or its semantics (e.g. a unit
-of measurement).
+Metadata provides context about your twin's data without becoming part of the data itself. Common
+use cases include:
+* **Timestamps** -- when a property was last updated
+* **Audit trails** -- who changed a value
+* **Semantic annotations** -- units of measurement, data type descriptions
+* **Change tracking** -- recording the source system that provided a value
-## Modifying metadata
+Metadata is **not** returned by default. You must request it explicitly.
-Modifying arbitrary `metadata` is possible by using the `put-metadata` header
-(e.g. for HTTP requests, set it as HTTP header, for Ditto Protocol requests, put it in the `"headers"` section of the
-protocol message), see [here for an overview of the available headers](protocol-specification.html#headers).
+## How it works
-The value of the `put-metadata` is a JSON array containing JSON objects with `"key"` and `"value"` parts:
-* `"key"`: describes the hierarchical position in the Thing where the metadata should be placed
-* `"value"`: is an arbitrary JSON value to set as metadata (could also be a nested JSON object)
+Metadata mirrors the structure of the Thing it describes. If your Thing has a property at
+`features/lamp/properties/color/r`, the metadata for that property lives at the same path
+inside the `_metadata` object.
-The `put-metadata` header can be specified at all API levels and will then modify the metadata object at the same level,
-relative to the request. For example a `PUT` on a feature property `color` with a `put-metadata` header `"key"`
-`/r/changeLog` will only set the metadata for this sub property.
+Metadata has no dedicated API endpoint. Instead, you modify, read, and delete it using HTTP headers
+as a side effect of regular Thing operations.
-### Example for setting metadata
+## Setting metadata
+
+Use the `put-metadata` header on any modifying request (`PUT`, `PATCH`, or via the Ditto Protocol
+`"headers"` section). The value is a JSON array of objects, each with a `"key"` and `"value"`:
+
+```json
+[
+ {
+ "key": "/features/lamp/properties/color/description",
+ "value": "Color represented with RGB values"
+ },
+ {
+ "key": "/features/lamp/properties/status/description",
+ "value": "Status of the lamp"
+ }
+]
+```
+
+### Example: setting metadata on a Thing update
+
+When you update your twin's lamp color:
-Assuming you modify your twin's lamp color with a call:
```json
{
"thingId": "org.eclipse.ditto:my-lamp-1",
"features": {
"lamp": {
"properties": {
- "color": {
- "r": 100,
- "g": 0,
- "b": 255
- },
- "status": {
- "on": "true"
- }
- }
- }
+ "color": { "r": 100, "g": 0, "b": 255 },
+ "status": { "on": "true" }
+ }
+ }
}
}
```
-In case you want to set metadata on all properties of a feature (`"r"`, `"g"` and `"b"`) plus some
-extra metadata to only set for the `"color"` property.
-The content of the `put-metadata` in order to do that would look like this:
+Send the `put-metadata` header with a description for each property group:
```json
[
{
- "key": "*/changeLog",
- "value": {
- "changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name":"ditto",
- "mail":"ditto@mail.com"
- }
- }
+ "key": "features/lamp/properties/color/description",
+ "value": "Color represented with RGB values"
},
{
- "key": "/features/lamp/properties/color/description",
- "value": "Color represented with RGB values"
+ "key": "features/lamp/properties/status/description",
+ "value": "Status of the lamp"
}
]
```
-The resulting Thing JSON including its `_metadata` would look like this:
+The resulting `_metadata` on the Thing:
+
```json
{
- "thingId": "org.eclipse.ditto:my-lamp-1",
+ "_metadata": {
"features": {
"lamp": {
"properties": {
"color": {
- "r": 100,
- "g": 0,
- "b": 255
+ "description": "Color represented with RGB values"
},
"status": {
- "on": "true"
- }
- }
- }
- },
- "_metadata": {
- "features": {
- "lamp": {
- "properties": {
- "color": {
- "r": {
- "changeLog": {
- "changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
- }
- },
- "g": {
- "changeLog": {
- "changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
- }
- },
- "b": {
- "changeLog": {
- "changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
- }
- },
- "description": "Color represented with RGB values"
- }
+ "description": "Status of the lamp"
}
}
}
}
+ }
}
```
-Another example for the `put-metadata` header can look like this where we want to add a "description" to both feature properties.
+### Combining wildcards with absolute paths
+
+You can mix wildcard keys and absolute-path keys in a single `put-metadata` array. For example, to
+add a change log to every leaf **and** a description to the `color` property:
```json
[
- {
- "key": "features/lamp/properties/color/description",
- "value": "Color represented with RGB values"
+ {
+ "key": "*/changeLog",
+ "value": {
+ "changedAt": "2022-08-02T04:30:07",
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
+ }
},
- {
- "key": "features/lamp/properties/status/description",
- "value": "Status of the lamp"
- }
+ {
+ "key": "/features/lamp/properties/color/description",
+ "value": "Color represented with RGB values"
+ }
]
```
-The resulting Thing JSON with the previously added metadata would look like this:
+The resulting `_metadata` contains both the per-leaf `changeLog` entries and the absolute-path
+`description`:
+
```json
{
- "thingId": "org.eclipse.ditto:my-lamp-1",
+ "_metadata": {
"features": {
"lamp": {
"properties": {
"color": {
- "r": 100,
- "g": 0,
- "b": 255
- },
- "status": {
- "on": "true"
- }
- }
- }
- },
- "_metadata": {
- "features": {
- "lamp": {
- "properties": {
- "color": {
- "description": "Color represented with RGB values"
+ "r": {
+ "changeLog": {
+ "changedAt": "2022-08-02T04:30:07",
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
+ }
},
- "status": {
- "description": "Status of the lamp"
- }
+ "g": {
+ "changeLog": {
+ "changedAt": "2022-08-02T04:30:07",
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
+ }
+ },
+ "b": {
+ "changeLog": {
+ "changedAt": "2022-08-02T04:30:07",
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
+ }
+ },
+ "description": "Color represented with RGB values"
}
}
}
}
+ }
}
```
-### Modifying metadata on all JSON leaves
+### Setting metadata on all JSON leaves
-A special syntax for the key is `*/{key}` which means that all JSON leaves of the modify operation will
-get the metadata key `{key}` with the given value. So if, for example, only the affected JSON leaves should
-get the timestamp where the changed values were recorded, one would set the `put-metadata` header as shown in the
-following example:
+Use the special key syntax `*/{key}` to set the same metadata on every leaf value affected by your
+update. For example, to record a change log on all modified properties:
```json
[
@@ -202,33 +174,17 @@ following example:
"key": "*/changeLog",
"value": {
"changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name":"ditto",
- "mail":"ditto@mail.com"
- }
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
}
]
```
-The resulting Thing JSON including its `_metadata` would look like this:
+This adds a `changeLog` entry to each individual leaf property (`r`, `g`, `b`, `on`) rather than
+to the parent objects. The resulting `_metadata` would look like:
+
```json
{
- "thingId": "org.eclipse.ditto:my-lamp-1",
- "features": {
- "lamp": {
- "properties": {
- "color": {
- "r": 100,
- "g": 0,
- "b": 255
- },
- "status": {
- "on": "true"
- }
- }
- }
- },
"_metadata": {
"features": {
"lamp": {
@@ -237,39 +193,27 @@ The resulting Thing JSON including its `_metadata` would look like this:
"r": {
"changeLog": {
"changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
},
"g": {
"changeLog": {
"changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
},
"b": {
"changeLog": {
"changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
- },
- "status": {
- "on": {
- "changeLog": {
- "changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
- }
+ }
+ },
+ "status": {
+ "on": {
+ "changeLog": {
+ "changedAt": "2022-08-02T04:30:07",
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
}
}
@@ -280,23 +224,19 @@ The resulting Thing JSON including its `_metadata` would look like this:
}
```
-## Reading metadata information
+## Reading metadata
-Metadata of a Thing can be retrieved by querying a full thing, e.g. via the [HTTP API](http-api-doc.html), and
-specifying an (additional) [field selector](httpapi-concepts.html#with-field-selector) `_metadata`,
-e.g.: `?fields=thingId,attributes,_metadata`.
+You have two ways to retrieve metadata.
-Metadata can also be queried by using the `get-metadata` header
-(e.g. for HTTP requests, set it as HTTP header, for Ditto Protocol requests, put it in the `"headers"` section of the
-protocol message), see [here for an overview of the available headers](protocol-specification.html#headers).
-The `get-metadata` header expects a comma separated list of metadata `{key}`. This will return the relative metadata
-in the `ditto-metadata` header.
+### Using field selectors
-### Example for reading metadata with field selector
+Add `_metadata` to the `fields` query parameter:
-For example a `GET` request to
-`https://{ditto-instance}/api/2/things/{namespace}:{name}?fields=thingId,policyId,features,_created,_modified,_revision,_metadata`
-will yield the metadata stored for the given Thing, in the following format:
+```bash
+GET /api/2/things/org.eclipse.ditto:my-lamp-1?fields=thingId,policyId,features,_created,_modified,_revision,_metadata
+```
+
+This returns the Thing along with its full `_metadata` object:
```json
{
@@ -305,14 +245,8 @@ will yield the metadata stored for the given Thing, in the following format:
"features": {
"lamp": {
"properties": {
- "color": {
- "r": 0,
- "g": 255,
- "b": 255
- },
- "status": {
- "on": "true"
- }
+ "color": { "r": 0, "g": 255, "b": 255 },
+ "status": { "on": "true" }
}
}
},
@@ -326,29 +260,20 @@ will yield the metadata stored for the given Thing, in the following format:
"color": {
"r": {
"changeLog": {
- "changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
+ "changedAt": "2022-08-02T04:30:07",
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
},
"g": {
"changeLog": {
"changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
},
"b": {
"changeLog": {
- "changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
+ "changedAt": "2022-08-02T04:30:07",
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
},
"description": "Color represented with RGB values"
@@ -357,10 +282,7 @@ will yield the metadata stored for the given Thing, in the following format:
"on": {
"changeLog": {
"changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
},
"description": "Status of the Lamp"
@@ -372,10 +294,13 @@ will yield the metadata stored for the given Thing, in the following format:
}
```
-### Example for reading metadata with header
+### Using the get-metadata header
+
+Set the `get-metadata` HTTP header to a comma-separated list of metadata paths. Ditto returns
+the matching metadata in the `ditto-metadata` response header.
-For example a `GET` request to `https://{ditto-instance}/api/2/things/{namespace}:{name}` with HTTP header `get-metadata`
-and value `features/lamp/properties/color/r` will return the following content in the `ditto-metadata` header:
+For example, to get the metadata for a specific leaf property, set the header
+`get-metadata: features/lamp/properties/color/r`. The `ditto-metadata` response header contains:
```json
{
@@ -386,10 +311,7 @@ and value `features/lamp/properties/color/r` will return the following content i
"r": {
"changeLog": {
"changedAt": "2022-08-02T04:30:07",
- "changedBy": {
- "name": "ditto",
- "mail": "ditto@mail.com"
- }
+ "changedBy": { "name": "ditto", "mail": "ditto@mail.com" }
}
}
}
@@ -399,9 +321,8 @@ and value `features/lamp/properties/color/r` will return the following content i
}
```
-Metadata can also be queried using a wildcard which expands the level where it is specified, e.g. a `GET` request
-to `https://{ditto-instance}/api/2/things/{namespace}:{name}` with HTTP header `get-metadata` and value
-`features/lamp/properties/*/description` will return:
+You can use wildcards to expand one level. For example,
+`get-metadata: features/lamp/properties/*/description` returns:
```json
{
@@ -420,44 +341,47 @@ to `https://{ditto-instance}/api/2/things/{namespace}:{name}` with HTTP header `
}
```
-## Deleting metadata information
+## Deleting metadata
-Metadata can be deleted by using the `delete-metadata` header with modifying requests (`PUT` or `PATCH`).
-For HTTP requests, set it as HTTP header, for Ditto Protocol requests, put it in the `"headers"` section of the
-protocol message, see [here for an overview of the available headers](protocol-specification.html#headers).
-The `delete-metadata` header expects a comma separated list of metadata `{key}`.
+Use the `delete-metadata` header on a modifying request (`PUT` or `PATCH`). Provide a
+comma-separated list of metadata paths to remove.
-For example a `PATCH` request to `https://{ditto-instance}/api/2/things/{namespace}:{name}` with HTTP header `delete-metadata`
-and value `features/lamp/properties/color` will remove the complete `color` property from the thing metadata.
+For example, to remove all color metadata:
-Note: When deleting things or parts of a thing, like feature properties or attributes, their relative metadata
+```
+delete-metadata: features/lamp/properties/color
+```
+
+When you delete a Thing or part of a Thing (like a feature or attribute), the associated metadata
is also deleted.
-## Wildcard usage for metadata requests
-
-When working with metadata there are some wildcards which can be used to modify, retrieve or delete metadata.
-The following table gives an overview which Wildcards can be used on top-level for what requests.
-
-| Wildcard | PUT/PATCH | GET | DELETE |
-|-----------------------------------------|------------------------------------------------------------------------|------------------------------------------------------------------------------|----------------------------------------------------------------------------|
-| `*` | x | retrieve all metadata relative to the path | delete all metadata relative to the path |
-| `*/key` | add metadata for the given `key` to all JSON leaves | retrieve all metadata with `key` | delete all metadata with `key` |
-| `attributes/*/key` | add metadata for the given `key` to all attributes | retrieve metadata for given `key` from all attributes | delete metadata for given `key` from all attributes |
-| `features/*/properties/*/key` | add metadata for the given `key` to all feature properties | retrieve metadata for given `key` from all feature properties | delete metadata for given `key` from all feature properties |
-| `features/*/properties/{property}/key` | add metadata for the given `key` to all features with a given property | retrieve metadata for given `key` from all features with a specific property | delete metadata for given `key` from all features with a specific property |
-| `features/{feature}/properties/*/key` | add metadata for the given `key` to all properties of a feature | retrieve metadata for given `key` from all properties of a feature | delete metadata for given `key` from all properties of a feature |
-
-Wildcards can also be used on the /features resource:
-
-| Wildcard | PUT/PATCH | GET | DELETE |
-|-------------------------------|------------------------------------------------------------------------|------------------------------------------------------------------------------|----------------------------------------------------------------------------|
-| `*/properties/*/key` | add metadata for the given `key` to all feature properties | retrieve metadata for given `key` from all feature properties | delete metadata for given `key` from all feature properties |
-| `*/properties/{property}/key` | add metadata for the given `key` to all features with a given property | retrieve metadata for given `key` from all features with a specific property | delete metadata for given `key` from all features with a specific property |
-| `{feature}/properties/*/key` | add metadata for the given `key` to all properties of a feature | retrieve metadata for given `key` from all properties of a feature | delete metadata for given `key` from all properties of a feature |
-
-Note: With the `*` wildcard it is only possible to skip one level in the resource path. You can not use the wildcard `*` other than in the examples above.
-
-## Multiple metadata header
-It is not possible to use multiple metadata headers in one request. For GET requests it is only possible to use the `get-metadata`header.
-For PUT/PATCH requests it is only possible to use either the `put-metadata` or `delete-metadata` header.
-In case multiple headers are used in a request an exception will be the result.
\ No newline at end of file
+## Wildcard reference
+
+| Wildcard | PUT/PATCH | GET | DELETE |
+|----------|-----------|-----|--------|
+| `*` | -- | Retrieve all metadata relative to the path | Delete all metadata relative to the path |
+| `*/key` | Add `key` metadata to all JSON leaves | Retrieve all metadata with `key` | Delete all metadata with `key` |
+| `attributes/*/key` | Add `key` to all attributes | Retrieve `key` from all attributes | Delete `key` from all attributes |
+| `features/*/properties/*/key` | Add `key` to all feature properties | Retrieve `key` from all feature properties | Delete `key` from all feature properties |
+| `features/*/properties/{prop}/key` | Add `key` to `{prop}` in all features | Retrieve `key` from `{prop}` in all features | Delete `key` from `{prop}` in all features |
+| `features/{feat}/properties/*/key` | Add `key` to all properties of `{feat}` | Retrieve `key` from all properties of `{feat}` | Delete `key` from all properties of `{feat}` |
+
+The same wildcards work at the `/features` resource level, using relative paths
+(`*/properties/*/key`, `{feature}/properties/*/key`, etc.).
+
+The `*` wildcard skips exactly one level in the path. It cannot be used in other positions.
+
+## Constraints
+
+You can use only one metadata header per request:
+* For GET requests: `get-metadata` only
+* For PUT/PATCH requests: either `put-metadata` or `delete-metadata`, but not both
+
+Using multiple metadata headers in one request results in an error.
+
+## Further reading
+
+* [Protocol headers reference](protocol-specification.html#headers) -- all available Ditto headers
+* [Things](basic-thing.html) -- the entity that metadata attaches to
+* [HTTP API concepts](httpapi-concepts.html#field-selectors) -- field selectors for partial
+ retrieval
diff --git a/documentation/src/main/resources/pages/ditto/basic-namespaces-and-names.md b/documentation/src/main/resources/pages/ditto/basic-namespaces-and-names.md
index 573b1d64f9f..a08a0d919ab 100644
--- a/documentation/src/main/resources/pages/ditto/basic-namespaces-and-names.md
+++ b/documentation/src/main/resources/pages/ditto/basic-namespaces-and-names.md
@@ -1,70 +1,76 @@
---
-title: Namespaces and Names
+title: Namespaces & Names
keywords: namespace, name, id, entity, model, regex
tags: [model]
permalink: basic-namespaces-and-names.html
---
-Ditto uses namespaces and names for the IDs of important entity types like Things or Policies. Due to the fact that
-those IDs often need to be set in the path of HTTP requests, we have restricted the set of allowed characters.
+Ditto uses namespaced identifiers for Things, Policies, and other entities. Every ID combines a
+namespace and a name separated by a colon.
+
+{% include callout.html content="**TL;DR**: Entity IDs follow the format `namespace:name`, with a maximum length of
+256 characters. Namespaces use dot-separated segments (like Java packages), and names can contain
+most characters except slashes and control characters." type="primary" %}
## Namespace
-The namespace must conform to the following notation:
-* must start with a lower- or uppercase character from a-z
-* may use dots (`.`) or dashes (`-`) to separate characters
-* a dot or dash must be followed by a lower- or uppercase character from a-z
-* numbers may be used
-* underscore may be used
-
-When writing a Java application, you can use the following regex to validate your namespaces:
- ``(?|(?:(?:[a-zA-Z]\w*+)(?:[.-][a-zA-Z]\w*+)*+))``
- (see [RegexPatterns#NAMESPACE_REGEX](https://github.com/eclipse-ditto/ditto/blob/master/base/model/src/main/java/org/eclipse/ditto/base/model/entity/id/RegexPatterns.java)).
-
-Examples for valid namespaces:
-* `org.eclipse.ditto`,
-* `com.some-domain`,
-* `com.google`,
+The namespace identifies the organizational scope for an entity. It must:
+
+* Start with a letter (`a-z` or `A-Z`)
+* Use dots (`.`) or dashes (`-`) to separate segments, each starting with a letter
+* Contain only letters, digits, and underscores within segments
+
+**Valid namespaces:**
+* `org.eclipse.ditto`
+* `com.some-domain`
+* `com.google`
* `foo.bar_42`
+**Regex (Java):**
+``(?|(?:(?:[a-zA-Z]\w*+)(?:[.-][a-zA-Z]\w*+)*+))``
+(see [RegexPatterns#NAMESPACE_REGEX](https://github.com/eclipse-ditto/ditto/blob/master/base/model/src/main/java/org/eclipse/ditto/base/model/entity/id/RegexPatterns.java))
+
## Name
-The name must conform to the following notation:
-* may not be empty
-* may not contain `/` (slash)
-* may not contain control characters
+The name identifies the entity within its namespace. It must:
-When writing a Java application, you can use the following regex to validate your thing name:
- ``(?[^\x00-\x1F\x7F-\xFF/]++)``
- (see [RegexPatterns#ENTITY_NAME_REGEX](https://github.com/eclipse-ditto/ditto/blob/master/base/model/src/main/java/org/eclipse/ditto/base/model/entity/id/RegexPatterns.java)).
+* Not be empty
+* Not contain `/` (slash)
+* Not contain control characters
-Examples for valid names:
- * `ditto`,
- * `smart-coffee-1`,
- * `foo%2Fbar`
- * `foo bar`
- * `foo+bar%20`
+**Valid names:**
+* `ditto`
+* `smart-coffee-1`
+* `foo%2Fbar`
+* `foo bar`
+* `foo+bar%20`
-## Namespaced ID
+**Regex (Java):**
+``(?[^\x00-\x1F\x7F-\xFF/]++)``
+(see [RegexPatterns#ENTITY_NAME_REGEX](https://github.com/eclipse-ditto/ditto/blob/master/base/model/src/main/java/org/eclipse/ditto/base/model/entity/id/RegexPatterns.java))
-A namespaced ID must conform to the following expectations:
-* namespace and name separated by a `:` (colon)
-* have a maximum length of 256 characters
+## Namespaced ID
-When writing a Java application, you can use the following regex to validate your namespaced IDs:
- ``(?|(?:(?:[a-zA-Z]\w*+)(?:[.-][a-zA-Z]\w*+)*+)):(?[^\x00-\x1F\x7F-\xFF/]++)``
- (see [RegexPatterns#ID_REGEX](https://github.com/eclipse-ditto/ditto/blob/master/base/model/src/main/java/org/eclipse/ditto/base/model/entity/id/RegexPatterns.java)).
+A complete entity ID joins the namespace and name with a colon (`:`). The combined ID must not
+exceed 256 characters.
-Examples for valid IDs:
-* `org.eclipse.ditto:smart-coffee-1`,
-* `foo:bar`,
-* `org.eclipse.ditto_42:smart-coffeee`,
-* `com.some-domain.ditto-rocks:foobar`,
-* `org.eclipse:admin-policy`,
+**Valid IDs:**
+* `org.eclipse.ditto:smart-coffee-1`
+* `foo:bar`
+* `org.eclipse.ditto_42:smart-coffeee`
+* `com.some-domain.ditto-rocks:foobar`
+* `org.eclipse:admin-policy`
* `org.eclipse:admin policy`
+**Regex (Java):**
+``(?|(?:(?:[a-zA-Z]\w*+)(?:[.-][a-zA-Z]\w*+)*+)):(?[^\x00-\x1F\x7F-\xFF/]++)``
+(see [RegexPatterns#ID_REGEX](https://github.com/eclipse-ditto/ditto/blob/master/base/model/src/main/java/org/eclipse/ditto/base/model/entity/id/RegexPatterns.java))
+
## Encoding and decoding
-If hex encoded characters or spaces are used in the Thing name, the protocol dependent de- or encoding must be
-taken into account. If a Thing is created with the ID `eclipse.ditto:foo bar` and is to be queried via the HTTP API,
-the space must be encoded accordingly: `GET /things/eclipse.ditto:foo%20bar`.
+When a Thing name contains spaces or special characters, you must URL-encode them in HTTP requests.
+For example, if you create a Thing with the ID `eclipse.ditto:foo bar`, query it as:
+
+```bash
+GET /things/eclipse.ditto:foo%20bar
+```
diff --git a/documentation/src/main/resources/pages/ditto/basic-overview.md b/documentation/src/main/resources/pages/ditto/basic-overview.md
index 466386fdc99..5b1f80525c9 100644
--- a/documentation/src/main/resources/pages/ditto/basic-overview.md
+++ b/documentation/src/main/resources/pages/ditto/basic-overview.md
@@ -1,33 +1,36 @@
---
-title: Basic concepts overview
+title: Data Model Overview
keywords: basic concepts, overview, thing, feature, domain model, model
tags: [model]
permalink: basic-overview.html
---
-## Domain model
+Ditto's data model organizes IoT device data into a hierarchy of Things, Features, and Policies.
-Eclipse Ditto does not claim to know exactly which structure Things in the
-IoT have or should have.
-Its idea is to be as agnostic as possible when it comes to `Thing` data.
+{% include callout.html content="**TL;DR**: A Thing has attributes (static metadata) and features (dynamic state). A
+Policy controls who can read and write each part. That is the entire data model." type="primary" %}
-Nevertheless, two coarse elements are defined in order to structure `Thing`s (see also [Thing](basic-thing.html)):
-* Attributes: intended for managing static metadata of a `Thing` - as JSON object - which does not change frequently.
-* [Features](basic-feature.html): intended for managing state data (e.g. sensor data or configuration data) of a `Thing`.
+## How the model works
-## API version 2
+Ditto does not enforce a specific schema for your
+IoT data. Instead, it provides
+two structural elements inside each [Thing](basic-thing.html):
-In API version 2 the information which _subjects_ are allowed to READ, WRITE Things are managed separately via
-[Policies](basic-policy.html).
-The `Thing` only contains a `policyId` which links to a Policy containing the authorization information.
-This class diagram shows the structure Ditto requires for **API version 2**:
+* **Attributes** -- static metadata about the `Thing` (location, serial number, manufacturer) stored as
+ a JSON object. These values do not change frequently.
+* **[Features](basic-feature.html)** -- dynamic state data (sensor readings, configuration,
+ operational status). Each Feature groups related properties under a named identifier.
+
+Access control is managed separately through [Policies](basic-policy.html). The `Thing` references its
+Policy by `policyId`, and that Policy defines which authenticated subjects may read or write the
+`Thing` or parts of it.
{% include image.html file="pages/basic/ditto-class-diagram-v2.png" alt="Ditto Class Diagram"
- caption="Class diagram of Ditto's most basic entities in API version 2." max-width=600 %}
+ caption="Class diagram of Ditto's core entities." max-width=600 %}
-### JSON Format
+## Minimal Thing
-In **API version 2** the most minimalistic representation of a Thing is for example the following:
+The smallest valid Thing contains only an ID and a reference to its Policy:
```json
{
@@ -36,10 +39,11 @@ In **API version 2** the most minimalistic representation of a Thing is for exam
}
```
-Attributes and Features are optional (as also shown in the class diagram above), so in the example JSON above they are
-omitted.
+Attributes and Features are optional.
+
+## A Thing with data
-A minimalistic Thing with one attribute, one Feature, and a definition could look like this:
+A more realistic Thing includes a definition, an attribute, and a feature:
```json
{
@@ -51,10 +55,17 @@ A minimalistic Thing with one attribute, one Feature, and a definition could loo
},
"features": {
"transmission": {
- "properties": {
- "cur_speed": 90
- }
- }
+ "properties": {
+ "cur_speed": 90
+ }
+ }
}
}
```
+
+## Further reading
+
+* [Things](basic-thing.html) -- full details on Thing structure, IDs, and attributes
+* [Features](basic-feature.html) -- properties, desired properties, and definitions
+* [Policies](basic-policy.html) -- fine-grained access control for Things and Features
+* [Namespaces & Names](basic-namespaces-and-names.html) -- naming conventions for entity IDs
diff --git a/documentation/src/main/resources/pages/ditto/basic-placeholders.md b/documentation/src/main/resources/pages/ditto/basic-placeholders.md
index 71ce63e2293..58dbaa51dd6 100644
--- a/documentation/src/main/resources/pages/ditto/basic-placeholders.md
+++ b/documentation/src/main/resources/pages/ditto/basic-placeholders.md
@@ -31,7 +31,7 @@ Which placeholder values are available depends on the context where the placehol
| Placeholder | Description |
|----------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `{%raw%}{{ thing-json: }}{%endraw%}` | Value (in string representation) of the JSON identified by the provided ''json-pointer'' in JsonPointer notation - e.g., `thing-json:attributes/location` for the "location" attribute or `thing-json:features/temperature/properties/value` for the temperature property. Also supported in the [migrate definition](httpapi-concepts.html#things-in-api-2---migrate-thing-definitions) migration payload: when the value is exactly one placeholder (brace or legacy, no pipeline), the resolved value preserves its JSON type (number, boolean, string, object, array); pipelines and multiple placeholders yield a string. |
+| `{%raw%}{{ thing-json: }}{%endraw%}` | Value (in string representation) of the JSON identified by the provided ''json-pointer'' in JsonPointer notation - e.g., `thing-json:attributes/location` for the "location" attribute or `thing-json:features/temperature/properties/value` for the temperature property. Also supported in the [migrate definition](httpapi-concepts.html#things-endpoints---migrate-thing-definitions) migration payload: when the value is exactly one placeholder (brace or legacy, no pipeline), the resolved value preserves its JSON type (number, boolean, string, object, array); pipelines and multiple placeholders yield a string. |
### Feature Placeholder
diff --git a/documentation/src/main/resources/pages/ditto/basic-policy.md b/documentation/src/main/resources/pages/ditto/basic-policy.md
index 99c24e7d72b..5fea6d18d07 100644
--- a/documentation/src/main/resources/pages/ditto/basic-policy.md
+++ b/documentation/src/main/resources/pages/ditto/basic-policy.md
@@ -802,8 +802,8 @@ namespace org.eclipse.ditto (the pattern requires at least one sub-
## Tools for editing a Policy
-The Policy can be edited with a text editor of your choice.
-Just make sure it is in valid JSON representation, and that at least one valid subject is granted write permission at
+You can edit the Policy with any text editor.
+Make sure it is valid JSON and that at least one valid subject is granted write permission at
the root resources.
{%
@@ -817,7 +817,7 @@ granted or revoked the permissions your use case is supposed to support.
## Example
-Given you need to support the following scenario:
+Consider this access control scenario:
* Owner: The Thing *my.namespace:thing-0123* is owned by a user. Thus, she needs full access and admin rights for the
complete Thing.
diff --git a/documentation/src/main/resources/pages/ditto/basic-rql.md b/documentation/src/main/resources/pages/ditto/basic-rql.md
index 32c1fd82d99..cbaac6e8349 100644
--- a/documentation/src/main/resources/pages/ditto/basic-rql.md
+++ b/documentation/src/main/resources/pages/ditto/basic-rql.md
@@ -5,24 +5,21 @@ tags: [rql, protocol]
permalink: basic-rql.html
---
-Ditto utilizes a subset of RQL
-as language for specifying queries.
+You query Ditto using a subset of Resource Query Language (RQL) for specifying queries, filters, and conditions.
-The [RQL project page](https://github.com/persvr/rql) says about it:
+{% include callout.html content="**TL;DR**: RQL provides nestable operators like `eq()`, `gt()`, `like()`, and `exists()` combined with `and()`, `or()`, `not()`. Use it for search queries, change notification filters, and conditional requests." type="primary" %}
-> Resource Query Language (RQL) is a query language designed for use in URIs with object style data structures. [...]
-RQL can be thought as basically a set of nestable named operators which each have a set of arguments.
-RQL is designed to have an extremely simple, but extensible grammar that can be written in a URL friendly query string.
+## Overview
-An example helps more than a thousand words, so that would be the example of a simple RQL query querying
-for `foo="ditto"` and `bar<10`:
-```
+RQL is a query language designed for use in URIs with object-style data structures (see the [RQL project page](https://github.com/persvr/rql)).
+
+Here is a simple RQL query that filters for `foo="ditto"` and `bar<10`:
+
+```text
and(eq(foo,"ditto"),lt(bar,10))
```
-That query consists of one [logical operator ](#logical-operators) ["and"](#and),
-two [relational operators](#relational-operators) of which each consists of a [property](#query-property)
-and a [value](#query-value).
+This query consists of one [logical operator](#logical-operators) (`and`) containing two [relational operators](#relational-operators), each with a [property](#query-property) and a [value](#query-value).
The following sections describe what the RQL syntax is capable of and which RQL operators are supported in
Eclipse Ditto.
@@ -31,244 +28,230 @@ Eclipse Ditto.
`eq(foo,3)`, is supported by Eclipse Ditto and that Ditto added more non-specified operators, e.g.
`like`, `exists` and `empty`." %}
-
## RQL filter
The RQL filter specifies "what" to filter.
### Query property
-```
+
+```text
= url-encoded-string
```
-When not starting with a prefix `:`, the RQL query property specifies a field in the JSON representation
-of a [Thing](basic-thing.html#api-version-2).
-For example a query property `thingId` selects the Thing ID as property, a query property `attributes/location` selects
-an attribute with the name `location` as query property.
+A query property specifies a field in the JSON representation of a [Thing](basic-thing.html#model-specification). For example, `thingId` selects the Thing ID, and `attributes/location` selects the `location` attribute.
-To filter nested properties, Ditto uses the JSON Pointer notation ([RFC-6901](https://tools.ietf.org/html/rfc6901)),
-where the property can also start with a slash `/` or omit it, so those 2 query properties are semantically the same:
+Ditto uses JSON Pointer notation ([RFC-6901](https://tools.ietf.org/html/rfc6901)) for nested properties. You can include or omit the leading slash -- these are equivalent:
* `/attributes/location`
* `attributes/location`
-The following example shows how to apply a filter for the sub property `location` of the parent property `attributes`
-with a forward slash as separator:
-```
+**Example** -- filter by a nested attribute:
+
+```text
eq(attributes/location,"kitchen")
```
#### Placeholders as query properties
-When using an RQL expression in order to e.g. filter for certain
-[change notifications](basic-changenotifications.html#by-rql-expression), the query property might be a
-[placeholder](basic-placeholders.html#scope-rql-expressions-when-filtering-for-ditto-protocol-messages) instead of a
-field in JSON representation inside the Thing.
+When you use RQL to filter [change notifications](basic-changenotifications.html#filter-by-rql-expression), you can use [placeholders](basic-placeholders.html#scope-rql-expressions-when-filtering-for-ditto-protocol-messages) instead of Thing JSON fields:
-Currently supported placeholders for RQL expressions are:
* `topic:`
* `resource:`
* `time:`
-The [placeholder](basic-placeholders.html#scope-rql-expressions-when-filtering-for-ditto-protocol-messages)
-documentation describes which placeholder names are supported.
-
+See the [placeholder documentation](basic-placeholders.html#scope-rql-expressions-when-filtering-for-ditto-protocol-messages) for supported names.
### Query value
-```
+
+```text
= , , , true, false, null
= , = "url-encoded-string", 'url-encoded-string'
= time:now, time:now_epoch_millis
```
-String values may either be delimited using single or double quotes.
+String values can use single or double quotes.
-**Comparison of string values**
+**String comparison**
-{% include note.html content="Comparison operators such as `gt`, `ge`, `lt` and `le`, do not support a special
+{% include note.html content="Comparison operators such as `gt`, `ge`, `lt` and `le`, do not support a special
\"semantics\" of string comparison (e.g. regarding alphabetical or lexicographical ordering).
- However, you can rely on the alphabetical sorting of strings with the same length (e.g. \"aaa\" < \"zzz\") and that the
+ However, you can rely on the alphabetical sorting of strings with the same length (e.g. \"aaa\" < \"zzz\") and that the
order stays the same over multiple/different filter requests." %}
-
-**Comparison of other data types**
-{% include note.html content="Please note that the comparison of other data types is supported by the API, but it
- only supports comparison of same data types, and does not do any conversion during comparison." %}
+**Other data types**
+{% include note.html content="Please note that the comparison of other data types is supported by the API, but it
+ only supports comparison of same data types, and does not do any conversion during comparison." %}
### Relational operators
-The following relational operators are supported.
-
#### eq
+
Filter property values equal to ``.
-```
+```text
eq(,)
```
-**Example - filter things owned by "SID123"**
-```
+**Example** -- filter Things owned by "SID123":
+
+```text
eq(attributes/owner,"SID123")
```
#### ne
+
Filter property values not equal to ``.
-```
+```text
ne(,)
-```
-
-**Example - filter things with owner different than "SID123"**
```
+
+**Example** -- filter Things with owner different from "SID123":
+
+```text
ne(attributes/owner,"SID123")
```
-The response will contain only things which **do** provide an owner attribute (in this case with value 0 or not SID123).
+The response only contains Things that have an owner attribute (with a value other than "SID123").
#### gt
-Filter property values greater than a ``.
-```
-gt(,)
-```
+Filter property values greater than ``.
-**Example - filter things with thing ID greater than "A000"**
+```text
+gt(,)
```
+
+**Example**:
+
+```text
gt(thingId,"A000")
```
#### ge
-Filter property values greater than or equal to a ``.
-```
-ge(,)
-```
+Filter property values greater than or equal to ``.
-**Example - filter things with thing ID "A000" or greater**
+```text
+ge(,)
```
+
+**Example**:
+
+```text
ge(thingId,"A000")
```
#### lt
-Filter property values less than a ``.
+Filter property values less than ``.
+
+```text
+lt(,)
```
-lt(,)
-```
-
-**Example - filter things with thing ID lower than "A000"**
-```
+
+**Example**:
+
+```text
lt(thingId,"A000")
```
#### le
-Filter property values less than or equal to a ``.
-```
-le(,)
-```
+Filter property values less than or equal to ``.
-**Example - filter things with thing ID "A000" or lower**
+```text
+le(,)
```
+
+**Example**:
+
+```text
le(thingId,"A000")
```
#### in
-Filter property values which contains at least one of the listed ``s.
-```
-in(,,, ...)
-```
+Filter property values matching at least one of the listed values.
-**Example - filter things with thing ID "A000" or "AB00" or "AZ99"**
+```text
+in(,,, ...)
```
+
+**Example** -- filter Things with ID "A000", "AB00", or "AZ99":
+
+```text
in(thingId,"A000","AB00","AZ99")
```
#### like
-Filter property values which are like (similar) a ``.
-```
-like(,)
+Filter property values matching a pattern (Ditto-specific operator).
+
+```text
+like(,)
```
{% include note.html content="The `like` operator is not defined in the linked RQL grammar, it is a Ditto
specific operator." %}
-**Details concerning the like-operator**
+Supported pattern expressions:
+* `*endswith` -- match at the end of a string
+* `startswith*` -- match at the beginning of a string
+* `*contains*` -- match if a string contains the pattern
+* `Th?ng` -- match a single wildcard character
-The `like` operator provides some regular expression capabilities for pattern matching Strings.
-The following expressions are supported:
+**Examples**:
-* \*endswith => match at the end of a specific String.
-* startswith\* => match at the beginning of a specific String.
-* \*contains\* => match if contains a specific String.
-* Th?ng => match for a wildcard character.
-
-**Examples**
-```
+```text
like(attributes/key1,"*known-chars-at-end")
-
-like(attributes/key1,"known-chars-at-start*")
-
-like(attributes/key1,"*known-chars-in-between*")
-
+like(attributes/key1,"known-chars-at-start*")
+like(attributes/key1,"*known-chars-in-between*")
like(attributes/key1,"just-som?-char?-unkn?wn")
```
#### ilike
-Filter property values which are like (similar) and case insensitive ``.
-```
-ilike(,)
+Filter property values matching a pattern, case-insensitively (Ditto-specific operator).
+
+```text
+ilike(,)
```
{% include note.html content="The `ilike` operator is not defined in the linked RQL grammar, it is a Ditto
specific operator." %}
-**Details concerning the ilike-operator**
-
-The `ilike` operator provides some regular expression capabilities for pattern matching Strings with case insensitivity.
-
-The following expressions are supported:
+Supports the same patterns as `like`, but ignores case:
-* \*endswith => match at the end of a specific String.
-* startswith\* => match at the beginning of a specific String.
-* \*contains\* => match if contains a specific String.
-* Th?ng => match for a wildcard character.
-
-**Examples**
-```
+```text
ilike(attributes/key1,"*known-CHARS-at-end")
-
-ilike(attributes/key1,"known-chars-AT-start*")
-
-ilike(attributes/key1,"*KNOWN-CHARS-IN-BETWEEN*")
-
+ilike(attributes/key1,"known-chars-AT-start*")
+ilike(attributes/key1,"*KNOWN-CHARS-IN-BETWEEN*")
ilike(attributes/key1,"just-som?-char?-unkn?wn")
```
#### exists
-Filter property values which exist.
+Filter for properties that exist (Ditto-specific operator).
-```
+```text
exists()
```
{% include note.html content="The `exists` operator is not defined in the linked RQL grammar, it is a Ditto
specific operator." %}
+**Example** -- filter Things that have a Feature with ID "feature_1":
-**Example - filter things which have a feature with ID "feature_1"**
-```
+```text
exists(features/feature_1)
```
-**Example - filter lamps which are located in the "living-room"**
-```
+**Example** -- filter lamps located in the "living-room":
+
+```text
and(exists(features/lamp),eq(attributes/location,"living-room"))
```
@@ -311,79 +294,82 @@ or(empty(attributes/tags),eq(attributes/tags,"default"))
### Logical operators
#### and
-Ensure that all given queries match.
-```
-and(,, ...)
-```
+All given queries must match.
-**Example - filter things which are located on the "upper floor" in the "living-room"**
+```text
+and(,, ...)
```
+
+**Example** -- filter Things on the "upper floor" in the "living-room":
+
+```text
and(eq(attributes/floor,"upper floor"),eq(attributes/location,"living-room"))
```
-
+
#### or
-At least one of the given queries match.
-```
-or(,, ...)
-```
+At least one of the given queries must match.
-**Example - filter all things located on the "upper floor", and all things with location "living-room"**
+```text
+or(,, ...)
```
+
+**Example** -- filter Things on the "upper floor" or in the "living-room":
+
+```text
or(eq(attributes/floor,"upper floor"),eq(attributes/location,"living-room"))
```
#### not
-Negates the given query.
-```
-not()
-```
+Negate the given query.
-**Example - filter things whose ID do not start with a common prefix**
+```text
+not()
```
+
+**Example** -- filter Things whose ID does not start with a common prefix:
+
+```text
not(like(thingId,"org.eclipse.ditto:blocked*"))
```
## RQL sorting
-The RQL sorting part specifies in which order the result should be returned.
+The sorting part specifies result order:
-```
+```text
sort(<+|->,<+|->,...)
```
-* Use **+** for an ascending sort order (URL encoded character **%2B**)
-* Use **-** for a descending sort order (URL encoded character **%2D**)
+* Use **+** for ascending order (URL encoded: `%2B`)
+* Use **-** for descending order (URL encoded: `%2D`)
-**Example - sort the list ascending by the thing ID**
-```
-sort(+thingId)
-```
+**Examples**:
-**Example - sort the list ascending by an attribute**
-```
+```text
+sort(+thingId)
sort(+attributes/location)
-```
-
-**Example - multiple sort options**
-```
sort(-attributes/location,+thingId)
```
-This expression will sort the list descending by location attribute.
-In case there are multiple things with the same location attribute, these are sorted ascending by their ID.
+The last example sorts descending by location, then ascending by Thing ID for ties.
### Sorting of string values
-{% include note.html content="Sorting does not support a special \"semantics\" of string comparison (e.g. regarding
- alphabetical or lexicographical ordering). However, you can rely on the alphabetical sorting of strings with the
+{% include note.html content="Sorting does not support a special \"semantics\" of string comparison (e.g. regarding
+ alphabetical or lexicographical ordering). However, you can rely on the alphabetical sorting of strings with the
same length (e.g. \"aaa\" < \"zzz\") and that the order stays the same over multiple/different filter requests." %}
### Sorting of other values
-{% include note.html content="Sorting does not support a special \"semantics\" of comparison for fields with values of
- different data types (e.g. numbers vs. strings). However, you can rely on the fact that values of the same type are
+{% include note.html content="Sorting does not support a special \"semantics\" of comparison for fields with values of
+ different data types (e.g. numbers vs. strings). However, you can rely on the fact that values of the same type are
sorted respectively." %}
-
+
+## Further reading
+
+- [Search](basic-search.html) -- search concepts and paging
+- [Conditional requests](basic-conditional-requests.html) -- using RQL as conditions for updates
+- [Change notifications](basic-changenotifications.html) -- filtering notifications with RQL
diff --git a/documentation/src/main/resources/pages/ditto/basic-search.md b/documentation/src/main/resources/pages/ditto/basic-search.md
index 26078db69f2..d2793b543e8 100644
--- a/documentation/src/main/resources/pages/ditto/basic-search.md
+++ b/documentation/src/main/resources/pages/ditto/basic-search.md
@@ -5,93 +5,75 @@ tags: [search, rql]
permalink: basic-search.html
---
-Ditto provides a search functionality as one of the services around its managed **digital twins**.
-The functionality is available for the following APIs.
+Ditto provides a search service that lets you query across all managed digital twins using [RQL expressions](basic-rql.html).
+
+{% include callout.html content="**TL;DR**: You search Things using RQL filter expressions, with results sorted and paged. The search index updates asynchronously (eventual consistency) -- typically within 1-2 seconds." type="primary" %}
+
+## Overview
+
+You can access the search functionality through three APIs:
| API | Access Method | Characteristics |
-|-----|---------------|-----------------|
-|[HTTP](httpapi-search.html)|HTTP request-response|Stateless|
-|[Ditto protocol](protocol-specification-things-search.html)|[Websocket](httpapi-protocol-bindings-websocket.html) and [connections](basic-connections.html)| [Reactive-streams](https://reactive-streams.org) compatible |
-|[Server-sent events](httpapi-sse.html#sse-api-searchthings)|[HTML5 server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html)|Streaming with resumption|
+|---|---|---|
+| [HTTP](httpapi-search.html) | HTTP request-response | Stateless |
+| [Ditto protocol](protocol-specification-things-search.html) | [WebSocket](httpapi-protocol-bindings-websocket.html) and [connections](basic-connections.html) | [Reactive-streams](https://reactive-streams.org) compatible |
+| [Server-sent events](httpapi-sse.html#sse-for-search-results) | [HTML5 server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html) | Streaming with resumption |
-## Search index
+## How it works
-Ditto's microservice [things-search](architecture-services-things-search.html) automatically consumes all
-[events](basic-signals-event.html) which are emitted for changes to `Things` and `Policies` and updates an for search
-optimized representation of the `Thing` data into its own database.
+### Search index
-No custom indexes have to be defined as the structure in the database is "flattened" so that all data contained in
-[Things](basic-thing.html) can be searched for efficiently.
+Ditto's [things-search](architecture-services-things-search.html) microservice automatically consumes all [events](basic-signals-event.html) emitted for changes to `Things` and `Policies`. It updates a search-optimized representation in its own database.
-## Consistency
+You do not need to define custom indexes. The database structure is "flattened" so that all data in [Things](basic-thing.html) can be searched efficiently.
-Ditto's search index provides **eventual consistency**.
+### Consistency
-In order to reduce load to the database when processing updates in a high frequency, the search index is updated
-with a default interval of 1 second (configurable via environment variable `THINGS_SEARCH_UPDATER_STREAM_WRITE_INTERVAL`).
+The search index provides **eventual consistency**. Updates are written at a default interval of 1 second (configurable via `THINGS_SEARCH_UPDATER_STREAM_WRITE_INTERVAL`).
-That means that when a thing is updated and the API (e.g. the HTTP endpoint) returns a success response, the search index
-will not reflect that change in that instant. The change will most likely be reflected in the search index within
-1-2 seconds. In rare cases the duration until consistency is reached again might be higher.
+When you update a Thing and receive a success response, the search index does not reflect that change immediately. The change typically appears within 1-2 seconds.
-If it is important to know when a twin modification is reflected in the search index, request the
-[built-in acknowledgement](basic-acknowledgements.html#built-in-acknowledgement-labels) `search-persisted`
-in the corresponding command.
-Search index update is successful if the status code of `search-persisted` in the command response is 204 "no content".
-Status codes at or above 400 indicate failed search index update due to client or server errors.
+If you need to know when a modification is reflected in the search index, request the [built-in acknowledgement](basic-acknowledgements.html#built-in-acknowledgement-labels) `search-persisted` in the command. A status code of `204` confirms successful index update. Status codes at or above `400` indicate failure.
## Search queries
-Queries can be made via Ditto's APIs ([HTTP](httpapi-search.html) or
-[Ditto Protocol](protocol-specification-things-search.html) e.g. via [WebSocket](httpapi-protocol-bindings-websocket.html)).
+You formulate search queries using the Ditto-supported subset of [RQL](basic-rql.html). Queries work through the [HTTP API](httpapi-search.html) or the [Ditto Protocol](protocol-specification-things-search.html) (e.g., via [WebSocket](httpapi-protocol-bindings-websocket.html)).
-Search queries are formulated using the by Ditto supported subset of [RQL](basic-rql.html).
+### Querying scalar JSON values
-### Search queries on scalar JSON values
+The [query property](basic-rql.html#query-property) can target scalar JSON values: booleans, numbers, or strings.
-The [query property](basic-rql.html#query-property) used in a search can contain either a scalar JSON value:
-* JSON boolean
-* JSON number
-* JSON string
+**Example** -- find all Things located in "living-room", sorted ascending by Thing ID, returning the first 5 results:
-**Example:** Search for all things located in "living-room", reorder the list to start with the lowest thing ID as
-the first element, and return the first 5 results:
-```
+```text
Filter: eq(attributes/location,"living-room")
Sorting: sort(+thingId)
Paging: size(5),cursor(CURSOR_ID)
```
-### Search queries in JSON arrays
+### Querying JSON arrays
-Or the [query property](basic-rql.html#query-property) used in a search it can also contain a JSON array.
-The search index will index any values in that array, even arrays if mixed types are supported.
+The query property can also target JSON arrays. Ditto indexes all values in the array, including mixed types.
+
+Given a Thing with tags of different types:
-For example, assuming that we have the following thing containing special "tags" of different types:
```json
{
"thingId": "org.eclipse.ditto:tagged-thing-1",
"policyId": "org.eclipse.ditto:tagged-thing-1",
"attributes": {
"tags": [
- "misc",
- "no-due-date",
- "high-priority",
- 2,
- 3,
- 5,
- false,
- {
- "room": "kitchen",
- "floor": 2
- }
+ "misc", "no-due-date", "high-priority",
+ 2, 3, 5, false,
+ { "room": "kitchen", "floor": 2 }
]
}
}
```
-We can formulate various different queries on different scalar values:
-```
+You can query against scalar values in the array:
+
+```text
eq(attributes/tags,"high-priority")
-> match: "high-priority" is contained
@@ -111,97 +93,94 @@ gt(attributes/tags,6)
-> no match: as no number > 6 is contained
```
-And we can even formulate queries on JSON objects contained in the JSON array:
-```
-exists(attributes/tags/room)
--> match: array contains one object having a key "room"
+You can also query JSON objects inside the array:
-eq(attributes/tags/room,"kitchen")
--> match: array contains one object with "room"="kitchen"
-
-ge(attributes/tags/floor,2)
--> match: array contains one object where floor is >= 2
+```text
+exists(attributes/tags/room) -> match
+eq(attributes/tags/room,"kitchen") -> match
+ge(attributes/tags/floor,2) -> match
```
-
## Search count queries
-The same syntax applies for search count queries - only the [sorting](basic-rql.html#rql-sorting) and
-[paging](#rql-paging-deprecated) makes no sense here, so there are not necessary to specify.
-
+Count queries use the same filter syntax. Sorting and paging options are not applicable for count queries.
## Namespaces
-The Search supports specifying in which `namespaces` it should be searched. This may significantly improve the search
-performance when many Things of different namespaces are managed in Ditto's search index.
-
+You can restrict the search to specific `namespaces`. This can significantly improve performance when many Things from different namespaces exist in the index.
## RQL
-In order to apply queries when searching, Ditto uses the [RQL notation](basic-rql.html) which is also applied for other
-scenarios (e.g. filtering [notifications](basic-changenotifications.html)).
-
+Ditto uses [RQL notation](basic-rql.html) for search queries and other scenarios such as filtering [notifications](basic-changenotifications.html).
## Sorting and paging options
-The [`sort` option](basic-rql.html#rql-sorting) governs the order of search results.
+### Sorting
-```
+The [`sort` option](basic-rql.html#rql-sorting) controls the order of search results:
+
+```text
sort(<+|->,<+|->,...)
```
-If not given, search results are listed in the ascending order of thing IDs, namely `sort(+thingId)`.
+If not specified, results are sorted ascending by Thing ID: `sort(+thingId)`.
-The `size` option
-```
+### Paging with cursor
+
+The `size` option limits results per response:
+
+```text
size()
```
-limits the search results delivered in one HTTP response or one Ditto protocol message to `` items.
-If the paging option is not explicitly specified a **default value** of _25_ is used.
-The **maximum** allowed count is _200_.
+Default: **25**. Maximum: **200**.
-```
+Use cursor-based paging to iterate through large result sets:
+
+```text
cursor()
```
-Starts the search at the position of the cursor with ID ``. The cursor ID is obtained from the field
-`cursor` of a previous response and marks the **position after the last entry** of the previous search. A response
-includes no cursor if there are no more results.
-If a request has a `cursor` option, then any included `filter` or `sort` option may not differ from the original request
-of the cursor. Otherwise, the request is rejected.
+The cursor ID comes from the `cursor` field of a previous response and marks the position after the last returned entry. No cursor in the response means no more results.
-**Example - return ten items with a cursor**
-```
+If a request includes a `cursor`, the `filter` and `sort` options must match the original request that produced the cursor.
+
+**Example** -- return ten items with a cursor:
+
+```text
option=size(10),cursor()
```
## RQL paging (deprecated)
-{% include note.html content="The limit option is deprecated, it may be removed in future releases. Use [cursor-based
+{% include note.html content="The limit option is deprecated, it may be removed in future releases. Use [cursor-based
paging](basic-search.html#sorting-and-paging-options) instead." %}
-The RQL limiting part specifies which part (or "page") should be returned of a large search result set.
+The `limit` option specifies which page to return:
-```
+```text
limit(,)
```
-Limits the search results to `` items, starting with the item at index ``.
-* if the paging option is not explicitly specified, the **default** value `limit(0,25)` is used,
- i.e. the first `25` results are returned.
-* the **maximum** allowed count is `200`.
+Default: `limit(0,25)`. Maximum count: `200`.
-**Example - return the first ten items**
-```
+**Example** -- return the first ten items:
+
+```text
limit(0,10)
```
-**Example - return the items 11 to 20**
-```
+**Example** -- return items 11 to 20:
+
+```text
limit(10,10)
```
-i.e. Return the next ten items (from index 11 to 20)
{% include note.html content="We recommend **not to use high offsets** (e.g. higher than 10000) for paging
because of potential performance degradations." %}
+
+## Further reading
+
+- [RQL expressions](basic-rql.html) -- query language reference
+- [Search protocol](protocol-specification-things-search.html) -- reactive-streams search via Ditto Protocol
+- [HTTP search API](httpapi-search.html) -- REST-based search
diff --git a/documentation/src/main/resources/pages/ditto/basic-signals-announcement.md b/documentation/src/main/resources/pages/ditto/basic-signals-announcement.md
index d79ef0a6970..607d1b4d56e 100644
--- a/documentation/src/main/resources/pages/ditto/basic-signals-announcement.md
+++ b/documentation/src/main/resources/pages/ditto/basic-signals-announcement.md
@@ -5,11 +5,4 @@ tags: [signal]
permalink: basic-signals-announcement.html
---
-Announcements are special signals which are published in order to announce something before it actually happens.
-For example, before an [event](basic-signals-event.html) is created and published, an announcement could signal that
-the event will happen soon.
-
-Announcements have the following characteristics:
-* they are **not** persisted/appended into any data store
-* they are published to interested and authorized parties via the [WebSocket API](httpapi-protocol-bindings-websocket.html)
- as well as [connection targets](basic-connections.html#targets) via [change notifications](basic-changenotifications.html).
+{% include callout.html content="This content has been merged into the [Signals & Communication Pattern](basic-signals.html#announcements) page. Please refer to that page for full details on announcements." type="primary" %}
diff --git a/documentation/src/main/resources/pages/ditto/basic-signals-command.md b/documentation/src/main/resources/pages/ditto/basic-signals-command.md
index 435e9e424ef..d62688442fd 100644
--- a/documentation/src/main/resources/pages/ditto/basic-signals-command.md
+++ b/documentation/src/main/resources/pages/ditto/basic-signals-command.md
@@ -5,27 +5,4 @@ tags: [signal]
permalink: basic-signals-command.html
---
-Commands involve the need to change or retrieve something of a **digital twin** managed by Ditto or an actual device
-connected to Ditto.
-
-Commands always contain an identifier of the entity they address (e.g. a `Thing ID`).
-
-## Modify Commands
-
-Commands which modify a **digital twin** or an actual device are grouped as "Modify Commands".
-In CQRS system those are simply
-referred to as *commands*.
-
-An overview of all Thing related modify commands can be found in the appropriate chapter of the Ditto Protocol:
-* [Create/Modify protocol specification](protocol-specification-things-create-or-modify.html),
-* [Merge protocol specification](protocol-specification-things-merge.html),
-* [Delete protocol specification.](protocol-specification-things-delete.html)
-
-## Query Commands
-
-Commands which only retrieve information about a **digital twin** or an actual device are grouped as "Query Commands".
-In CQRS system those are simply
-referred to as *queries*.
-
-An overview of all Thing related query commands can be found in the chapter
-["Retrieve protocol specification"](protocol-specification-things-retrieve.html) of the Ditto Protocol.
+{% include callout.html content="This content has been merged into the [Signals & Communication Pattern](basic-signals.html#commands) page. Please refer to that page for full details on commands, including modify commands and query commands." type="primary" %}
diff --git a/documentation/src/main/resources/pages/ditto/basic-signals-commandresponse.md b/documentation/src/main/resources/pages/ditto/basic-signals-commandresponse.md
index 1f72bf2a9d0..0c22e7558f1 100644
--- a/documentation/src/main/resources/pages/ditto/basic-signals-commandresponse.md
+++ b/documentation/src/main/resources/pages/ditto/basic-signals-commandresponse.md
@@ -5,8 +5,4 @@ tags: [signal]
permalink: basic-signals-commandresponse.html
---
-CommandResponses are the answer to [Commands](basic-signals-command.html) and include information about whether the
-intention of changing something via a `ModifyCommand` has worked or if there was an [Error](basic-signals-errorresponse.html)
-instead.
-
-The CommandResponse of QueryCommands contains the requested information.
+{% include callout.html content="This content has been merged into the [Signals & Communication Pattern](basic-signals.html#command-responses) page. Please refer to that page for full details on command responses." type="primary" %}
diff --git a/documentation/src/main/resources/pages/ditto/basic-signals-errorresponse.md b/documentation/src/main/resources/pages/ditto/basic-signals-errorresponse.md
index 3d87bbf15fc..4e829f57994 100644
--- a/documentation/src/main/resources/pages/ditto/basic-signals-errorresponse.md
+++ b/documentation/src/main/resources/pages/ditto/basic-signals-errorresponse.md
@@ -5,14 +5,4 @@ tags: [signal]
permalink: basic-signals-errorresponse.html
---
-If an issued [command](basic-signals-command.html) or [message](basic-messages.html) could not be applied, an
-appropriate error response conveys this [error](basic-errors.html) information back to the issuer.
-Failure of a command or message can have various reasons, starting from missing permissions to internal server errors
-during processing of the command.
-
-The [Ditto Protocol for Errors](protocol-specification-errors.html) defines how error responses look in Ditto Protocol.
-
-An overview of some possible error responses can be found in the examples chapters:
-* [Things error response examples](protocol-examples-errorresponses.html)
-* [Policies error response examples](protocol-examples-policies-errorresponses.html)
-
+{% include callout.html content="This content has been merged into the [Signals & Communication Pattern](basic-signals.html#error-responses) page. Please refer to that page for full details on error responses." type="primary" %}
diff --git a/documentation/src/main/resources/pages/ditto/basic-signals-event.md b/documentation/src/main/resources/pages/ditto/basic-signals-event.md
index 9741528f3a1..75feaad6971 100644
--- a/documentation/src/main/resources/pages/ditto/basic-signals-event.md
+++ b/documentation/src/main/resources/pages/ditto/basic-signals-event.md
@@ -5,13 +5,4 @@ tags: [signal]
permalink: basic-signals-event.html
---
-Events report that something took place in a **digital twin** in Ditto. Important is the "past tense" here; it took
-already place (it was for example persisted into the data store) and cannot be reversed or stopped.
-
-Events are one of the centerpieces of Ditto:
-* they are persisted/appended into the data store,
-* they are published in the Ditto cluster, so other Ditto back end services can react on them (e.g. in order to update
- the search index) and
-* they are published to interested and authorized parties via the [WebSocket API](httpapi-protocol-bindings-websocket.html) as
- well as via [HTTP Server Sent Events](httpapi-sse.html) as well as [connection targets](basic-connections.html#targets)
- via [change notifications](basic-changenotifications.html).
+{% include callout.html content="This content has been merged into the [Signals & Communication Pattern](basic-signals.html#events) page. Please refer to that page for full details on events." type="primary" %}
diff --git a/documentation/src/main/resources/pages/ditto/basic-signals.md b/documentation/src/main/resources/pages/ditto/basic-signals.md
index 5ff6a19fcc8..daf870babb4 100644
--- a/documentation/src/main/resources/pages/ditto/basic-signals.md
+++ b/documentation/src/main/resources/pages/ditto/basic-signals.md
@@ -1,59 +1,148 @@
---
-title: Signals
-keywords: command, communication, CQRS, DDD, event, EventSourcing, response, signal
+title: Signals & Communication Pattern
+keywords: command, communication, CQRS, DDD, event, EventSourcing, response, signal, announcement
tags: [signal]
permalink: basic-signals.html
---
-Ditto has a concept called `Signal` which combines common functionality of
-* [Commands](basic-signals-command.html),
-* [Command Responses](basic-signals-commandresponse.html),
-* [Error Responses](basic-signals-errorresponse.html),
-* [Events](basic-signals-event.html) and
-* [Announcements](basic-signals-announcement.html).
+Signals are the messages that flow through Ditto. Every interaction -- creating a Thing, querying
+a property, receiving a change notification -- is carried by a signal.
-Such common functionality is for example that all those have header fields in which they can be for example correlated
-to each other.
+{% include callout.html content="**TL;DR**: Ditto uses five signal types: Commands (requests to read or change data),
+Command Responses (success replies), Error Responses (failure replies), Events (records of changes
+that already happened), and Announcements (advance notices of upcoming changes)." type="primary" %}
-Signals are one of the core concepts of Ditto but they mostly are used internally for communication in the Ditto
-cluster.
-Nevertheless it is very helpful to have a basic understanding of what the Signal types are and in which communication
-pattern they occur.
+## Overview
+Ditto follows the CQRS and
+Event Sourcing architectural patterns.
+[This page](https://cqrs.nu/Faq) provides a good explanation of the basic concepts:
-## Architectural style
-
-Ditto uses Commands, Events,
-CQRS and EventSourcing.
-[This page](https://cqrs.nu/Faq) provides a quite good explanation of the basic concepts on all of those aspects:
-
-### Command
-
-> People request changes to the domain by sending commands.
-They are named with a verb in the imperative mood plus and may include the aggregate type, for example `ConfirmOrder`.
-Unlike an event, a command is not a statement of fact; it's only a request, and thus may be refused.
-(A typical way to convey refusal is to throw an exception).
-
-### Event
+> People request changes to the domain by sending commands.
+They are named with a verb in the imperative mood plus and may include the aggregate type, for example
+`ConfirmOrder`. Unlike an event, a command is not a statement of fact; it's only a request, and thus may be refused.
> An event represents something that took place in the domain.
-They are always named with a past-participle verb, such as `OrderConfirmed`.
-It's not unusual but also not required for an event to name an aggregate or entity that it relates to; let the domain
-language be your guide.
+They are always named with a past-participle verb, such as `OrderConfirmed`.
Since an event represents something in the past, it can be considered a statement of fact and used to take decisions in
other parts of the system.
+Every change goes through a well-defined flow:
-## Communication pattern
-
-1. A **command** is sent to Ditto where it is then processed.
-2. Either a **success response** or an **error response** is sent back to the issuer of the **command**.
-3. In addition an **event** is both persisted into the datastore and published.
- The event describes that the change was applied to an entity (e.g. a `Thing`).
- Interested parties can subscribe for such **events** and follow the evolving entity.
+1. A **Command** arrives requesting a change or a query.
+2. Ditto processes it and sends back either a **Command Response** (success) or an **Error Response**
+ (failure).
+3. If the command modified data, Ditto persists an **Event** and publishes it to subscribers.
{% include note.html
- content="Events caused by commands from a **[connection](basic-connections.html)** or a
- [websocket session](httpapi-protocol-bindings-websocket.html) are not published
- **to the same origin**. The connection can receive a response, but will not additionally get an event."
+ content="Events caused by commands from a **[connection](basic-connections.html)** or a
+ [WebSocket session](httpapi-protocol-bindings-websocket.html) are not published
+ **to the same origin**. The connection receives the response, but not the event."
%}
+
+All signal types share common header fields (like correlation IDs) that let you trace a request
+through its entire lifecycle.
+
+## Commands
+
+Commands are requests to change or retrieve data from a digital twin or a connected device. Every
+command targets a specific entity by its ID (for example, a Thing ID).
+
+See [Commands detail](basic-signals-command.html) for the full command lifecycle.
+
+### Modify commands
+
+Modify commands change the state of a digital twin or trigger an action on a device. In CQRS
+terminology, these are the "commands" (write side).
+
+Related protocol specifications:
+* [Create/Modify protocol specification](protocol-specification-things-create-or-modify.html)
+* [Merge protocol specification](protocol-specification-things-merge.html)
+* [Delete protocol specification](protocol-specification-things-delete.html)
+
+### Query commands
+
+Query commands retrieve data without changing anything. In CQRS terminology, these are the
+"queries" (read side).
+
+Related protocol specification:
+* [Retrieve protocol specification](protocol-specification-things-retrieve.html)
+
+## Command Responses
+
+A command response is the reply to a command. It tells you whether the operation succeeded:
+
+* For **modify commands**: the response confirms the change was applied (for example, `201 Created`
+ or `204 No Content`).
+* For **query commands**: the response contains the requested data.
+
+If something goes wrong, you receive an [error response](#error-responses) instead.
+
+See [Command Responses detail](basic-signals-commandresponse.html) for response structure and examples.
+
+## Error Responses
+
+When a command or [message](basic-messages.html) fails, Ditto sends an error response explaining
+what went wrong. Failures can happen for many reasons:
+
+* Missing permissions
+* Entity not found
+* Invalid input data
+* Internal server errors
+
+The [Ditto Protocol for Errors](protocol-specification-errors.html) defines the error response format.
+See also [Error Responses detail](basic-signals-errorresponse.html).
+
+Example error responses:
+* [Things error responses](protocol-examples-errorresponses.html)
+* [Policies error responses](protocol-examples-policies-errorresponses.html)
+
+## Events
+
+Events record that something **already happened**. They are past tense and irreversible -- the
+change is already persisted.
+
+See [Events detail](basic-signals-event.html) for event structure and persistence.
+
+Events serve three purposes in Ditto:
+
+1. **Persistence** -- appended to the event journal (event sourcing) as the source of truth
+2. **Internal coordination** -- published within the Ditto cluster so services can react (for
+ example, the search index updates itself based on events)
+3. **External notification** -- delivered to authorized subscribers via the
+ [WebSocket API](httpapi-protocol-bindings-websocket.html),
+ [HTTP Server Sent Events](httpapi-sse.html), and
+ [connection targets](basic-connections.html#targets) as
+ [change notifications](basic-changenotifications.html)
+
+## Announcements
+
+Announcements signal that something **is about to happen**. Unlike events, announcements are
+forward-looking and are **not** persisted.
+
+For example, before a [Policy subject expires](basic-policy.html#expiring-subjects), Ditto can
+publish an announcement so your application can react -- perhaps by renewing the subject's
+credentials.
+
+Announcements are published to authorized subscribers via the
+[WebSocket API](httpapi-protocol-bindings-websocket.html) and
+[connection targets](basic-connections.html#targets).
+
+See [Announcements detail](basic-signals-announcement.html) for announcement types and examples.
+
+## Communication pattern summary
+
+| Step | Signal type | Direction | Persisted? |
+|------|-------------|-----------|------------|
+| 1 | Command | Client to Ditto | No |
+| 2a | Command Response | Ditto to client | No |
+| 2b | Error Response | Ditto to client (on failure) | No |
+| 3 | Event | Ditto to subscribers | Yes |
+| -- | Announcement | Ditto to subscribers (preemptive) | No |
+
+## Further reading
+
+* [Change Notifications](basic-changenotifications.html) -- subscribe to events via different APIs
+* [Signal Enrichment](basic-enrichment.html) -- add extra context to events
+* [Messages](basic-messages.html) -- send arbitrary payloads to/from devices
+* [Ditto Protocol specification](protocol-specification.html) -- wire format for all signal types
diff --git a/documentation/src/main/resources/pages/ditto/basic-thing.md b/documentation/src/main/resources/pages/ditto/basic-thing.md
index af3e28c96df..b030f196800 100644
--- a/documentation/src/main/resources/pages/ditto/basic-thing.md
+++ b/documentation/src/main/resources/pages/ditto/basic-thing.md
@@ -1,77 +1,85 @@
---
-title: Thing
+title: Things
keywords: entity, feature, model, namespace, thing
tags: [model]
permalink: basic-thing.html
---
-The versatile assets in IoT applications can be managed as Things.
+A Thing is Ditto's core entity. It represents any asset you want to manage as a digital twin -- a
+physical device, a virtual grouping, or any concept you can model as structured data.
+{% include callout.html content="**TL;DR**: A Thing is a JSON object with an ID, a Policy reference, optional
+attributes (static metadata), and optional features (dynamic state data)." type="primary" %}
-## Thing
+## What is a Thing?
-Things are very generic entities and are mostly used as a “handle” for multiple features belonging to this Thing.
+A Thing is a generic container that acts as a handle for all the data related to one asset. You
+decide what a Thing represents:
-Examples:
-
-* Physical Device: a lawn mower, a sensor, a vehicle, a lamp.
-* Virtual Device: a room in a house, a virtual power plant spanning multiple power plants, the weather information for
- a specific location collected by various sensors.
-* Transactional entity: a tour of a vehicle (from start until stop), a series of measurements of a machine.
-* Master data entity: a supplier of devices or a service provider for devices, an entity representing a city/region.
-* Anything else - if it can be modeled and managed appropriately by the supported concepts/capabilities.
+* **Physical device** -- a sensor, a lamp, a vehicle, a lawn mower
+* **Virtual device** -- a room in a building, a virtual power plant, weather data for a location
+* **Transactional entity** -- a vehicle trip from start to stop, a series of machine measurements
+* **Master data entity** -- a device supplier, a service provider, a geographic region
+* **Anything else** that fits the [data model](basic-overview.html)
+## Thing structure
### Thing ID
-Unique identifier of a Thing. For choosing custom Thing IDs when creating a Thing, the rules for
-[namespaced IDs](basic-namespaces-and-names.html#namespaced-id) apply.
-
-### Access control
+Every Thing has a unique identifier that follows the [namespaced ID](basic-namespaces-and-names.html#namespaced-id)
+format: `:`. For example: `com.example:temperature-sensor-42`.
-A Thing in API version 2 contains a link to a [Policy](basic-policy.html) in form of a `policyId`.
-This Policy defines which
-authenticated subjects may READ and WRITE the Thing or even parts of it (hierarchically specified).
+### Policy reference
+Each Thing links to a [Policy](basic-policy.html) via its `policyId`. The
+Policy defines which
+authenticated subjects may read and write the Thing -- down to individual attributes and feature
+properties.
### Definition
-A Thing may contain a definition. The definition can also be used to find Things. The definition is used to link a thing
-to a corresponding model defining the capabilities/features of it.
-The definition can for example point to a:
-* a valid HTTP(s) URL (e.g. in order to define that the Thing is described by a [WoT (Web of Things) Thing Model](basic-wot-integration.html#thing-model-describing-a-ditto-thing))
-* or some other model reference using the syntax `::`
+A Thing can include a `definition` that links it to a model describing its capabilities. The
+definition can be:
+* A valid HTTP(s) URL -- for example pointing to a [WoT (Web of Things) Thing Model](basic-wot-integration.html#thing-model-describing-a-ditto-thing)
+* A reference using the syntax `::` -- for example `org.eclipse.ditto:HeatingDevice:2.1.0`
### Attributes
-Attributes describe the Thing in more detail and can be of any type. Attributes can also be used to find Things.
-Attributes are typically used to model rather static properties at the Thing level. Static means that the values do not
-change as frequently as property values of Features.
+Attributes store static metadata about the Thing as a JSON object. Use them for data that does
+not change frequently -- manufacturer, serial number, installation location. You can nest values
+to any depth:
+```json
+{
+ "manufacturer": "ACME corp",
+ "location": {
+ "building": "HQ",
+ "floor": 3,
+ "room": "Lab-A"
+ }
+}
+```
### Features
-A Thing may contain an arbitrary amount of [Features](basic-feature.html).
+A Thing can contain any number of [Features](basic-feature.html). Each Feature groups related
+dynamic state data (sensor readings, configuration, operational status) under a named identifier.
{% include image.html file="pages/basic/ditto-thing-feature.png" alt="Thing and Feature" caption="One Thing can have
many Features" max-width=100 %}
-
### Metadata
-A Thing may contain additional [metadata](basic-metadata.html) for all of its attributes and features describing the
-semantics of the data or adding other useful information about the data points of the twin.
-
+A Thing can carry additional [metadata](basic-metadata.html) attached to any of its attributes or
+feature properties -- for example, timestamps recording when values last changed or units of
+measurement.
-### Model specification
-
-#### API version 2
+## Model specification
{% include docson.html schema="jsonschema/thing_v2.json" %}
-
-### Example
+## Example
```json
{
@@ -79,29 +87,36 @@ semantics of the data or adding other useful information about the data points o
"policyId": "the.namespace:thePolicyName",
"definition": "org.eclipse.ditto:HeatingDevice:2.1.0",
"attributes": {
- "someAttr": 32,
- "manufacturer": "ACME corp"
+ "someAttr": 32,
+ "manufacturer": "ACME corp"
},
"features": {
- "heating-no1": {
- "properties": {
- "connected": true,
- "complexProperty": {
- "street": "my street",
- "house no": 42
- }
- },
- "desiredProperties": {
- "connected": false
- }
+ "heating-no1": {
+ "properties": {
+ "connected": true,
+ "complexProperty": {
+ "street": "my street",
+ "house no": 42
+ }
},
- "switchable": {
- "definition": [ "org.eclipse.ditto:Switcher:1.0.0" ],
- "properties": {
- "on": true,
- "lastToggled": "2017-11-15T18:21Z"
- }
+ "desiredProperties": {
+ "connected": false
}
+ },
+ "switchable": {
+ "definition": [ "org.eclipse.ditto:Switcher:1.0.0" ],
+ "properties": {
+ "on": true,
+ "lastToggled": "2017-11-15T18:21Z"
+ }
+ }
}
}
```
+
+## Further reading
+
+* [Features](basic-feature.html) -- properties, desired properties, and definitions
+* [Policies](basic-policy.html) -- access control for Things
+* [Namespaces & Names](basic-namespaces-and-names.html) -- ID format and naming rules
+* [Metadata](basic-metadata.html) -- attach contextual information to Thing data
diff --git a/documentation/src/main/resources/pages/ditto/basic-wot-integration-example.md b/documentation/src/main/resources/pages/ditto/basic-wot-integration-example.md
index f41e0e5c8eb..8131ff72a65 100644
--- a/documentation/src/main/resources/pages/ditto/basic-wot-integration-example.md
+++ b/documentation/src/main/resources/pages/ditto/basic-wot-integration-example.md
@@ -1,63 +1,60 @@
---
-title: WoT integration example
+title: WoT Integration Example
keywords: WoT, TD, TM, ThingDescription, ThingModel, W3C, Semantic, Model, definition, ThingDefinition, FeatureDefinition, example
tags: [wot]
permalink: basic-wot-integration-example.html
---
-Wrapping up the [WoT integration](basic-wot-integration.html) with a practical example.
+This page walks you through a complete WoT integration example -- from creating a Thing with a WoT Thing Model to inspecting the generated Thing Descriptions for both the Thing and its Features.
+
+{% include callout.html content="**TL;DR**: Create a Thing with a `definition` pointing to a WoT Thing Model URL. Ditto generates the full JSON skeleton automatically. Then request `Accept: application/td+json` to get a complete Thing Description with API endpoints." type="primary" %}
{% include tip.html content="To experiment with Thing Models and having them exposed as HTTP resources, simply create them as a [GitHub Gist](https://gist.github.com).
Each revision of the file will get a unique HTTP endpoint which you can use as endpoint for your Thing Model." %}
-## Thing Model
-
-You can provide a WoT Thing Model via any HTTP(s) URL addressable endpoint, for example simply put your WoT TMs into
-a GitHub repository.
-For this example, Ditto added a model into its `ditto-examples` GitHub Repo:
-[floor-lamp-1.0.0.tm.jsonld](https://github.com/eclipse-ditto/ditto-examples/blob/master/wot/models/floor-lamp-1.0.0.tm.jsonld)
-
-This file is available as HTTP served file at:
-[https://eclipse-ditto.github.io/ditto-examples/wot/models/floor-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/floor-lamp-1.0.0.tm.jsonld)
-
-The example model is composed of the following submodels:
-* [dimmable-colored-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/dimmable-colored-lamp-1.0.0.tm.jsonld), instanceName: "Spot1"
- * which `tm:extends` [colored-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/colored-lamp-1.0.0.tm.jsonld)
- * which `tm:extends` [switchable-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/switchable-1.0.0.tm.jsonld)
-* [dimmable-colored-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/dimmable-colored-lamp-1.0.0.tm.jsonld), instanceName: "Spot2"
- * which `tm:extends` [colored-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/colored-lamp-1.0.0.tm.jsonld)
- * which `tm:extends` [switchable-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/switchable-1.0.0.tm.jsonld)
-* [dimmable-colored-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/dimmable-colored-lamp-1.0.0.tm.jsonld), instanceName: "Spot3"
- * which `tm:extends` [colored-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/colored-lamp-1.0.0.tm.jsonld)
- * which `tm:extends` [switchable-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/switchable-1.0.0.tm.jsonld)
-* [connection-status-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/connection-status-1.0.0.tm.jsonld), instanceName: "ConnectionStatus"
-* [power-consumption-aware-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/power-consumption-aware-1.0.0.tm.jsonld), instanceName: "PowerConsumptionAwareness"
-* [smoke-detector-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/smoke-detector-1.0.0.tm.jsonld), instanceName: "SmokeDetection"
-* [colored-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/colored-lamp-1.0.0.tm.jsonld), instanceName: "Status-LED"
- * which `tm:extends` [switchable-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/switchable-1.0.0.tm.jsonld)
-
-Summarizing, our example model is the model of a "Floor Lamp" with:
-* 3 (dimmable, colored, switchable) Spots
-* a connection status indicating whether the lamp is currently connected
-* awareness of its current power consumption
-* an included smoke detector
-* and a (colored, switchable) status LED
-
-
-## Creating a new Thing based on the TM
-
-To create a Thing (instance) and create the Thing JSON skeleton following the WoT Thing Model, simply create
-a Thing via the Ditto HTTP API (e.g. `PUT /api/2/things/`):
+## Overview
+
+This example uses a "Floor Lamp" model that demonstrates WoT composition with multiple sub-models. You will:
+
+1. Understand the Thing Model structure
+2. Create a Thing that references the model
+3. Inspect the generated Thing Description for the Thing
+4. Inspect the generated Thing Description for a Feature
+
+## Step 1: Understand the Thing Model
+
+The example uses a floor lamp model hosted in the Ditto examples repository:
+[floor-lamp-1.0.0.tm.jsonld](https://eclipse-ditto.github.io/ditto-examples/wot/models/floor-lamp-1.0.0.tm.jsonld)
+
+Source: [GitHub](https://github.com/eclipse-ditto/ditto-examples/blob/master/wot/models/floor-lamp-1.0.0.tm.jsonld)
+
+This model composes several sub-models:
+
+| Sub-model | Instance name | Inherits from |
+|-----------|---------------|---------------|
+| [dimmable-colored-lamp](https://eclipse-ditto.github.io/ditto-examples/wot/models/dimmable-colored-lamp-1.0.0.tm.jsonld) | Spot1, Spot2, Spot3 | colored-lamp -> switchable |
+| [connection-status](https://eclipse-ditto.github.io/ditto-examples/wot/models/connection-status-1.0.0.tm.jsonld) | ConnectionStatus | -- |
+| [power-consumption-aware](https://eclipse-ditto.github.io/ditto-examples/wot/models/power-consumption-aware-1.0.0.tm.jsonld) | PowerConsumptionAwareness | -- |
+| [smoke-detector](https://eclipse-ditto.github.io/ditto-examples/wot/models/smoke-detector-1.0.0.tm.jsonld) | SmokeDetection | -- |
+| [colored-lamp](https://eclipse-ditto.github.io/ditto-examples/wot/models/colored-lamp-1.0.0.tm.jsonld) | Status-LED | switchable |
+
+The floor lamp has 3 dimmable colored spots, a connection status indicator, power consumption awareness, a smoke detector, and a status LED.
+
+## Step 2: Create a Thing from the model
+
+Send a `PUT` request with just the `definition` field pointing to the Thing Model URL. Ditto generates the entire JSON skeleton for you:
```bash
-curl --location --request PUT -u ditto:ditto 'https://ditto.eclipseprojects.io/api/2/things/io.eclipseprojects.ditto:floor-lamp-0815' \
---header 'Content-Type: application/json' \
---data-raw '{
+curl --location --request PUT -u ditto:ditto \
+ 'https://ditto.eclipseprojects.io/api/2/things/io.eclipseprojects.ditto:floor-lamp-0815' \
+ --header 'Content-Type: application/json' \
+ --data-raw '{
"definition": "https://eclipse-ditto.github.io/ditto-examples/wot/models/floor-lamp-1.0.0.tm.jsonld"
-}'
+ }'
```
-That should result in an HTTP status code `201` (Created) and return the following body:
+Ditto returns `201 Created` with the generated Thing:
+
```json
{
"thingId": "io.eclipseprojects.ditto:floor-lamp-0815",
@@ -76,11 +73,7 @@ That should result in an HTTP status code `201` (Created) and return the followi
],
"properties": {
"dimmer-level": 0.0,
- "color": {
- "r": 0,
- "g": 0,
- "b": 0
- },
+ "color": { "r": 0, "g": 0, "b": 0 },
"on": false
}
},
@@ -92,11 +85,7 @@ That should result in an HTTP status code `201` (Created) and return the followi
],
"properties": {
"dimmer-level": 0.0,
- "color": {
- "r": 0,
- "g": 0,
- "b": 0
- },
+ "color": { "r": 0, "g": 0, "b": 0 },
"on": false
}
},
@@ -108,11 +97,7 @@ That should result in an HTTP status code `201` (Created) and return the followi
],
"properties": {
"dimmer-level": 0.0,
- "color": {
- "r": 0,
- "g": 0,
- "b": 0
- },
+ "color": { "r": 0, "g": 0, "b": 0 },
"on": false
}
},
@@ -144,11 +129,7 @@ That should result in an HTTP status code `201` (Created) and return the followi
"https://eclipse-ditto.github.io/ditto-examples/wot/models/switchable-1.0.0.tm.jsonld"
],
"properties": {
- "color": {
- "r": 0,
- "g": 0,
- "b": 0
- },
+ "color": { "r": 0, "g": 0, "b": 0 },
"on": false
}
}
@@ -156,23 +137,24 @@ That should result in an HTTP status code `201` (Created) and return the followi
}
```
-You see that:
-* all of the `properties` defined in the `floor-lamp` itself were generated (as `attributes` in the Thing) with default values
-* all included `tm:submodel`s were generated as Features
-* all submodel `properties` were generated (as `properties` in the Feature) with default values
-* Feature `definition`s were set accordingly, including the extension hierarchy
-
+Notice what Ditto generated automatically:
+* **Attributes** from the TM's top-level `properties` (with empty string defaults)
+* **Features** from each `tm:submodel` (using `instanceName` as the Feature ID)
+* **Feature properties** from each submodel's `properties` (with default or neutral values)
+* **Feature definitions** including the full extension hierarchy
-## Inspecting the Thing Description of the Thing
+## Step 3: Inspect the Thing Description
-The Thing we just created can now be asked for its capabilities / interaction affordances by sending the following request:
+Request the Thing Description using the `Accept: application/td+json` header:
```bash
-curl --location --request GET -u ditto:ditto 'https://ditto.eclipseprojects.io/api/2/things/io.eclipseprojects.ditto:floor-lamp-0815' \
---header 'Accept: application/td+json'
+curl --location --request GET -u ditto:ditto \
+ 'https://ditto.eclipseprojects.io/api/2/things/io.eclipseprojects.ditto:floor-lamp-0815' \
+ --header 'Accept: application/td+json'
```
That should result in an HTTP status code `200` (OK) and return the following body:
+
```json
{
"@context": [
@@ -518,24 +500,27 @@ That should result in an HTTP status code `200` (OK) and return the following bo
}
```
-You see that:
-* a complete and valid WoT Thing Description was generated and returned
-* you can inspect which `properties`, `actions` and `events` the Thing level TD supports
-* in the `forms` section you have concrete API endpoints (defined relatively to the top-level `base`) which describe how to e.g. read a single property, or to observe properties, or how to invoke actions
-* you see in the `links` section that this TD contains other `"item"` relations with relative `"href"` link to the Thing's Features
-* you get also possible `uriVariables` and a `dittoError` `schemaDefinition` which may be returned when accessing a `property` or invoking an `action`
+Key observations:
+* A complete and valid WoT Thing Description was generated and returned
+* The `properties` section describes each attribute with its data type, read/write capabilities, and API forms
+* The `actions` section defines invocable operations with their HTTP endpoints
+* The `forms` section at root level provides bulk operations (read/write all properties, observe, subscribe)
+* The `links` section contains `"item"` relations pointing to each Feature's TD (relative to the `base` URL)
+* The `uriVariables` define query parameters (`channel`, `timeout`, `response-required`, `fields`)
+* The `schemaDefinitions` provide a `dittoError` schema for error responses
+## Step 4: Inspect a Feature's Thing Description
-## Inspecting the Thing Description of a Feature
-
-In order to inspect which capabilities / interaction affordances now a Feature provides, simply perform such a query:
+Request the TD for a specific Feature:
```bash
-curl --location --request GET -u ditto:ditto 'https://ditto.eclipseprojects.io/api/2/things/io.eclipseprojects.ditto:floor-lamp-0815/features/Spot1' \
---header 'Accept: application/td+json'
+curl --location --request GET -u ditto:ditto \
+ 'https://ditto.eclipseprojects.io/api/2/things/io.eclipseprojects.ditto:floor-lamp-0815/features/Spot1' \
+ --header 'Accept: application/td+json'
```
That should result in an HTTP status code `200` (OK) and return the following body:
+
```json
{
"@context": [
@@ -578,80 +563,44 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"href": "/properties{?channel,timeout}",
"htv:methodName": "GET",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "readmultipleproperties",
"href": "/properties{?fields,channel,timeout}",
"htv:methodName": "GET",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "writeallproperties",
"href": "/properties{?channel,timeout,response-required}",
"htv:methodName": "PUT",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "writemultipleproperties",
"href": "/properties{?channel,timeout,response-required}",
"htv:methodName": "PATCH",
"contentType": "application/merge-patch+json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
- "op": [
- "observeallproperties",
- "unobserveallproperties"
- ],
+ "op": ["observeallproperties", "unobserveallproperties"],
"href": "/properties",
"htv:methodName": "GET",
"subprotocol": "sse",
"contentType": "text/event-stream",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
- "op": [
- "subscribeallevents",
- "unsubscribeallevents"
- ],
+ "op": ["subscribeallevents", "unsubscribeallevents"],
"href": "/outbox/messages",
"htv:methodName": "GET",
"subprotocol": "sse",
"contentType": "text/event-stream",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
}
],
"properties": {
@@ -670,52 +619,29 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"href": "/properties/dimmer-level{?channel,timeout}",
"htv:methodName": "GET",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "writeproperty",
"href": "/properties/dimmer-level{?channel,timeout,response-required}",
"htv:methodName": "PUT",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "writeproperty",
"href": "/properties/dimmer-level{?channel,timeout,response-required}",
"htv:methodName": "PATCH",
"contentType": "application/merge-patch+json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
- "op": [
- "observeproperty",
- "unobserveproperty"
- ],
+ "op": ["observeproperty", "unobserveproperty"],
"href": "/properties/dimmer-level",
"htv:methodName": "GET",
"subprotocol": "sse",
"contentType": "text/event-stream",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
}
]
},
@@ -724,30 +650,11 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"description": "The current color.",
"type": "object",
"properties": {
- "r": {
- "title": "Red",
- "type": "integer",
- "minimum": 0,
- "maximum": 255
- },
- "g": {
- "title": "Green",
- "type": "integer",
- "minimum": 0,
- "maximum": 255
- },
- "b": {
- "title": "Blue",
- "type": "integer",
- "minimum": 0,
- "maximum": 255
- }
+ "r": { "title": "Red", "type": "integer", "minimum": 0, "maximum": 255 },
+ "g": { "title": "Green", "type": "integer", "minimum": 0, "maximum": 255 },
+ "b": { "title": "Blue", "type": "integer", "minimum": 0, "maximum": 255 }
},
- "required": [
- "r",
- "g",
- "b"
- ],
+ "required": ["r", "g", "b"],
"observable": true,
"forms": [
{
@@ -755,52 +662,29 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"href": "/properties/color{?channel,timeout}",
"htv:methodName": "GET",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "writeproperty",
"href": "/properties/color{?channel,timeout,response-required}",
"htv:methodName": "PUT",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "writeproperty",
"href": "/properties/color{?channel,timeout,response-required}",
"htv:methodName": "PATCH",
"contentType": "application/merge-patch+json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
- "op": [
- "observeproperty",
- "unobserveproperty"
- ],
+ "op": ["observeproperty", "unobserveproperty"],
"href": "/properties/color",
"htv:methodName": "GET",
"subprotocol": "sse",
"contentType": "text/event-stream",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
}
]
},
@@ -815,52 +699,29 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"href": "/properties/on{?channel,timeout}",
"htv:methodName": "GET",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "writeproperty",
"href": "/properties/on{?channel,timeout,response-required}",
"htv:methodName": "PUT",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
"op": "writeproperty",
"href": "/properties/on{?channel,timeout,response-required}",
"htv:methodName": "PATCH",
"contentType": "application/merge-patch+json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
},
{
- "op": [
- "observeproperty",
- "unobserveproperty"
- ],
+ "op": ["observeproperty", "unobserveproperty"],
"href": "/properties/on",
"htv:methodName": "GET",
"subprotocol": "sse",
"contentType": "text/event-stream",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
}
]
}
@@ -879,12 +740,7 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"href": "/inbox/messages/toggle{?timeout,response-required}",
"htv:methodName": "POST",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
}
]
},
@@ -903,12 +759,7 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"href": "/inbox/messages/switch-on-for-duration{?timeout,response-required}",
"htv:methodName": "POST",
"contentType": "application/json",
- "additionalResponses": [
- {
- "success": false,
- "schema": "dittoError"
- }
- ]
+ "additionalResponses": [{ "success": false, "schema": "dittoError" }]
}
]
}
@@ -918,10 +769,7 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"type": "string",
"title": "The Ditto channel to interact with.",
"description": "Defines to which channel to route the command: 'twin' (digital twin) or 'live' (the device).",
- "enum": [
- "twin",
- "live"
- ],
+ "enum": ["twin", "live"],
"default": "twin"
},
"timeout": {
@@ -979,12 +827,34 @@ That should result in an HTTP status code `200` (OK) and return the following bo
"maximum": 599
}
},
- "required": [
- "status",
- "error",
- "message"
- ]
+ "required": ["status", "error", "message"]
}
}
}
-```
\ No newline at end of file
+```
+
+Key observations:
+* A complete and valid WoT Thing Description was generated and returned
+* The `properties` section describes each attribute with its data type, read/write capabilities, and API forms
+* The `actions` section defines invocable operations with their HTTP endpoints
+* The `forms` section at root level provides bulk operations (read/write all properties, observe, subscribe)
+* The `links` section contains `"item"` relations pointing to each Feature's TD (relative to the `base` URL)
+* The `uriVariables` define query parameters (`channel`, `timeout`, `response-required`, `fields`)
+* The `schemaDefinitions` provide a `dittoError` schema for error responses
+
+## Summary
+
+In this walkthrough you:
+
+1. **Created a Thing** with just a `definition` URL -- Ditto generated all attributes, features, and properties
+2. **Retrieved a Thing Description** that documents every property, action, and event with concrete API endpoints
+3. **Retrieved a Feature Description** that provides detailed interaction affordances including data types, constraints, and API forms
+
+The generated Thing Descriptions are fully compliant WoT TDs that any WoT-compatible tool or library can consume.
+
+## Further reading
+
+* [WoT Overview](basic-wot-integration.html) -- concepts and configuration
+* [WoT Validation Configuration](basic-wot-validation-config.html) -- runtime validation API
+* [W3C WoT Thing Description 1.1](https://www.w3.org/TR/wot-thing-description11/) -- the specification
+* [Eclipse edi{TD}or](https://eclipse.github.io/editdor/) -- online WoT model editor
diff --git a/documentation/src/main/resources/pages/ditto/basic-wot-integration.md b/documentation/src/main/resources/pages/ditto/basic-wot-integration.md
index 15c984aa000..568f6d37f44 100644
--- a/documentation/src/main/resources/pages/ditto/basic-wot-integration.md
+++ b/documentation/src/main/resources/pages/ditto/basic-wot-integration.md
@@ -13,8 +13,7 @@ Using this integration, Ditto managed digital twins can be linked to WoT "Thing
WoT "Thing Descriptions" containing the API descriptions of the twins.
The WoT integration is considered stable and therefore active by default starting with Ditto version `3.0.0`.
-If it should be disabled, it can be deactivated via a "feature toggle":
-In order to deactivate the WoT integration, configure the following environment variable for all Ditto services:
+To disable the WoT integration, configure the following environment variable for all Ditto services:
```bash
DITTO_DEVOPS_FEATURE_WOT_INTEGRATION_ENABLED=false
@@ -49,12 +48,12 @@ But also other protocol bindings may be defined in a `form`, e.g. MQTT or CoAP.
The "WoT Thing Description" specification version 1.0 was already published as
["W3C Recommendation" in April 2020](https://www.w3.org/TR/wot-thing-description/), the next version 1.1 adds the
-concept of "Thing Models" (TM) which can be seen as a template for generating "Thing Descriptions" but without some of
+concept of "Thing Models" (TM) which serves as a template for generating "Thing Descriptions" but without some of
its mandatory fields, e.g. `forms` including the protocol bindings.
With the addition of the "Thing Model" concept, WoT becomes a perfect fit for describing the capabilities of
[Digital Twins](intro-digitaltwins.html) managed in Ditto.
-It is completely optional and even possible as "retrofit" model addition for already connected devices / already existing twins.
+You can optionally add it as a "retrofit" model addition for already connected devices / already existing twins.
The benefits of adding such a "Thing Model" reference to digital twins managed in Ditto are:
* possibility to define model for data (Ditto Thing `attributes` + Ditto Feature `properties`), e.g. containing:
@@ -103,7 +102,7 @@ This description contains not only the interaction capabilities (`properties` th
### WoT Thing Model
-A [Thing Model](https://www.w3.org/TR/wot-thing-description11/#introduction-tm) can be seen as the model
+A [Thing Model](https://www.w3.org/TR/wot-thing-description11/#introduction-tm) serves as the model
(or interface in OOP terminology) for a potentially huge population of instances (Thing Descriptions) all "implementing"
this contract.
It does not need to contain the instance specific parts which a TD must include (e.g. `security` definitions or `forms`).
@@ -254,8 +253,8 @@ This table shows an overview of how those elements map to Ditto concepts for the
|---------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|
| [Thing](https://www.w3.org/TR/wot-thing-description11/#thing) | [Ditto Thing](basic-thing.html) |
| [Properties](https://www.w3.org/TR/wot-thing-description11/#propertyaffordance) | Thing [attributes](basic-thing.html#attributes) |
-| [Actions](https://www.w3.org/TR/wot-thing-description11/#actionaffordance) | Thing [messages](basic-messages.html#elements) with **Direction** *to* (messages in the "inbox") of a Thing ID. |
-| [Events](https://www.w3.org/TR/wot-thing-description11/#eventaffordance) | Thing [messages](basic-messages.html#elements) with **Direction** *from* (messages in the "outbox") of a Thing ID. |
+| [Actions](https://www.w3.org/TR/wot-thing-description11/#actionaffordance) | Thing [messages](basic-messages.html#message-elements) with **Direction** *to* (messages in the "inbox") of a Thing ID. |
+| [Events](https://www.w3.org/TR/wot-thing-description11/#eventaffordance) | Thing [messages](basic-messages.html#message-elements) with **Direction** *from* (messages in the "outbox") of a Thing ID. |
| [Composition via `tm:submodel`](https://www.w3.org/TR/wot-thing-description11/#thing-model-composition) | Thing [features](basic-thing.html#features) representing different aspects of a Ditto Thing. |
@@ -267,8 +266,8 @@ This table shows an overview of how those elements map to Ditto concepts for the
|---------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Thing](https://www.w3.org/TR/wot-thing-description11/#thing) | Feature. In Ditto, a Feature is an aspect of a [Ditto Thing](basic-thing.html). As the Feature is defined by its properties and messages it supports, it maps to a WoT Thing. |
| [Properties](https://www.w3.org/TR/wot-thing-description11/#propertyaffordance) | Feature [properties](basic-feature.html#feature-properties) |
-| [Actions](https://www.w3.org/TR/wot-thing-description11/#actionaffordance) | Feature [messages](basic-messages.html#elements) with **Direction** *to* (messages in the "inbox") of a Thing ID + Feature ID combination. |
-| [Events](https://www.w3.org/TR/wot-thing-description11/#eventaffordance) | Feature [messages](basic-messages.html#elements) with **Direction** *from* (messages in the "outbox") of a Thing ID + Feature ID combination. |
+| [Actions](https://www.w3.org/TR/wot-thing-description11/#actionaffordance) | Feature [messages](basic-messages.html#message-elements) with **Direction** *to* (messages in the "inbox") of a Thing ID + Feature ID combination. |
+| [Events](https://www.w3.org/TR/wot-thing-description11/#eventaffordance) | Feature [messages](basic-messages.html#message-elements) with **Direction** *from* (messages in the "outbox") of a Thing ID + Feature ID combination. |
## Integration in Ditto
@@ -444,7 +443,7 @@ The implementation supports validation or enforcing the following Ditto concepts
follow the contract defined by the TM `properties`
* by default, non-modeled `attributes` are forbidden to be created/updated
* `attributes` which are not marked as optional (via `"tm:optional"`) are ensured not to be deleted
- * Thing [messages](basic-messages.html#sending-messages):
+ * Thing [messages](basic-messages.html#sending-and-receiving-messages):
* based on the linked TM of a Thing's [definition](basic-thing.html#definition), the Thing `messages`
* sent to its `inbox` have to follow the contract defined by the TM `actions`
* both, the defined `input` payload and the `output` payload is validated
@@ -461,7 +460,7 @@ The implementation supports validation or enforcing the following Ditto concepts
* Feature [properties](basic-feature.html#feature-properties):
* based on the linked TM of a Feature's [definition](basic-feature.html#feature-definition), the Feature `properties`
have to follow the contract defined by the TM `properties`
- * Feature [messages](basic-messages.html#sending-messages):
+ * Feature [messages](basic-messages.html#sending-and-receiving-messages):
* based on the linked TM of a Feature's [definition](basic-feature.html#feature-definition), the Feature `messages`
* sent to its `inbox` have to follow the contract defined by the TM `actions`
* both, the defined `input` payload and the `output` payload is validated
@@ -684,7 +683,7 @@ To use the Ditto WoT extension in your Thing Models or Thing Descriptions, add i
}
```
-### Ditto WoT Extension: category
+### The `category` term
The [category](https://ditto.eclipseprojects.io/wot/ditto-extension#category) is a term which can be added in scope of
WoT TM/TD [Property Affordances](https://www.w3.org/TR/wot-thing-description11/#propertyaffordance).
@@ -750,7 +749,7 @@ Based on the example above, a generated feature JSON would for example look like
}
```
-### Ditto WoT Extension: deprecationNotice
+### The `deprecationNotice` term
The [deprecationNotice](https://ditto.eclipseprojects.io/wot/ditto-extension#deprecationNotice) is a term which can be
added at the WoT TM/TD (Thing) level, or in scope of
@@ -1055,3 +1054,10 @@ wot-directory {
The example can be found on a [dedicated page](basic-wot-integration-example.html) as the JSONs included in the
example are quite long.
+
+## Further reading
+
+* [WoT Integration Example](basic-wot-integration-example.html) -- step-by-step walkthrough
+* [WoT Validation Configuration](basic-wot-validation-config.html) -- runtime validation API
+* [W3C WoT Thing Description 1.1](https://www.w3.org/TR/wot-thing-description11/) -- the specification
+* [Web of Things in a Nutshell](https://www.w3.org/WoT/documentation/) -- W3C introduction
diff --git a/documentation/src/main/resources/pages/ditto/basic-wot-validation-config.md b/documentation/src/main/resources/pages/ditto/basic-wot-validation-config.md
index d7eee5d0d77..97fce2f5041 100644
--- a/documentation/src/main/resources/pages/ditto/basic-wot-validation-config.md
+++ b/documentation/src/main/resources/pages/ditto/basic-wot-validation-config.md
@@ -1,42 +1,58 @@
---
-title: WoT Validation Config API
+title: WoT Validation Configuration
keywords: WoT, validation, config, API, DData, distribution, recovery, example
tags: [wot]
permalink: basic-wot-validation-config.html
---
-# WoT Validation Config API
+You manage WoT Thing Model validation rules at runtime through the Ditto devops API, enabling dynamic control over which validation checks apply to specific users, models, or deployment stages.
-This page documents the management of WoT (Web of Things) validation configuration in Eclipse Ditto, including API endpoints, dynamic configuration, distributed state, recovery, and practical examples.
+{% include callout.html content="**TL;DR**: Use `PUT /devops/wot/config` to set global validation rules, and `PUT /devops/wot/config/dynamicConfigs/{scopeId}` to override rules for specific users or models. This is especially useful during model migrations." type="primary" %}
-## Introduction
+## Overview
-Eclipse Ditto allows you to configure WoT Thing Model validation both statically (via config files or environment variables) and dynamically at runtime via HTTP API endpoints. This enables flexible, fine-grained control over validation behavior in distributed deployments.
+Eclipse Ditto supports both static WoT validation (via config files or environment variables) and dynamic runtime configuration via HTTP API endpoints. Dynamic configuration lets you:
-## Use case: Migrating to a new WoT model version
+* Temporarily relax validation during model migrations
+* Apply different validation rules per user, connection, or model
+* Adjust validation without restarting Ditto
-When you update your WoT Thing Model to a new version, some existing Things or updates may not immediately comply with the new, stricter validation rules. Ditto's dynamic validation config endpoint allows you to temporarily relax validation for specific Things. This means you can:
+## How it works
-- Allow updates that would otherwise be rejected as invalid under the new model, enabling a smooth transition period.
-- Gradually adapt your data or processes to the new model requirements without downtime or disruption.
-- Once migration is complete and all data is compliant, you can restore strict validation to enforce the new model for all future updates.
+### Use case: model migration
-This approach helps ensure a seamless migration to new WoT model versions, minimizing interruptions and validation errors during the transition.
+When you upgrade a WoT Thing Model to a new version, existing Things may not immediately comply with stricter validation rules. The dynamic validation API lets you:
-## API Endpoints
+1. Temporarily relax validation for affected Things
+2. Migrate data and processes to the new model
+3. Restore strict validation once migration is complete
-| Endpoint | Method | Purpose |
-|----------|--------|-------------------------------------------------------------------|
-| `/devops/wot/config` | GET | Get the current global WoT validation config |
-| `/devops/wot/config` | PUT | Upsert (create or update) the global WoT validation config |
-| `/devops/wot/config` | DELETE | Delete the global WoT validation config (revert to static config) |
-| `/devops/wot/config/merged` | GET | Get the merged (static + dynamic) config |
-| `/devops/wot/config/dynamicConfigs` | GET | List all dynamic config sections |
-| `/devops/wot/config/dynamicConfigs/{scopeId}` | GET | Get a specific dynamic config section |
-| `/devops/wot/config/dynamicConfigs/{scopeId}` | PUT | Upsert (create or update) a dynamic config section |
-| `/devops/wot/config/dynamicConfigs/{scopeId}` | DELETE | Delete a dynamic config section |
+### Configuration layers
-## Example: Enable WoT Validation Globally
+Ditto evaluates validation rules in layers:
+
+1. **Static config** -- defined in `things.conf` or environment variables
+2. **Global dynamic config** -- set via the `/devops/wot/config` API
+3. **Scoped dynamic overrides** -- set via `/devops/wot/config/dynamicConfigs/{scopeId}` for specific contexts
+
+The **merged config** (viewable at `/devops/wot/config/merged`) is what Ditto actually uses for validation.
+
+## API endpoints
+
+| Endpoint | Method | Purpose |
+|----------|--------|---------|
+| `/devops/wot/config` | GET | Get the current global WoT validation config |
+| `/devops/wot/config` | PUT | Create or update the global config |
+| `/devops/wot/config` | DELETE | Delete the global config (revert to static config) |
+| `/devops/wot/config/merged` | GET | Get the effective merged config |
+| `/devops/wot/config/dynamicConfigs` | GET | List all dynamic config sections |
+| `/devops/wot/config/dynamicConfigs/{scopeId}` | GET | Get a specific dynamic config section |
+| `/devops/wot/config/dynamicConfigs/{scopeId}` | PUT | Create or update a dynamic config section |
+| `/devops/wot/config/dynamicConfigs/{scopeId}` | DELETE | Delete a dynamic config section |
+
+## Examples
+
+### Enable global validation
```http
PUT /devops/wot/config
@@ -70,7 +86,9 @@ Content-Type: application/json
}
```
-## Example: Add a Dynamic Override for a Specific User
+### Add a dynamic override for a specific user
+
+This example disables validation and switches to warning-only mode for an admin user:
```http
PUT /devops/wot/config/dynamicConfigs/my-scope
@@ -92,33 +110,30 @@ Content-Type: application/json
}
```
-## How Dynamic Config Works
-
-- **Dynamic config sections** allow you to override the global validation config for specific API calls, based on:
- - Ditto headers (e.g., user or connection)
- - Thing or Feature model URLs
-- The **merged config** is what Ditto actually uses for validation, combining static, global, and dynamic settings.
+### How dynamic overrides match
-## API Schema
+Dynamic config sections match API calls based on three criteria (all must match for an override to apply):
-For details on the request/response structure, see the [Ditto HTTP API reference](/http-api-doc.html) or the Ditto API documentation.
+| Criteria | Description |
+|----------|-------------|
+| `ditto-headers-patterns` | Match Ditto headers (e.g., `ditto-originator` for user or connection identity). Multiple header blocks are OR-combined; fields within a block are AND-combined. |
+| `thing-definition-patterns` | Match the Thing's WoT Thing Model URL (OR-combined) |
+| `feature-definition-patterns` | Match the Feature's WoT Thing Model URL (OR-combined) |
## Technical details
-### Distributed State and Recovery
-
-In a clustered Ditto deployment, the WoT validation configuration is managed as distributed state using [Pekko Distributed Data (DData)](https://pekko.apache.org/docs/pekko/current/typed/distributed-data.html). This ensures that configuration changes are automatically replicated across all nodes in the cluster, providing consistency, high availability, and resilience.
-
-#### DData-based Distribution
+### Distributed state and recovery
-- **Replication:** Whenever the WoT validation config is created, updated, or deleted (including dynamic config sections), the change is published to the DData replicator. All Ditto nodes with the `wot-validation-config-aware` cluster role receive and apply the update.
-- **Consistency:** DData uses CRDTs (Conflict-free Replicated Data Types) to ensure eventual consistency across the cluster, even in the presence of network partitions or node failures.
-- **Immediate Effect:** API changes to the config (via the endpoints described above) are visible to all nodes almost immediately, ensuring that validation behavior is consistent regardless of which node handles a request.
+In a clustered Ditto deployment, the WoT validation configuration uses [Pekko Distributed Data (DData)](https://pekko.apache.org/docs/pekko/current/typed/distributed-data.html) for replication:
-#### Recovery and Database Synchronization
+* **Replication** -- configuration changes propagate to all nodes with the `wot-validation-config-aware` cluster role
+* **Consistency** -- DData uses CRDTs (Conflict-free Replicated Data Types) for eventual consistency
+* **Persistence** -- configuration is also stored in MongoDB via event sourcing and snapshots
+* **Recovery** -- on startup, each node recovers the latest configuration from the database and publishes it to DData
+* **Resilience** -- rolling restarts and failover do not result in configuration loss
-- **Persistence:** The WoT validation config is also persisted in the database (MongoDB) using event sourcing and snapshots. This ensures that the config survives full cluster restarts or catastrophic failures.
-- **Startup Recovery:** On startup, each node recovers the latest WoT validation config from the database. Once recovery is complete, the config is published to DData, ensuring that all nodes are synchronized with the most recent state.
-- **Rolling Restarts and Failover:** Because the config is both persisted and distributed, rolling restarts or failover scenarios do not result in loss of configuration. Any node joining the cluster will receive the current config from DData or recover it from the database if needed.
+## Further reading
-This architecture ensures that the WoT validation configuration is always up-to-date, consistent, and highly available, even in the face of failures, restarts, or network issues.
\ No newline at end of file
+* [WoT Overview](basic-wot-integration.html) -- WoT integration concepts and static configuration
+* [WoT Integration Example](basic-wot-integration-example.html) -- hands-on walkthrough
+* [Ditto HTTP API reference](http-api-doc.html) -- full API schema details
diff --git a/documentation/src/main/resources/pages/ditto/client-sdk-java.md b/documentation/src/main/resources/pages/ditto/client-sdk-java.md
index ddc90f6aa27..771cdf61f30 100644
--- a/documentation/src/main/resources/pages/ditto/client-sdk-java.md
+++ b/documentation/src/main/resources/pages/ditto/client-sdk-java.md
@@ -1,30 +1,30 @@
---
-title: Client SDK Java
-keywords:
+title: Java SDK
+keywords:
tags: [client_sdk]
permalink: client-sdk-java.html
---
-A client SDK for Java in order to interact with digital twins provided by an Eclipse Ditto backend.
+The Ditto Java client SDK lets you manage digital twins, receive change notifications, exchange messages, and search for Things -- all from Java applications.
-## Features
+{% include callout.html content="**TL;DR**: Add the `ditto-client` Maven dependency, create a `DittoClient` instance with WebSocket messaging, and use `client.twin()`, `client.live()`, and `client.policies()` to interact with your Ditto backend." type="primary" %}
-* Digital twin management: CRUD (create, read, update, delete) of Ditto [things](https://www.eclipse.dev/ditto/basic-thing.html)
-* [Change notifications](https://www.eclipse.dev/ditto/basic-changenotifications.html):
- consume notifications whenever a "watched" digital twin is modified
-* Send/receive [messages](https://www.eclipse.dev/ditto/basic-messages.html) to/from devices connected via a digital twin
-* Use the [live channel](https://www.eclipse.dev/ditto/protocol-twinlive.html#live) in order to react on commands directed
- to devices targeting their "live" state
+## Overview
-## Communication channel
+The Java SDK communicates with Ditto over [WebSocket](httpapi-protocol-bindings-websocket.html) using the [Ditto Protocol](protocol-overview.html). It provides a type-safe, asynchronous API for:
-The Ditto Java client interacts with an Eclipse Ditto backend via Ditto's
-[WebSocket](https://www.eclipse.dev/ditto/httpapi-protocol-bindings-websocket.html) sending and receiving messages
-in [Ditto Protocol](https://www.eclipse.dev/ditto/protocol-overview.html).
+* **Twin management** -- create, read, update, and delete [Things](basic-thing.html)
+* **Change notifications** -- receive events when digital twins are modified
+* **Messages** -- send and receive [messages](basic-messages.html) to/from devices
+* **Live channel** -- interact with devices via the [live channel](protocol-twinlive.html)
+* **Policy management** -- manage [Policies](basic-policy.html)
+* **Search** -- find Things using [RQL](basic-rql.html) expressions
-## Usage
+## Getting started
-Maven coordinates:
+### Installation
+
+Add the Maven dependency:
```xml
@@ -34,241 +34,273 @@ Maven coordinates:
```
-### Instantiate & configure a new Ditto client
-
-To configure your Ditto client instance, use the `org.eclipse.ditto.client.configuration` package in order to
-* create instances of `AuthenticationProvider` and `MessagingProvider`
-* create a `DisconnectedDittoClient` instance
-* obtain a `DittoClient` instance asynchronously by calling `.connect()`
+### Create a client instance
-For example:
+Configure authentication and messaging, then connect:
```java
-ProxyConfiguration proxyConfiguration =
- ProxyConfiguration.newBuilder()
- .proxyHost("localhost")
- .proxyPort(3128)
- .build();
-
AuthenticationProvider authenticationProvider =
- AuthenticationProviders.clientCredentials(ClientCredentialsAuthenticationConfiguration.newBuilder()
- .clientId("my-oauth-client-id")
- .clientSecret("my-oauth-client-secret")
- .scopes("offline_access email")
- .tokenEndpoint("https://my-oauth-provider/oauth/token")
- // optionally configure a proxy server
- .proxyConfiguration(proxyConfiguration)
- .build());
+ AuthenticationProviders.clientCredentials(
+ ClientCredentialsAuthenticationConfiguration.newBuilder()
+ .clientId("my-oauth-client-id")
+ .clientSecret("my-oauth-client-secret")
+ .scopes("offline_access email")
+ .tokenEndpoint("https://my-oauth-provider/oauth/token")
+ .build());
MessagingProvider messagingProvider =
- MessagingProviders.webSocket(WebSocketMessagingConfiguration.newBuilder()
- .endpoint("wss://ditto.eclipseprojects.io")
- // optionally configure a proxy server or a truststore containing the trusted CAs for SSL connection establishment
- .proxyConfiguration(proxyConfiguration)
- .trustStoreConfiguration(TrustStoreConfiguration.newBuilder()
- .location(TRUSTSTORE_LOCATION)
- .password(TRUSTSTORE_PASSWORD)
- .build())
- .build(), authenticationProvider);
-
-DisconnectedDittoClient disconnectedDittoClient = DittoClients.newInstance(messagingProvider);
+ MessagingProviders.webSocket(
+ WebSocketMessagingConfiguration.newBuilder()
+ .endpoint("wss://ditto.eclipseprojects.io")
+ .build(),
+ authenticationProvider);
+
+DisconnectedDittoClient disconnectedDittoClient =
+ DittoClients.newInstance(messagingProvider);
disconnectedDittoClient.connect()
.thenAccept(this::startUsingDittoClient)
.exceptionally(error -> disconnectedDittoClient.destroy());
+```
+You can optionally configure a proxy and TLS trust store:
+
+```java
+ProxyConfiguration proxyConfiguration =
+ ProxyConfiguration.newBuilder()
+ .proxyHost("localhost")
+ .proxyPort(3128)
+ .build();
+
+// Add to authentication configuration:
+// .proxyConfiguration(proxyConfiguration)
+
+// Add to messaging configuration:
+// .proxyConfiguration(proxyConfiguration)
+// .trustStoreConfiguration(TrustStoreConfiguration.newBuilder()
+// .location(TRUSTSTORE_LOCATION)
+// .password(TRUSTSTORE_PASSWORD)
+// .build())
```
-### Use the Ditto client
+## Examples
+
+### Manage twins
-#### Manage twins
+Create a Thing and set an attribute:
```java
-client.twin().create("org.eclipse.ditto:new-thing").handle((createdThing, throwable) -> {
- if (createdThing != null) {
- System.out.println("Created new thing: " + createdThing);
- } else {
- System.out.println("Thing could not be created due to: " + throwable.getMessage());
- }
- return client.twin().forId(thingId).putAttribute("first-updated-at", OffsetDateTime.now().toString());
-}).toCompletableFuture().get(); // this will block the thread! work asynchronously whenever possible!
+client.twin().create("org.eclipse.ditto:new-thing")
+ .handle((createdThing, throwable) -> {
+ if (createdThing != null) {
+ System.out.println("Created: " + createdThing);
+ } else {
+ System.out.println("Error: " + throwable.getMessage());
+ }
+ return client.twin().forId(thingId)
+ .putAttribute("first-updated-at",
+ OffsetDateTime.now().toString());
+ })
+ .toCompletableFuture()
+ .get(); // blocks -- work asynchronously when possible
```
-#### Subscribe for change notifications
+### Subscribe for change notifications
-In order to subscribe for [events](basic-signals-event.html) emitted by Ditto after a twin was modified, start the
-consumption on the `twin` channel:
+Listen for Thing modifications:
```java
-client.twin().startConsumption().toCompletableFuture().get(); // this will block the thread! work asynchronously whenever possible!
-System.out.println("Subscribed for Twin events");
-client.twin().registerForThingChanges("my-changes", change -> {
- if (change.getAction() == ChangeAction.CREATED) {
- System.out.println("An existing Thing was modified: " + change.getThing());
- // perform custom actions ..
- }
-});
+client.twin().startConsumption()
+ .toCompletableFuture().get();
+
+client.twin().registerForThingChanges("my-changes",
+ change -> {
+ if (change.getAction() == ChangeAction.CREATED) {
+ System.out.println("Thing modified: "
+ + change.getThing());
+ }
+ });
```
-There is also the possibility here to apply *server side filtering* of which events will get delivered to the client:
+Apply server-side filtering to receive only relevant events:
```java
client.twin().startConsumption(
- Options.Consumption.filter("gt(features/temperature/properties/value,23.0)")
-).toCompletableFuture().get(); // this will block the thread! work asynchronously whenever possible!
-System.out.println("Subscribed for Twin events");
-client.twin().registerForFeaturePropertyChanges("my-feature-changes", "temperature", "value", change -> {
- // perform custom actions ..
-});
+ Options.Consumption.filter(
+ "gt(features/temperature/properties/value,23.0)")
+).toCompletableFuture().get();
+
+client.twin().registerForFeaturePropertyChanges(
+ "my-feature-changes", "temperature", "value",
+ change -> { /* handle change */ });
```
-##### Subscribe to enriched change notifications
+#### Enriched change notifications
-In order to use [enrichment](basic-enrichment.html) in the Ditto Java client, the `startConsumption()` call can be
-enhanced with the additional extra fields:
+Request [extra fields](basic-enrichment.html) with each notification:
```java
client.twin().startConsumption(
- Options.Consumption.extraFields(JsonFieldSelector.newInstance("attributes/location"))
-).toCompletableFuture().get(); // this will block the thread! work asynchronously whenever possible!
-client.twin().registerForThingChanges("my-enriched-changes", change -> {
- Optional extra = change.getExtra();
- // perform custom actions, making use of the 'extra' data ..
-});
+ Options.Consumption.extraFields(
+ JsonFieldSelector.newInstance("attributes/location"))
+).toCompletableFuture().get();
+
+client.twin().registerForThingChanges("my-enriched-changes",
+ change -> {
+ Optional extra = change.getExtra();
+ // use extra data
+ });
```
-In combination with a `filter`, the extra fields may also be used as part of such a filter:
+Combine enrichment with a filter to use extra fields in the filter expression:
```java
client.twin().startConsumption(
- Options.Consumption.extraFields(JsonFieldSelector.newInstance("attributes/location")),
- Options.Consumption.filter("eq(attributes/location,\"kitchen\")")
-).toCompletableFuture().get(); // this will block the thread! work asynchronously whenever possible!
-// register the callbacks...
+ Options.Consumption.extraFields(
+ JsonFieldSelector.newInstance("attributes/location")),
+ Options.Consumption.filter("eq(attributes/location,\"kitchen\")")
+).toCompletableFuture().get();
```
+### Send and receive messages
-#### Send/receive messages
-
-Register for receiving messages with the subject `hello.world` on any thing:
+Register a handler for messages with subject `hello.world`:
```java
-client.live().startConsumption().toCompletableFuture().get(); // this will block the thread! work asynchronously whenever possible!
-System.out.println("Subscribed for live messages/commands/events");
-client.live().registerForMessage("globalMessageHandler", "hello.world", message -> {
- System.out.println("Received Message with subject " + message.getSubject());
- message.reply()
- .statusCode(HttpStatusCode.IM_A_TEAPOT)
- .payload("Hello, I'm just a Teapot!")
- .send();
-});
+client.live().startConsumption()
+ .toCompletableFuture().get();
+
+client.live().registerForMessage(
+ "globalMessageHandler", "hello.world",
+ message -> {
+ System.out.println("Received: " + message.getSubject());
+ message.reply()
+ .statusCode(HttpStatusCode.IM_A_TEAPOT)
+ .payload("Hello, I'm just a Teapot!")
+ .send();
+ });
```
-Send a message with the subject `hello.world` to the thing with ID `org.eclipse.ditto:new-thing`:
+Send a message to a specific Thing:
```java
client.live().forId("org.eclipse.ditto:new-thing")
- .message()
- .from()
- .subject("hello.world")
- .payload("I am a Teapot")
- .send(String.class, (response, throwable) ->
- System.out.println("Got response: " + response.getPayload().orElse(null))
- );
+ .message()
+ .from()
+ .subject("hello.world")
+ .payload("I am a Teapot")
+ .send(String.class, (response, throwable) ->
+ System.out.println("Response: "
+ + response.getPayload().orElse(null)));
```
-#### Manage policies
+### Manage policies
-Read a policy:
-```java
-Policy retrievedPolicy = client.policies().retrieve(PolicyId.of("org.eclipse.ditto:new-policy"))
- .toCompletableFuture().get(); // this will block the thread! work asynchronously whenever possible!
-```
+Read and create Policies:
-Create a policy:
```java
-Policy newPolicy = Policy.newBuilder(PolicyId.of("org.eclipse.ditto:new-policy"))
- .forLabel("DEFAULT")
- .setSubject(Subject.newInstance(SubjectIssuer.newInstance("nginx"), "ditto"))
- .setGrantedPermissions(PoliciesResourceType.policyResource("/"), "READ", "WRITE")
- .setGrantedPermissions(PoliciesResourceType.thingResource("/"), "READ", "WRITE")
- .build();
+// Read
+Policy policy = client.policies()
+ .retrieve(PolicyId.of("org.eclipse.ditto:new-policy"))
+ .toCompletableFuture().get();
+
+// Create
+Policy newPolicy = Policy.newBuilder(
+ PolicyId.of("org.eclipse.ditto:new-policy"))
+ .forLabel("DEFAULT")
+ .setSubject(Subject.newInstance(
+ SubjectIssuer.newInstance("nginx"), "ditto"))
+ .setGrantedPermissions(
+ PoliciesResourceType.policyResource("/"),
+ "READ", "WRITE")
+ .setGrantedPermissions(
+ PoliciesResourceType.thingResource("/"),
+ "READ", "WRITE")
+ .build();
client.policies().create(newPolicy)
- .toCompletableFuture().get(); // this will block the thread! work asynchronously whenever possible!
+ .toCompletableFuture().get();
```
-Updating and deleting policies is also possible via the Java client API, please follow the API and the JavaDoc.
+### Search for Things
-#### Search for things
+Use the Java Stream API:
-Search for things using the Java 8 `java.util.Stream` API:
```java
client.twin().search()
- .stream(queryBuilder -> queryBuilder.namespace("org.eclipse.ditto")
- .filter("eq(attributes/location,'kitchen')") // apply RQL expression here
- .options(builder -> builder.sort(s -> s.desc("thingId")).size(1))
- )
- .forEach(foundThing -> System.out.println("Found thing: " + foundThing));
+ .stream(queryBuilder -> queryBuilder
+ .namespace("org.eclipse.ditto")
+ .filter("eq(attributes/location,'kitchen')")
+ .options(builder -> builder
+ .sort(s -> s.desc("thingId"))
+ .size(1)))
+ .forEach(thing ->
+ System.out.println("Found: " + thing));
```
-Use an [RQL](basic-rql.html) query in order to filter for the searched things.
+Or use the reactive streams Publisher API:
-Search for things using the reactive streams `org.reactivestreams.Publisher` API:
```java
Publisher> publisher = client.twin().search()
- .publisher(queryBuilder -> queryBuilder.namespace("org.eclipse.ditto")
- .filter("eq(attributes/location,'kitchen')") // apply RQL expression here
- .options(builder -> builder.sort(s -> s.desc("thingId")).size(1))
- );
+ .publisher(queryBuilder -> queryBuilder
+ .namespace("org.eclipse.ditto")
+ .filter("eq(attributes/location,'kitchen')")
+ .options(builder -> builder
+ .sort(s -> s.desc("thingId"))
+ .size(1)));
+
// integrate the publisher in the reactive streams library of your choice, e.g. Pekko streams:
-org.apache.pekko.stream.javadsl.Source things = org.apache.pekko.stream.javadsl.Source.fromPublisher(publisher)
- .flatMapConcat(Source::from);
+org.apache.pekko.stream.javadsl.Source things =
+ org.apache.pekko.stream.javadsl.Source.fromPublisher(publisher)
+ .flatMapConcat(Source::from);
// .. proceed working with the Pekko Source ..
```
+### Request and issue acknowledgements
-#### Request and issue acknowledgements
-
-[Requesting acknowledgements](basic-acknowledgements.html#requesting-acks) is possible in the Ditto Java
-client in the following way:
+Request [acknowledgements](basic-acknowledgements.html) for a command:
```java
DittoHeaders dittoHeaders = DittoHeaders.newBuilder()
- .acknowledgementRequest(
- AcknowledgementRequest.of(DittoAcknowledgementLabel.PERSISTED),
- AcknowledgementRequest.of(AcknowledgementLabel.of("my-custom-ack"))
- )
- .timeout("5s")
- .build();
-
-client.twin().forId(ThingId.of("org.eclipse.ditto:my-thing"))
- .putAttribute("counter", 42, Options.dittoHeaders(dittoHeaders))
- .whenComplete((aVoid, throwable) -> {
- if (throwable instanceof AcknowledgementsFailedException) {
- Acknowledgements acknowledgements = ((AcknowledgementsFailedException) throwable).getAcknowledgements();
- System.out.println("Acknowledgements could not be fulfilled: " + acknowledgements);
- }
- });
+ .acknowledgementRequest(
+ AcknowledgementRequest.of(
+ DittoAcknowledgementLabel.PERSISTED),
+ AcknowledgementRequest.of(
+ AcknowledgementLabel.of("my-custom-ack")))
+ .timeout("5s")
+ .build();
+
+client.twin()
+ .forId(ThingId.of("org.eclipse.ditto:my-thing"))
+ .putAttribute("counter", 42,
+ Options.dittoHeaders(dittoHeaders))
+ .whenComplete((aVoid, throwable) -> {
+ if (throwable instanceof AcknowledgementsFailedException) {
+ Acknowledgements acks =
+ ((AcknowledgementsFailedException) throwable)
+ .getAcknowledgements();
+ System.out.println("Acks failed: " + acks);
+ }
+ });
```
-[Issuing requested acknowledgements](basic-acknowledgements.html#issuing-acknowledgements) can be done like this
-whenever a `Change` callback is invoked with a change notification:
+Issue a custom acknowledgement when handling a change:
```java
client.twin().registerForThingChanges("REG1", change -> {
- change.handleAcknowledgementRequest(AcknowledgementLabel.of("my-custom-ack"), ackHandle ->
- ackHandle.acknowledge(HttpStatusCode.NOT_FOUND, JsonObject.newBuilder()
- .set("error-detail", "Could not be found")
- .build()
- )
- );
+ change.handleAcknowledgementRequest(
+ AcknowledgementLabel.of("my-custom-ack"),
+ ackHandle -> ackHandle.acknowledge(
+ HttpStatusCode.NOT_FOUND,
+ JsonObject.newBuilder()
+ .set("error-detail", "Could not be found")
+ .build()));
});
```
+## Further reading
-## Further examples
-
-For further examples on how to use the Ditto client, please have a look at the class
-[DittoClientUsageExamples](https://github.com/eclipse-ditto/ditto-clients/blob/master/java/src/test/java/org/eclipse/ditto/client/DittoClientUsageExamples.java)
- which is configured to connect to the [Ditto sandbox](https://ditto.eclipseprojects.io).
+* [DittoClientUsageExamples](https://github.com/eclipse-ditto/ditto-clients/blob/master/java/src/test/java/org/eclipse/ditto/client/DittoClientUsageExamples.java) -- complete example code
+* [Ditto sandbox](https://ditto.eclipseprojects.io) -- test environment for the examples
+* [WebSocket binding](httpapi-protocol-bindings-websocket.html) -- underlying transport
+* [Ditto Protocol](protocol-overview.html) -- message format reference
diff --git a/documentation/src/main/resources/pages/ditto/client-sdk-javascript.md b/documentation/src/main/resources/pages/ditto/client-sdk-javascript.md
index be88bf1130c..d011c13e560 100644
--- a/documentation/src/main/resources/pages/ditto/client-sdk-javascript.md
+++ b/documentation/src/main/resources/pages/ditto/client-sdk-javascript.md
@@ -1,63 +1,70 @@
---
-title: Client SDK JavaScript
-keywords:
+title: JavaScript SDK
+keywords:
tags: [client_sdk]
permalink: client-sdk-javascript.html
---
-A TypeScript library to facilitate working the the REST-like HTTP API and web socket API of Eclipse Ditto.
+You use the Ditto JavaScript SDK to work with the Ditto HTTP API and WebSocket API from browser and Node.js environments.
-## How to use it
-Install `@eclipse-ditto/ditto-javascript-client-dom` for the DOM (browser) implementation,
-`@eclipse-ditto/ditto-javascript-client-node` for the NodeJS implementation, or `@eclipse/ditto-javascript-client-api-ditto` for
-the API and build your own client implementation.
+{% include callout.html content="**TL;DR**: Install `@eclipse-ditto/ditto-javascript-client-dom` for browsers or `@eclipse-ditto/ditto-javascript-client-node` for Node.js, then use the client to manage Things, subscribe to events, and send messages." type="primary" %}
-More information can be found in the descriptions of the subpackages:
-* [@eclipse-ditto/ditto-javascript-client-api](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/api/README.md)
-* [@eclipse-ditto/ditto-javascript-client-dom](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/dom/README.md)
-* [@eclipse-ditto/ditto-javascript-client-node](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/node/README.md)
+## Overview
+
+The JavaScript SDK provides separate packages for different environments:
+
+| Package | Environment | Description |
+|---------|-------------|-------------|
+| [`@eclipse-ditto/ditto-javascript-client-dom`](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/dom/README.md) | Browser (DOM) | Uses browser-native HTTP and WebSocket |
+| [`@eclipse-ditto/ditto-javascript-client-node`](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/node/README.md) | Node.js | Uses Node.js HTTP and WebSocket |
+| [`@eclipse-ditto/ditto-javascript-client-api`](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/api/README.md) | Any | API-only -- build your own client implementation |
All released versions are published on [npmjs.com](https://www.npmjs.com/~eclipse_ditto).
-## Compatibility with [Eclipse Ditto](https://github.com/eclipse-ditto/ditto)
+## Getting started
-The newest release of the JavaScript client will always try to cover as much API
-functionality of the same Eclipse Ditto major version as possible. There might
-however be missing features for which we would be very happy to accept contributions.
+### Installation
+Install the package for your environment:
-## Coding
+```bash
+# For browser applications:
+npm install @eclipse-ditto/ditto-javascript-client-dom
+
+# For Node.js applications:
+npm install @eclipse-ditto/ditto-javascript-client-node
```
+
+### Compatibility
+
+The JavaScript SDK tracks the same major version as Eclipse Ditto. The latest release covers as much API functionality as possible for the corresponding Ditto version.
+
+## Building from source
+
+```bash
npm install
npm run build
npm run lint
npm test
-# or npm run test:watch
```
-## Troubleshooting
-If you get strange errors, it would be best cleaning all dependencies and
-starting from the beginning again:
-```
+### Troubleshooting build issues
+
+If you encounter build errors, clean everything and start fresh:
+
+```bash
npm run clean
-# by hand delete node_modules in the root folder, or use a tool like rm, rimraf, etc.
+# Delete node_modules in the root folder
npm install
npm run build
-# ...
```
-It is important to know that during install and build some extra processes
-are triggered by e.g. lerna which will symlink the `api` dependency into
-the node_modules of `dom` and `node` packages.
-
-## Internals
-This project is using [lerna](https://github.com/lerna/lerna) to split up the
-client into different packages. This way we can have standalone codeable
-subprojects (`api`, `dom` and `node`) but still are able to control dependencies,
-build processes or release processes globally.
-
-Furthermore we use [rollup.js](https://rollupjs.org/) for providing multiple
-module types of the packages, e.g. the `api` will be published as IIFE,
-ES Module and CommonJS module.
-
-For automatically generating barrel files, [barrelsby](https://github.com/bencoveney/barrelsby)
-is used during the build process.
+
+The build process uses [lerna](https://github.com/lerna/lerna) for multi-package management and [rollup.js](https://rollupjs.org/) for generating multiple module formats (IIFE, ES Module, CommonJS). During install and build, lerna symlinks the `api` dependency into the `dom` and `node` packages.
+
+## Further reading
+
+* [@eclipse-ditto/ditto-javascript-client-dom README](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/dom/README.md) -- browser usage details
+* [@eclipse-ditto/ditto-javascript-client-node README](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/node/README.md) -- Node.js usage details
+* [@eclipse-ditto/ditto-javascript-client-api README](https://github.com/eclipse-ditto/ditto-clients/blob/master/javascript/lib/api/README.md) -- API package details
+* [HTTP API overview](httpapi-overview.html) -- the REST API the SDK wraps
+* [WebSocket binding](httpapi-protocol-bindings-websocket.html) -- the WebSocket transport
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-header-mapping.md b/documentation/src/main/resources/pages/ditto/connectivity-header-mapping.md
index f1aa276f5c9..a00908194eb 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-header-mapping.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-header-mapping.md
@@ -1,44 +1,93 @@
---
-title: Header mapping for connections
+title: Header Mapping
keywords: header, mapping, placeholder
tags: [connectivity]
permalink: connectivity-header-mapping.html
---
-When receiving messages from external systems or sending messages to external systems, the external headers of the
-messages can be mapped to and from Ditto protocol headers.
+Header mapping lets you translate between external message headers and Ditto protocol headers, enabling correlation IDs, content types, and custom metadata to flow through Ditto.
-That way the headers can be passed through Ditto, or defined Ditto protocol headers like for example `correlation-id`
-may be mapped to a header used for message correlation in the external system.
+{% include callout.html content="**TL;DR**: Define a `headerMapping` object on any source or target to map external headers to/from Ditto protocol headers using placeholders." type="primary" %}
-A header mapping can be defined individually for every source and target of a connection. For examples of a definition
-see [source header mapping](basic-connections.html#source-header-mapping)
-and [target header mapping](basic-connections.html#target-header-mapping).
+## Overview
+
+When Ditto receives messages from or sends messages to external systems, you can map headers between
+the external protocol and the Ditto protocol. This allows you to:
+
+* Pass headers through Ditto (for example, correlation IDs)
+* Map protocol-specific headers to Ditto headers
+* Set custom external headers on outgoing messages
+
+You define header mappings individually for each source and target. For configuration examples, see
+[source header mapping](basic-connections.html#source-header-mapping) and
+[target header mapping](basic-connections.html#target-header-mapping).
{% include note.html content="Do not map headers prefixed by 'ditto-'. Ditto uses them internally. Setting them in header mapping has no effect." %}
+## How it works
+
+A header mapping is a JSON object where:
+
+* **Keys** are the target header names
+* **Values** are strings that can include [placeholders](basic-placeholders.html#scope-connections)
+
+For **inbound** messages (sources), the mapping translates external headers to Ditto protocol headers.
+For **outbound** messages (targets), the mapping translates Ditto protocol headers to external headers.
+
+If a placeholder fails to resolve, that header is skipped. Other headers and the message itself
+are still processed.
+
+## Configuration
+
+### Source header mapping example
+
+```json
+{
+ "addresses": ["telemetry"],
+ "authorizationContext": ["ditto:inbound-auth-subject"],
+ "headerMapping": {
+ "correlation-id": "{%raw%}{{ header:message-id }}{%endraw%}",
+ "content-type": "{%raw%}{{ header:content-type }}{%endraw%}",
+ "device-id": "{%raw%}{{ header:device_id }}{%endraw%}"
+ }
+}
+```
+
+### Target header mapping example
+
+```json
+{
+ "address": "events/twin",
+ "topics": ["_/_/things/twin/events"],
+ "authorizationContext": ["ditto:outbound-auth-subject"],
+ "headerMapping": {
+ "message-id": "{%raw%}{{ header:correlation-id }}{%endraw%}",
+ "content-type": "{%raw%}{{ header:content-type }}{%endraw%}",
+ "subject": "{%raw%}{{ topic:subject }}{%endraw%}",
+ "reply-to": "all-replies"
+ }
+}
+```
+
## Supported placeholders
-The supported placeholders for header mapping are defined in the
+The supported placeholders for header mapping are defined in the
[Placeholders - Scope: Connections](basic-placeholders.html#scope-connections) section.
-If a placeholder fails to resolve for a header value, then that header is not set. Placeholder resolution failure
-does not prevent sending of the message or setting other headers with resolved values.
## Special header mapping keys
-In addition to general header mapping capabilities, Ditto recognizes several special header mapping keys that control connectivity behavior:
+Ditto recognizes several special header keys that control connectivity behavior.
### Response diversion headers
-These headers control [response diversion](connectivity-response-diversion.html) functionality:
+These headers control [response diversion](connectivity-response-diversion.html):
-| Header Key | Description | Example Values |
-|------------|-------------|----------------------------------------------------------------|
-| `divert-response-to-connection` | Target connection ID for response diversion | `"target-connection-id"`, `"target-connection-id"` |
-| `divert-expected-response-types` | Response types to divert (comma-separated) | `"response"`, `"error"`, `"nack"` |
-| `diverted-response-from-connection` | Source connection of diverted response (automatically set) | `"source-connection-id"` |
+| Header Key | Description |
+|------------|-------------|
+| `divert-response-to-connection` | Target connection ID for response diversion |
+| `divert-expected-response-types` | Response types to divert (comma-separated: `response`, `error`, `nack`) |
+| `diverted-response-from-connection` | Source connection of the diverted response (set automatically) |
-Example source configuration with response diversion:
```json
{
"headerMapping": {
@@ -51,9 +100,18 @@ Example source configuration with response diversion:
### Protocol-specific headers
-Different protocols support different sets of headers. Some protocols also have special header behavior:
+Different protocols support different header capabilities:
+
+* **AMQP 1.0** -- full application properties and message annotations
+* **MQTT 5** -- user-defined properties
+* **MQTT 3.1.1** -- no application headers (only special `mqtt.*` headers)
+* **HTTP** -- standard HTTP headers plus `http.query` and `http.path`
+* **Kafka** -- Kafka record headers
+
+{% include note.html content="Response diversion headers are processed by Ditto internally and are not sent as external protocol headers. They control internal routing behavior only." %}
+
+## Further reading
-{% include note.html content="Response diversion headers (`ditto-divert-*`)
-are processed by Ditto internally and are not sent as external protocol headers.
-They control internal routing behavior only."
-%}
+* [Connections overview](basic-connections.html) -- source and target header mapping configuration
+* [Placeholders](basic-placeholders.html) -- placeholder syntax and available placeholders
+* [Response diversion](connectivity-response-diversion.html) -- redirect responses to other connections
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-hmac-signing.md b/documentation/src/main/resources/pages/ditto/connectivity-hmac-signing.md
index ad11a5a2d1e..a6f96555c1b 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-hmac-signing.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-hmac-signing.md
@@ -1,22 +1,29 @@
---
-title: HMAC signing
+title: HMAC Signing
keywords: hmac, hmac-sha256, authorization, azure, aws, azure iot hub, azure service bus, azure monitor, aws version 4 signing, aws sns, signing
tags: [connectivity]
permalink: connectivity-hmac-signing.html
---
-## HMAC signing
+You use HMAC signing to authenticate Ditto at external services like AWS and Azure without transmitting credentials directly.
-Ditto provides an extensible framework for HMAC-based signing authentication processes for HTTP Push and
+{% include callout.html content="**TL;DR**: Set `credentials.type` to `\"hmac\"` and specify an `algorithm` (`aws4-hmac-sha256`, `az-monitor-2016-04-01`, or `az-sasl`) with the required parameters. Supported on HTTP Push and AMQP 1.0 connections." type="primary" %}
+
+## Overview
+
+Ditto provides an extensible framework for HMAC-based signing authentication for HTTP Push and
AMQP 1.0 connections. Three algorithms are available out of the box:
-- `aws4-hmac-sha256` (HTTP Push only): [Version 4 request signing](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html) for Amazon Web Services (AWS)
-- `az-monitor-2016-04-01` (HTTP Push only): [Version 2016-04-01 request signing](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#authorization) for Azure Monitor Data Collector
-- `az-sasl` (HTTP Push and AMQP 1.0): [Shared Access Signatures](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-dev-guide-sas?tabs=node) for Azure IoT Hub
-- `az-sasl` (HTTP Push only): [Shared Access Signatures](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-sas) for Azure Service Bus
+| Algorithm | Connection types | Use case |
+|-----------|-----------------|----------|
+| `aws4-hmac-sha256` | HTTP Push | [AWS Version 4 signing](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html) (SNS, S3, etc.) |
+| `az-monitor-2016-04-01` | HTTP Push | [Azure Monitor Data Collector](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#authorization) |
+| `az-sasl` | HTTP Push, AMQP 1.0 | [Azure IoT Hub](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-dev-guide-sas?tabs=node) and [Azure Service Bus](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-sas) Shared Access Signatures |
+
+## How it works
+Set the `credentials` field in your connection configuration:
-To use a request signing algorithm for authentication, set the `credentials` field of the connection as follows.
{%raw%}
```json
{
@@ -24,126 +31,56 @@ To use a request signing algorithm for authentication, set the `credentials` fie
"uri": "https://...:443",
"credentials": {
"type": "hmac",
- "algorithm": "", // e.g.: "az-monitor-2016-04-01"
+ "algorithm": "",
"parameters": {
- // parameters of the algorithm named above.
- ...
+ "...": "..."
}
- },
- ...
+ }
}
```
{%endraw%}
+## Configuration
-### Pre-defined algorithms
-
-#### aws4-hmac-sha256
-
-This algorithm works for AWS SNS and other services using [Version 4 request signing](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html).
-
-The parameters of the algorithm `aws4-hmac-sha256` are:
-- `region`: Region of the AWS endpoint.
-- `service`: Service name of the AWS endpoint.
-- `accessKey`: Access key of the signing user.
-- `secretKey`: Secret key of the signing user.
-- `doubleEncode`: Whether to double-encode and normalize path segments during request signing. Should be `false` for S3 and `true` for other services. Defaults to `true`.
-- `canonicalHeaders`: Array of names of headers to include in the signature. Default to `["host"]`.
-- `xAmzContentSha256`: Configuration for the header `x-amz-content-sha256`, which is mandatory for S3. Possible values are:
- - `EXCLUDED`: Do not send the header for non-S3 services. This is the default.
- - `INCLUDED`: Sign the payload hash as the value of the header for S3.
- - `UNSIGNED`: Omit the payload hash in the signature for S3.
-
-#### az-monitor-2016-04-01
-
-This algorithm works for [Version 2016-04-01 request signing](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#authorization)
-for Azure Monitor Data Collector.
-
-The parameters of the algorithm `az-monitor-2016-04-01` are:
-- `workspaceId`: ID of the Azure Monitor workspace.
-- `sharedKey`: Primary or secondary key of the Azure Monitor workspace.
-
-#### az-sasl
-
-This algorithm works for Azure IoT Hub (HTTP and AMQP) [Shared Access Signatures](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-dev-guide-sas?tabs=node)
-and Azure Service Bus (HTTP) [Shared Access Signatures](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-sas).
-
-The parameters of the algorithm `az-sasl` are:
-- `sharedKeyName`: Name of the used `sharedKey`.
-- `sharedKey`: Primary or secondary key of `sharedKeyName`. The key for Azure Service Bus needs an additional
- `Base64` encoding to work (e.g. a primary key `theKey` should be encoded to `dGhlS2V5` and used in this format).
-- `endpoint`: The endpoint which is used in the signature. For Azure IoT Hub this is expected
- to be the `resourceUri` without protocol (e.g. `myHub.azure-devices.net`, see the respective Azure documentation).
- For Azure Service Bus, this is expected to be the full URI of the resource to which access
- is claimed (e.g. `https://myNamespaces.servicebus.windows.net/myQueue`, see the respective Azure documentation)
-- `ttl` (optional): The time to live of a signature as a string in duration format. Allowed time units are "ms" (milliseconds),
- "s" (seconds), "m" (minutes) and "h" (hours), e.g. "10m" for ten minutes.
- `ttl` should only be set for AMQP connections and defines how long the connection signing is valid.
- The broker (e.g. Azure IoT Hub) will close the connection after `ttl` and Ditto will reconnect with a new signature.
- Defaults to 7 days.
-
-
-### Supported connection types
-
-| | HTTP Push connection | AMQP 1.0 connection |
-| ------------------------ | -------------------- | ------------------- |
-| `aws-hmac-sha256` | ✓ | |
-| `az-monitor-2016-04-01` | ✓ | |
-| `az-sasl` | ✓ | ✓ (for Azure IoT Hub) |
-
-### Configuration
-
-Algorithm names and implementations are configured in [`connectivity.conf`](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/resources/connectivity.conf).
-The default configuration provides the names and implementations of the available pre-defined algorithms for the given
-connection types.
-```hocon
-ditto.connectivity.connection {
- http-push.hmac-algorithms = {
-
- aws4-hmac-sha256 =
- "org.eclipse.ditto.connectivity.service.messaging.httppush.AwsRequestSigningFactory"
-
- az-monitor-2016-04-01 =
- "org.eclipse.ditto.connectivity.service.messaging.httppush.AzMonitorRequestSigningFactory"
+### aws4-hmac-sha256
- az-sasl =
- "org.eclipse.ditto.connectivity.service.messaging.signing.AzSaslSigningFactory"
+For AWS services using [Version 4 request signing](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html):
- // my-own-request-signing-algorithm =
- // "my.package.MyOwnImplementationOfHttpRequestSigningFactory"
- }
- amqp10.hmac-algorithms = {
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| `region` | AWS region | -- |
+| `service` | AWS service name | -- |
+| `accessKey` | IAM access key | -- |
+| `secretKey` | IAM secret key | -- |
+| `doubleEncode` | Double-encode path segments (`false` for S3, `true` for others) | `true` |
+| `canonicalHeaders` | Header names to include in the signature | `["host"]` |
+| `xAmzContentSha256` | S3 payload hash header: `EXCLUDED`, `INCLUDED`, or `UNSIGNED` | `EXCLUDED` |
- az-sasl =
- "org.eclipse.ditto.connectivity.service.messaging.signing.AzSaslSigningFactory"
+### az-monitor-2016-04-01
- // my-own-connection-signing-algorithm =
- // "my.package.MyOwnImplementationOfAmqpConnectionSigningFactory"
- }
-}
-```
+For [Azure Monitor Data Collector](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api#authorization):
-Users may add own request signing algorithms by implementing a defined interface and providing the fully
-qualified class name of the implementation in the config. The following table provides information where to update the configuration
-and which interface needs to be implemented.
+| Parameter | Description |
+|-----------|-------------|
+| `workspaceId` | Azure Monitor workspace ID |
+| `sharedKey` | Primary or secondary workspace key |
-| | HTTP Push connection | AMQP 1.0 connection |
-| ------------------------ | -------------------- | ------------------- |
-| Config path | `ditto.connectivity.connection.http-push.hmac-algorithms` | `ditto.connectivity.connection.amqp10.hmac-algorithms` |
-| Class to implement | [HttpRequestSigningFactory](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/httppush/HttpRequestSigningFactory.java) | [AmqpConnectionSigningFactory](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/amqp/AmqpConnectionSigningFactory.java) |
+### az-sasl
+For Azure IoT Hub and Azure Service Bus [Shared Access Signatures](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-dev-guide-sas?tabs=node):
-### Example integrations
+| Parameter | Description |
+|-----------|-------------|
+| `sharedKeyName` | Name of the shared access policy |
+| `sharedKey` | Primary or secondary key (Base64-encode for Service Bus) |
+| `endpoint` | Resource URI (for example, `myHub.azure-devices.net` or `https://myNs.servicebus.windows.net/myQueue`) |
+| `ttl` (optional) | Signature lifetime (for example, `10m`). For AMQP connections, the broker closes the connection after TTL and Ditto reconnects. Default: 7 days. |
-#### AWS SNS
+## Examples
-This example is an HTTP connection to [AWS SNS (Simple Notification Service)](https://aws.amazon.com/sns/) publishing
-twin events as messages to an SNS topic. Prerequisites are:
-- An IAM user with access to SNS,
-- An SNS topic,
-- A subscription on the SNS topic to receive twin events, e.g. by email.
+### AWS SNS
-The SNS connection is shown below.
+Publish twin events to an [AWS SNS](https://aws.amazon.com/sns/) topic:
{%raw%}
```json
@@ -165,71 +102,42 @@ The SNS connection is shown below.
},
"sources": [],
"targets": [{
- "address": "GET:/",
- "topics": [ "_/_/things/twin/events" ],
- "authorizationContext": [ "integration:ditto" ],
- "headerMapping": {},
- "payloadMapping": [ "javascript" ]
+ "address": "GET:/",
+ "topics": ["_/_/things/twin/events"],
+ "authorizationContext": ["integration:ditto"],
+ "payloadMapping": ["javascript"]
}],
- "clientCount": 1,
- "failoverEnabled": true,
- "validateCertificates": true,
- "processorPoolSize": 5,
- "specificConfig": {
- "parallelism": "1"
- },
+ "specificConfig": { "parallelism": "1" },
"mappingDefinitions": {
"javascript": {
"mappingEngine": "JavaScript",
"options": {
- "incomingScript": "function mapToDittoProtocolMsg() {\n return undefined;\n}",
- "outgoingScript": "function mapFromDittoProtocolMsg(namespace,name,group,channel,criterion,action,path,dittoHeaders,value,status,extra) {\n let textPayload = JSON.stringify(Ditto.buildDittoProtocolMsg(namespace, name, group, channel, criterion, action, path, dittoHeaders, value, status, extra));\n let query = 'Action=Publish&Message=' + encodeURIComponent(textPayload) + '&Subject=ThingModified&TopicArn=';\n let headers = {\"http.query\":query};\n return Ditto.buildExternalMsg(headers,'',null,'text/plain');\n}\n",
- "loadBytebufferJS": "false",
- "loadLongJS": "false"
+ "incomingScript": "function mapToDittoProtocolMsg() { return undefined; }",
+ "outgoingScript": "function mapFromDittoProtocolMsg(namespace,name,group,channel,criterion,action,path,dittoHeaders,value,status,extra) {\n let textPayload = JSON.stringify(Ditto.buildDittoProtocolMsg(namespace, name, group, channel, criterion, action, path, dittoHeaders, value, status, extra));\n let query = 'Action=Publish&Message=' + encodeURIComponent(textPayload) + '&Subject=ThingModified&TopicArn=';\n return Ditto.buildExternalMsg({\"http.query\":query},'',null,'text/plain');\n}"
}
}
- },
- "tags": []
+ }
}
```
{%endraw%}
-Parameters:
-- ``: The region of the SNS service.
-- `` The access key of the user authorized for SNS.
-- `` The secret key of the user authorized for SNS.
-- `` The ARN of the SNS topic. Note that it is a part of the URI and every `:` needs to be encoded as
-`%3A`.
-
-Here is the outgoing payload mapping in a readable format.
-Since the HTTP API of SNS requires GET requests with all necessary information in
-the query parameters, the payload mapper computes the query string and sets it via
-the [special header](connectivity-protocol-bindings-http.html#target-header-mapping) `http.query`.
-It is important to set payload to `null` so that the HMAC
-signature is computed from the SHA256 hash of the empty string, which SNS expects.
-```js
-function mapFromDittoProtocolMsg(namespace,name,group,channel,criterion,
- action,path,dittoHeaders,value,status,extra) {
+The outgoing mapper builds SNS query parameters and sets them via the `http.query` special header:
+
+```javascript
+function mapFromDittoProtocolMsg(namespace, name, group, channel, criterion,
+ action, path, dittoHeaders, value, status, extra) {
let textPayload = JSON.stringify(Ditto.buildDittoProtocolMsg(namespace, name,
group, channel, criterion, action, path, dittoHeaders, value, status, extra));
let query = 'Action=Publish&Message=' + encodeURIComponent(textPayload) +
'&Subject=ThingModified&TopicArn=';
- let headers = { "http.query": query };
- return Ditto.buildExternalMsg(headers, '', null, 'text/plain');
+ return Ditto.buildExternalMsg({ "http.query": query }, '', null, 'text/plain');
}
```
-#### AWS S3
+### AWS S3
-This example is an HTTP connection to [AWS S3 (Simple Storage Service)](https://aws.amazon.com/s3/) publishing
-twin events as objects in an S3 bucket. Prerequisites are:
-- An IAM user with access to S3,
-- An S3 bucket.
-
-The S3 connection is shown below. The credentials parameter `"xAmzContentSha256": "INCLUDED"` is necessary in order
-to include the payload hash as the value of the header `x-amz-content-sha256`. In addition,
-the parameter `"doubleEncode"` must be set to `false` because S3 performs URI-encoding only once when computing
-the signature, in contrast to other AWS services where the path segments of HTTP requests are encoded twice.
+Publish twin events as objects in an [AWS S3](https://aws.amazon.com/s3/) bucket. Note the S3-specific
+parameters `doubleEncode: false` and `xAmzContentSha256: "INCLUDED"`:
{%raw%}
```json
@@ -248,57 +156,36 @@ the signature, in contrast to other AWS services where the path segments of HTTP
"accessKey": "",
"secretKey": "",
"doubleEncode": false,
- "canonicalHeaders": [ "host", "x-amz-date" ],
+ "canonicalHeaders": ["host", "x-amz-date"],
"xAmzContentSha256": "INCLUDED"
}
},
"sources": [],
"targets": [{
- "address": "PUT:/",
- "topics": [ "_/_/things/twin/events" ],
- "authorizationContext": [ "integration:ditto" ],
- "headerMapping": {},
- "payloadMapping": [
- "javascript"
- ]
+ "address": "PUT:/",
+ "topics": ["_/_/things/twin/events"],
+ "authorizationContext": ["integration:ditto"],
+ "payloadMapping": ["javascript"]
}],
- "clientCount": 1,
- "failoverEnabled": true,
- "validateCertificates": true,
- "processorPoolSize": 5,
- "specificConfig": {
- "parallelism": "1"
- },
+ "specificConfig": { "parallelism": "1" },
"mappingDefinitions": {
"javascript": {
"mappingEngine": "JavaScript",
"options": {
- "incomingScript": "function mapToDittoProtocolMsg() {\n return undefined;\n}\n",
- "outgoingScript": "function mapFromDittoProtocolMsgWrapper(msg) {\n let topic = msg['topic'].split('/').join(':');\n let headers = {\n 'http.path': topic+':'+msg['revision']\n };\n let textPayload = JSON.stringify(msg);\n let bytePayload = null;\n let contentType = 'application/json';\n\n return Ditto.buildExternalMsg(\n headers,\n textPayload,\n bytePayload,\n contentType\n );\n}\n",
- "loadBytebufferJS": "false",
- "loadLongJS": "false"
+ "incomingScript": "function mapToDittoProtocolMsg() { return undefined; }",
+ "outgoingScript": "function mapFromDittoProtocolMsgWrapper(msg) {\n let topic = msg['topic'].split('/').join(':');\n let headers = { 'http.path': topic+':'+msg['revision'] };\n return Ditto.buildExternalMsg(headers, JSON.stringify(msg), null, 'application/json');\n}"
}
}
- },
- "tags": []
+ }
}
```
{%endraw%}
-Parameters:
-- ``: Name of the S3 bucket.
-- ``: The region of the S3 service.
-- `` The access key of the user authorized for S3.
-- `` The secret key of the user authorized for S3.
-
-In order to create a distinct object for each event,
-the payload mapper that computes the path string and sets it via
-the [special header](connectivity-protocol-bindings-http.html#target-header-mapping) `http.path`.
-The mapper overrides the function `mapFromDittoProtocolMsgWrapper` instead of the usual
-`mapFromDittoProtocolMsg` so that it has access to the revision number of twin events.
-The value of `http.path` is the name of the object, which consists of the topic of the event with `/` replaced by `:`
-followed by its revision. For example, a thing-modified event of revision 42 generates the object
-`::things:twin:events:modified:42`.
+The payload mapper creates a distinct S3 object per event by computing the path via the
+[special header](connectivity-protocol-bindings-http.html#target-header-mapping) `http.path`.
+It overrides `mapFromDittoProtocolMsgWrapper` instead of `mapFromDittoProtocolMsg` to access the revision number.
+The object name consists of the event topic with `/` replaced by `:` followed by its revision (for example,
+`::things:twin:events:modified:42`):
```js
function mapFromDittoProtocolMsgWrapper(msg) {
@@ -319,11 +206,9 @@ function mapFromDittoProtocolMsgWrapper(msg) {
}
```
-#### Azure Monitor Data Collector HTTP API
+### Azure Monitor Data Collector
-This example is an HTTP connection pushing twin events into
-[Azure Monitor Data Collector API](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api).
-It requires an Azure Log Analytics Workspace.
+Push twin events to [Azure Monitor](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api):
{%raw%}
```json
@@ -343,38 +228,23 @@ It requires an Azure Log Analytics Workspace.
},
"sources": [],
"targets": [{
- "address": "POST:/api/logs?api-version=2016-04-01",
- "topics": [ "_/_/things/twin/events" ],
- "authorizationContext": [ "integration:ditto" ],
- "headerMapping": {
- "Content-Type": "application/json",
- "Log-Type": "TwinEvent"
- }
+ "address": "POST:/api/logs?api-version=2016-04-01",
+ "topics": ["_/_/things/twin/events"],
+ "authorizationContext": ["integration:ditto"],
+ "headerMapping": {
+ "Content-Type": "application/json",
+ "Log-Type": "TwinEvent"
+ }
}],
- "clientCount": 1,
- "failoverEnabled": true,
- "validateCertificates": true,
- "processorPoolSize": 5,
- "specificConfig": {
- "parallelism": "1"
- },
- "tags": []
+ "specificConfig": { "parallelism": "1" }
}
```
{%endraw%}
-Parameters:
-- ``: The ID of the log analytics workspace.
-- ``: The primary or secondary shared key of the log analytics workspace.
+### Azure IoT Hub (HTTP)
-The connection publishes all fields of all twin events under the log type `TwinEvent`. After a maximum of 30 minutes
-after the first event, Azure Monitor should have created a custom log type `TwinEvent_CL` containing a twin event
-in each row.
-
-#### Azure IoT Hub HTTP API
-
-What follows is an example of a connection which forwards live messages sent to things as direct method calls to
-[Azure IoT Hub](https://docs.microsoft.com/en-us/azure/iot-hub/about-iot-hub) via HTTP. You'll receive the response of the direct method call.
+Forward live messages as direct method calls to
+[Azure IoT Hub](https://docs.microsoft.com/en-us/azure/iot-hub/about-iot-hub):
{%raw%}
```json
@@ -384,55 +254,40 @@ What follows is an example of a connection which forwards live messages sent to
"connectionType": "http-push",
"connectionStatus": "open",
"uri": "https://:443",
+ "credentials": {
+ "type": "hmac",
+ "algorithm": "az-sasl",
+ "parameters": {
+ "sharedKeyName": "",
+ "sharedKey": "",
+ "endpoint": ""
+ }
+ },
"sources": [],
"targets": [{
"address": "POST:/twins/{{ thing:id }}/methods?api-version=2018-06-30",
"topics": ["_/_/things/live/messages"],
"authorizationContext": ["integration:ditto"],
"issuedAcknowledgementLabel": "live-response",
- "headerMapping": {},
"payloadMapping": ["javascript"]
}],
- "clientCount": 1,
- "failoverEnabled": true,
- "validateCertificates": true,
- "processorPoolSize": 5,
- "specificConfig": {
- "parallelism": "1"
- },
+ "specificConfig": { "parallelism": "1" },
"mappingDefinitions": {
"javascript": {
"mappingEngine": "JavaScript",
"options": {
- "incomingScript": "function mapToDittoProtocolMsg() {\n return undefined;\n}\n",
- "outgoingScript": "function mapFromDittoProtocolMsg(\n namespace,\n name,\n group,\n channel,\n criterion,\n action,\n path,\n dittoHeaders,\n value,\n status,\n extra\n) {\n\n let headers = dittoHeaders;\n let payload = {\n \"methodName\": action,\n \"responseTimeoutInSeconds\": parseInt(dittoHeaders.timeout),\n \"payload\": value\n };\n let textPayload = JSON.stringify(payload);\n let bytePayload = null;\n let contentType = 'application/json';\n\n return Ditto.buildExternalMsg(\n headers, // The external headers Object containing header values\n textPayload, // The external mapped String\n bytePayload, // The external mapped byte[]\n contentType // The returned Content-Type\n );\n}\n",
- "loadBytebufferJS": "false",
- "loadLongJS": "false"
+ "incomingScript": "function mapToDittoProtocolMsg() { return undefined; }",
+ "outgoingScript": "function mapFromDittoProtocolMsg(namespace,name,group,channel,criterion,action,path,dittoHeaders,value,status,extra) {\n let payload = { \"methodName\": action, \"responseTimeoutInSeconds\": parseInt(dittoHeaders.timeout), \"payload\": value };\n return Ditto.buildExternalMsg(dittoHeaders, JSON.stringify(payload), null, 'application/json');\n}"
}
}
- },
- "credentials": {
- "type": "hmac",
- "algorithm": "az-sasl",
- "parameters": {
- "sharedKeyName": "",
- "sharedKey": "",
- "endpoint": ""
- }
- },
- "tags": []
+ }
}
```
{%endraw%}
-Parameters:
-* ``: The hostname of your iot hub instance. E.g. `my-hub.azure-devices.net`.
-* ``: The name of the (Azure IoT Hub) shared access policy, which has the `Service connect` permission. E.g. `service`.
-* ``: The primary or secondary key of above policy. E.g. `theKey`.
-
-The `outgoingScript` provides JavaScript payload mapping which maps the outgoing ditto protocol message into
-the required [direct method format](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-direct-methods#invoke-a-direct-method-from-a-back-end-app).
-I.e. it uses the live message subject as `methodName`, the timeout as `responseTimeoutInSeconds` and the value as `payload`:
+The outgoing JavaScript mapper transforms the Ditto Protocol message into the
+[direct method format](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-direct-methods#invoke-a-direct-method-from-a-back-end-app),
+using the live message subject as `methodName`, the timeout as `responseTimeoutInSeconds`, and the value as `payload`:
```javascript
function mapFromDittoProtocolMsg(
@@ -468,7 +323,8 @@ function mapFromDittoProtocolMsg(
}
```
-This is required since Azure IoT Hub direct methods require a specific format:
+Azure IoT Hub direct methods require this JSON format:
+
```json
{
"methodName": "reboot",
@@ -480,12 +336,10 @@ This is required since Azure IoT Hub direct methods require a specific format:
}
```
-Send live messages to a Thing, which contain as value the expected `payload`, as live message subject the expected
-`methodName` and as timeout the expected `responseTimeoutInSeconds`. Then they can successfully invoke
-the direct method on their respective devices.
+Send live messages to a Thing containing the expected `payload` as value, the expected `methodName` as live message
+subject, and the expected `responseTimeoutInSeconds` as timeout. For example, invoke the above direct method with a
+`POST` to `/api/2/things//inbox/messages/reboot?timeout=200s` with body:
-E.g. a live message for the above direct method would be a `POST` to
-`/api/2/things//inbox/messages/reboot?timeout=200s` with body:
```json
{
"input1": "someInput",
@@ -493,10 +347,9 @@ E.g. a live message for the above direct method would be a `POST` to
}
```
-#### Azure IoT Hub AMQP API
+### Azure IoT Hub (AMQP)
-What follows is an example of a connections which forwards live messages sent to things as Cloud To Device (C2D)
-messages to [Azure IoT Hub](https://docs.microsoft.com/en-us/azure/iot-hub/about-iot-hub) via AMQP 1.0.
+Forward live messages as Cloud-to-Device messages via AMQP 1.0:
{%raw%}
```json
@@ -506,6 +359,15 @@ messages to [Azure IoT Hub](https://docs.microsoft.com/en-us/azure/iot-hub/about
"connectionType": "amqp-10",
"connectionStatus": "open",
"uri": "amqps://:5671",
+ "credentials": {
+ "type": "hmac",
+ "algorithm": "az-sasl",
+ "parameters": {
+ "sharedKeyName": "",
+ "sharedKey": "",
+ "endpoint": ""
+ }
+ },
"sources": [],
"targets": [{
"address": "/messages/devicebound",
@@ -515,36 +377,14 @@ messages to [Azure IoT Hub](https://docs.microsoft.com/en-us/azure/iot-hub/about
"iothub-ack": "full",
"to": "/devices/{{thing:id}}/messages/devicebound"
}
- }
- ],
- "clientCount": 1,
- "failoverEnabled": true,
- "validateCertificates": true,
- "processorPoolSize": 5,
- "credentials": {
- "type": "hmac",
- "algorithm": "az-sasl",
- "parameters": {
- "sharedKeyName": "",
- "sharedKey": "",
- "endpoint": ""
- }
- },
- "tags": []
+ }]
}
```
{%endraw%}
-Parameters:
-* ``: The hostname of your iot hub instance. E.g. `my-hub.azure-devices.net`.
-* ``: The name of the (Azure IoT Hub) shared access policy, which has the `Service connect` permission. E.g. `service`.
-* ``: The primary or secondary key of above policy. E.g. `theKey`.
+### Azure Service Bus (HTTP)
-
-#### Azure Service Bus HTTP API
-
-What follows is an example of a connection which forwards live messages sent to things to a queue in
-[Azure Service Bus](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview).
+Forward live messages to an [Azure Service Bus](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview) queue:
{%raw%}
```json
@@ -554,40 +394,71 @@ What follows is an example of a connection which forwards live messages sent to
"connectionType": "http-push",
"connectionStatus": "open",
"uri": "https://:443",
- "sources": [],
- "targets": [{
- "address": "POST://messages",
- "topics": ["_/_/things/live/messages"],
- "authorizationContext": ["integration:ditto"],
- "issuedAcknowledgementLabel": "live-response",
- "headerMapping": {}
- }
- ],
- "clientCount": 1,
- "failoverEnabled": true,
- "validateCertificates": true,
- "processorPoolSize": 5,
- "specificConfig": {
- "parallelism": "1"
- },
"credentials": {
"type": "hmac",
"algorithm": "az-sasl",
"parameters": {
- "sharedKeyName": "",
- "sharedKey": "",
+ "sharedKeyName": "",
+ "sharedKey": "",
"endpoint": "https:///"
}
},
- "tags": []
+ "sources": [],
+ "targets": [{
+ "address": "POST://messages",
+ "topics": ["_/_/things/live/messages"],
+ "authorizationContext": ["integration:ditto"],
+ "issuedAcknowledgementLabel": "live-response"
+ }],
+ "specificConfig": { "parallelism": "1" }
}
```
{%endraw%}
-Parameters:
-* ``: The hostname of your service bus instance. E.g. `my-bus.servicebus.windows.net`.
-* ``: The queue name to which you want to publish in your service bus instance. E.g. `my-queue`'.
-* ``: The name of the (Azure Service Bus) shared access policy, which has `Send` and `Listen` permissions. E.g. `RootManageSharedAccessKey`.
-* ``: The `Base64` encoded primary or secondary key of above policy. The signing work if you
- encode the key with `Base64` (although it already has `Base64` encoding). E.g. if the primary key is `theKey`, you need to use
- its encoded version `dGhlS2V5`.
+For Azure Service Bus, the `sharedKey` must be additionally Base64-encoded (for example, `theKey`
+becomes `dGhlS2V5`).
+
+## Custom algorithms
+
+You can add custom signing algorithms by implementing the appropriate interface and registering it
+in [connectivity.conf](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/resources/connectivity.conf):
+
+| Connection type | Config path | Interface |
+|----------------|-------------|-----------|
+| HTTP Push | `ditto.connectivity.connection.http-push.hmac-algorithms` | [HttpRequestSigningFactory](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/httppush/HttpRequestSigningFactory.java) |
+| AMQP 1.0 | `ditto.connectivity.connection.amqp10.hmac-algorithms` | [AmqpConnectionSigningFactory](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/amqp/AmqpConnectionSigningFactory.java) |
+
+The default configuration registers the pre-defined algorithms as follows:
+
+```hocon
+ditto.connectivity.connection {
+ http-push.hmac-algorithms = {
+
+ aws4-hmac-sha256 =
+ "org.eclipse.ditto.connectivity.service.messaging.httppush.AwsRequestSigningFactory"
+
+ az-monitor-2016-04-01 =
+ "org.eclipse.ditto.connectivity.service.messaging.httppush.AzMonitorRequestSigningFactory"
+
+ az-sasl =
+ "org.eclipse.ditto.connectivity.service.messaging.signing.AzSaslSigningFactory"
+
+ // my-own-request-signing-algorithm =
+ // "my.package.MyOwnImplementationOfHttpRequestSigningFactory"
+ }
+ amqp10.hmac-algorithms = {
+
+ az-sasl =
+ "org.eclipse.ditto.connectivity.service.messaging.signing.AzSaslSigningFactory"
+
+ // my-own-connection-signing-algorithm =
+ // "my.package.MyOwnImplementationOfAmqpConnectionSigningFactory"
+ }
+}
+```
+
+## Further reading
+
+* [HTTP 1.1 binding](connectivity-protocol-bindings-http.html) -- HTTP push connection details
+* [AMQP 1.0 binding](connectivity-protocol-bindings-amqp10.html) -- AMQP 1.0 connection details
+* [Connections overview](basic-connections.html) -- connection model and configuration
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-manage-connections-piggyback.md b/documentation/src/main/resources/pages/ditto/connectivity-manage-connections-piggyback.md
index 117f0cd7a74..310f6f8c616 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-manage-connections-piggyback.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-manage-connections-piggyback.md
@@ -1,17 +1,16 @@
---
title: Manage connections via Piggyback commands
-keywords:
+keywords: connectivity, connections, piggyback, manage
tags: [connectivity]
permalink: connectivity-manage-connections-piggyback.html
---
-The recommended way of connection management (CRUD) is by using the [HTTP API to manage connections](connectivity-manage-connections.html).
-Although not the recommended way, it's still possible to manage connections in Ditto
-via DevOps [Piggyback commands](installation-operating.html#piggyback-commands) as well.
+You manage connections primarily through the [HTTP API](connectivity-manage-connections.html).
+Although not recommended, you can also manage connections via DevOps [Piggyback commands](installation-operating.html#piggyback-commands).
All connection related piggyback commands use the following HTTP endpoint:
-```
+```text
POST /devops/piggyback/connectivity
```
@@ -270,7 +269,7 @@ successfully established. The only parameter necessary for retrieving the connec
### Retrieve connection metrics
For details about the response of this command, please refer to
-[Retrieve connection logs using HTTP API](connectivity-manage-connections.html#retrieve-connection-metrics).
+[Retrieve connection logs using HTTP API](connectivity-manage-connections.html#connection-metrics).
```json
{
@@ -310,7 +309,7 @@ retrieving the connection metrics is the `connectionId`.
### Enable connection logs
For details about the this command, please refer to
-[Retrieve connection logs using HTTP API](connectivity-manage-connections.html#enable-connection-logs).
+[Retrieve connection logs using HTTP API](connectivity-manage-connections.html#connection-logs).
```json
{
@@ -330,7 +329,7 @@ For details about the this command, please refer to
### Retrieve connection logs
For details about the response of this command, please refer to
-[Retrieve connection logs using HTTP API](connectivity-manage-connections.html#retrieve-connection-logs).
+[Retrieve connection logs using HTTP API](connectivity-manage-connections.html#connection-logs).
```json
{
@@ -566,7 +565,7 @@ and progress will be saved to allow resuming later.
## Publishing connection logs
-Please refer to [Payload mapping configuration](connectivity-manage-connections.html#publishing-connection-logs) in
+Please refer to [Payload mapping configuration](connectivity-manage-connections.html#publishing-connection-logs-to-fluentd) in
HTTP API section about managing connections.
## Payload mapping configuration
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-manage-connections.md b/documentation/src/main/resources/pages/ditto/connectivity-manage-connections.md
index da222f593cb..1cad52bdec2 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-manage-connections.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-manage-connections.md
@@ -1,272 +1,368 @@
---
-title: Manage connections via HTTP API
+title: Managing Connections
keywords:
tags: [connectivity]
permalink: connectivity-manage-connections.html
---
-In order to manage (CRUD) connections in Ditto, the [HTTP API](http-api-doc.html#/Connections)
-is recommended to be used instead of the old fashion [Manage connections via Piggyback commands](connectivity-manage-connections-piggyback.html).
-As this is not a task for a developer, but more for a "DevOps engineer", the endpoint for creating new connections
-to external systems is authenticated using the ["devops"](installation-operating.html#devops-user) user.
+You create, modify, retrieve, and delete connections using the Ditto HTTP API or DevOps piggyback commands.
+
+{% include callout.html content="**TL;DR**: Use the HTTP API at `/api/2/connections` to manage connections. Authenticate as the `devops` user. You can also test connections with a dry run before persisting them." type="primary" %}
+
+## Overview
+
+Ditto provides two approaches to manage connections:
+
+1. **HTTP API** (recommended) -- RESTful endpoints under `/api/2/connections`
+2. **Piggyback commands** (legacy) -- DevOps commands sent to the connectivity service
+
+Both approaches require [devops authentication](operating-devops.html#devops-user).
## Authorization
-When creating new connections, an `authorizationContext` is needed providing the _authorization subjects_ (e.g. IDs of
-authorized users) to use in order to determine the permissions for when the connection executes commands (e.g. modifying
-a Thing). If you want to use a user for the basic auth (from the [HTTP API](connectivity-protocol-bindings-http.html))
-use the prefix `nginx:`, e.g. `nginx:ditto`.
-See [Basic Authentication](basic-auth.html#authorization-context-in-devops-commands) for more information.
+When you create a connection, you must specify an `authorizationContext` with authorization subjects
+(for example, user IDs). These subjects determine the permissions applied when the connection
+executes commands such as modifying a thing.
-## Encryption sensitive configuration data
-There is an option to enable encryption of credentials or other sensitive fields of the connection entities before they are
-persisted in the database on global service configuration level.
-For more details refer to [Connections sensitive data encryption](installation-operating.html#encrypt-sensitive-data-in-connections)
+If you use basic auth from the [HTTP API](connectivity-protocol-bindings-http.html), prefix the
+subject with `nginx:` (for example, `nginx:ditto`).
+See [Basic Authentication](basic-auth.html#authorization-context-in-devops-commands) for details.
-## CRUD endpoints
+## Encrypting sensitive data
-The following commands are available in order to manage connections:
+You can enable encryption of credentials and other sensitive fields before they are persisted.
+See [Connections sensitive data encryption](operating-configuration.html#encrypting-sensitive-connection-data)
+for configuration details.
-* [create](#create-connection)
-* [modify](#modify-connection)
-* [retrieve](#retrieve-connection)
-* [delete](#delete-connection)
+## Managing connections via HTTP API
-### Create connection
+### Create a connection
-Create a new connection by sending an HTTP `POST` request:
+Send an HTTP `POST` request:
-`POST /api/2/connections`
+```
+POST /api/2/connections
+```
```json
{
- "name": "",
- "connectionType": "",
- "connectionStatus": "",
- "uri": "",
- "sources": [],
- "targets": []
+ "name": "",
+ "connectionType": "",
+ "connectionStatus": "",
+ "uri": "",
+ "sources": [],
+ "targets": []
}
```
-The content of the connection configuration object is specified in the [Connections section](basic-connections.html).
-For protocol specific examples, consult the specific connection type binding respectively:
-* [AMQP-0.9.1 binding](connectivity-protocol-bindings-amqp091.html),
-* [AMQP-1.0 binding](connectivity-protocol-bindings-amqp10.html),
-* [MQTT-3.1.1 binding](connectivity-protocol-bindings-mqtt.html),
-* [MQTT-5 binding](connectivity-protocol-bindings-mqtt5.html),
-* [HTTP 1.1 binding](connectivity-protocol-bindings-http.html),
-* [Apache Kafka 2.x binding](connectivity-protocol-bindings-kafka2.html)
+The connection configuration format is described in the [Connections](basic-connections.html) section.
+For protocol-specific examples, see the relevant binding page:
+* [AMQP 0.9.1](connectivity-protocol-bindings-amqp091.html)
+* [AMQP 1.0](connectivity-protocol-bindings-amqp10.html)
+* [MQTT 3.1.1](connectivity-protocol-bindings-mqtt.html)
+* [MQTT 5](connectivity-protocol-bindings-mqtt5.html)
+* [HTTP 1.1](connectivity-protocol-bindings-http.html)
+* [Kafka 2.x](connectivity-protocol-bindings-kafka2.html)
-Additionally, you can test a connection before creating it:
+#### Dry run
-`POST /api/2/connections&dry-run=true`
+Test a connection before persisting it:
-Passing the `dry-run` query parameter checks the configuration and establishes a connection to the remote endpoint in order to validate the connection
-credentials. The connection is closed afterwards and will not be persisted.
+```
+POST /api/2/connections?dry-run=true
+```
-### Modify connection
+This validates the configuration and establishes a temporary connection to verify credentials.
+The connection is closed afterward and not persisted.
-Modify an existing connection by sending a HTTP `PUT` request to:
+### Modify a connection
-`PUT /api/2/connections/{connectionId}`
+Send an HTTP `PUT` request:
-```json
-{
- "connectionStatus": "",
- "...": ""
-}
+```
+PUT /api/2/connections/{connectionId}
```
-The connection with the specified ID needs to be created before one can modify it.
-
-### Retrieve connection
+The connection must already exist before you can modify it.
-The only parameter necessary for retrieving a connection is the `connectionId`:
+### Retrieve a connection
-`GET /api/2/connections/{connectionId}`
+```
+GET /api/2/connections/{connectionId}
+```
### Retrieve all connections
-Retrieves all created connections:
-`GET /api/2/connections`
-
-Additionally, you can get the connections ids only by providing the `ids-only=true` query parameter.
-
-`GET /api/2/connections?ids-only=true`
+```
+GET /api/2/connections
+```
-### Delete connection
+To retrieve only connection IDs:
-The only parameter necessary for deleting a connection is the `connectionId`.
+```
+GET /api/2/connections?ids-only=true
+```
-`DELETE /api/2/connections/{connectionId}`
+### Delete a connection
-## Helper endpoints
+```
+DELETE /api/2/connections/{connectionId}
+```
-### Retrieve connection status
+## Connection commands
-Returns the connection status by showing if a connection is currently enabled/disabled and if it is
-successfully established. The only parameter necessary for retrieving the connection status is the `connectionId`.
+Send commands to a connection via:
-`GET /api/2/connections/{connectionId}/status`
+```
+POST /api/2/connections/{connectionId}/command
+```
-### Retrieve connection metrics
+Send the command as `text/plain` payload:
-This command returns the connection metrics showing how many messages have been successfully or failingly `consumed`,
-`filtered`, `mapped`, `published`, `dropped`. The metrics are collected and returned in different time intervals:
-* the last minute
-* the last hour
-* the last 24 hours
+| Command | Description |
+|---------|-------------|
+| `connectivity.commands:openConnection` | Opens the connection. Publishes a [ConnectionOpenedAnnouncement](protocol-specification-connections-announcement.html). |
+| `connectivity.commands:closeConnection` | Closes the connection gracefully. Publishes a [ConnectionClosedAnnouncement](protocol-specification-connections-announcement.html). |
+| `connectivity.commands:resetConnectionMetrics` | Resets all metrics to zero. |
+| `connectivity.commands:enableConnectionLogs` | Enables logging for 24 hours. |
+| `connectivity.commands:resetConnectionLogs` | Clears all stored connection logs. |
-The only parameter necessary for retrieving the connection metrics is the `connectionId`.
+## Monitoring connections
-`GET /api/2/connections/{connectionId}/metrics`
+### Connection status
-### Retrieve connection logs
+```
+GET /api/2/connections/{connectionId}/status
+```
-Returns the connection logs. The only parameter necessary for retrieving the connection logs is the `connectionId`.
+Shows whether a connection is enabled/disabled and whether it is successfully established.
-This request will return a list of success and failure log entries containing information on messages processed by the
-connection. The logs have a maximum amount of entries that they can hold. If the connection produces more log entries,
-the older entries will be dropped. So keep in mind that you might miss some log entries.
+### Connection metrics
-The response will also provide information on how long the logging feature will still be enabled. Since the timer will
-always be reset when retrieving the logs, the timestamp will always be 24 hours from now.
+```
+GET /api/2/connections/{connectionId}/metrics
+```
-The default duration and the maximum amount of logs stored for one connection can be configured in Ditto's connectivity
-service configuration.
+Returns counts of messages `consumed`, `filtered`, `mapped`, `published`, and `dropped` across
+three time intervals: last minute, last hour, and last 24 hours.
-`GET /api/2/connections/{connectionId}/logs`
+### Connection logs
-### Connection commands
+```
+GET /api/2/connections/{connectionId}/logs
+```
-The only parameter necessary for sending a command to a connection is the `connectionId`.
+Returns success and failure log entries for messages processed by the connection. Logs have a
+maximum capacity; older entries are dropped when the limit is reached.
-`POST /api/2/connections/{connectionId}/command`
+{% include note.html content="When creating or opening a connection the logging is enabled per default. This allows
+to log possible errors on connection establishing." %}
-Supported commands sent as payload `text/plain`:
+## Publishing connection logs to Fluentd
-#### Open connection
+In addition to in-memory logs, you can publish connection logs to a
+[Fluentd](https://www.fluentd.org) or [Fluent Bit](https://fluentbit.io) endpoint.
-When opening a connection a [ConnectionOpenedAnnouncement](protocol-specification-connections-announcement.html) will be published.
+Configure this via environment variables in the [connectivity](architecture-services-connectivity.html) service:
-```
-connectivity.commands:openConnection
-```
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `CONNECTIVITY_LOGGER_PUBLISHER_ENABLED` | Enable log publishing | `false` |
+| `CONNECTIVITY_LOGGER_PUBLISHER_LOG_TAG` | Log tag (uses `connection:` if unset) | -- |
+| `CONNECTIVITY_LOGGER_PUBLISHER_FLUENCY_HOST` | Fluentd/Fluent Bit hostname | `localhost` |
+| `CONNECTIVITY_LOGGER_PUBLISHER_FLUENCY_PORT` | Fluentd/Fluent Bit port | `24224` |
-#### Close connection
+Each log entry contains:
-When gracefully closing a connection a [ConnectionClosedAnnouncement](protocol-specification-connections-announcement.html) will be published.
+| Field | Values |
+|-------|--------|
+| `level` | `success`, `failure` |
+| `category` | `connection`, `source`, `target`, `response` |
+| `type` | `consumed`, `dispatched`, `filtered`, `mapped`, `dropped`, `enforced`, `published`, `acknowledged` |
+| `connectionId` | ID of the connection which produced the log entry |
+| `correlationId` | Correlation ID (if available) |
+| `address` | Source/target address (if available) |
+| `entityId` | Thing or policy ID |
+| `entityType` | `"thing"` or `"policy"` |
+| `instanceId` | ID of the connectivity instance (helpful when `clientCount > 1`) |
+| `message` | The actual log message |
-```
-connectivity.commands:closeConnection
-```
+See [connectivity.conf](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/resources/connectivity.conf)
+at path `ditto.connectivity.monitoring.logger.publisher` for all options.
-#### Reset connection metrics
+## Payload mapping configuration
-This command resets the connection metrics - all metrics are set to `0` again.
+To use a custom [payload mapping](connectivity-mapping.html) for a source or target, define a
+`mappingDefinitions` entry in the connection and reference it by ID:
-```
-connectivity.commands:resetConnectionMetrics
+```json
+{
+ "mappingDefinitions": {
+ "customJs": {
+ "mappingEngine": "JavaScript",
+ "options": {
+ "incomingScript": "..",
+ "outgoingScript": ".."
+ }
+ }
+ },
+ "sources": [{
+ "addresses": "source",
+ "payloadMapping": ["customJs"]
+ }]
+}
```
-#### Enable connection logs
+If no mapping is specified, the [Ditto mapper](connectivity-mapping.html#ditto-mapper) is used by default.
-Enables the connection logging feature of a connection for 24 hours. As soon as connection logging is enabled, you will
-be able to [retrieve connection logs](#retrieve-connection-logs). The logs will contain a fixed amount of success and
-failure logs for each source and target of your connection and correlate with the metrics of the connection. This will
-allow you more insight in what goes well, and more importantly, what goes wrong.
+## Managing connections via piggyback commands
-The default duration and the maximum amount of logs stored for one connection can be configured in Ditto's connectivity
-service configuration.
+Although the HTTP API is recommended, you can still manage connections via DevOps
+[piggyback commands](operating-devops.html#piggyback-commands).
-{% include note.html content="When creating or opening a connection the logging is enabled per default. This allows
-to log possible errors on connection establishing." %}
+All piggyback commands use this endpoint:
```
-connectivity.commands:enableConnectionLogs
+POST /devops/piggyback/connectivity
```
-#### Reset connection logs
+### Create connection (piggyback)
-Clears all currently stored connection logs.
+```json
+{
+ "targetActorSelection": "/system/sharding/connection",
+ "headers": {
+ "aggregate": false,
+ "is-group-topic": false
+ },
+ "piggybackCommand": {
+ "type": "connectivity.commands:createConnection",
+ "connection": {}
+ }
+}
+```
+### Modify connection (piggyback)
+
+```json
+{
+ "targetActorSelection": "/system/sharding/connection",
+ "headers": {
+ "aggregate": false,
+ "is-group-topic": false,
+ "ditto-sudo": true
+ },
+ "piggybackCommand": {
+ "type": "connectivity.commands:modifyConnection",
+ "connection": {}
+ }
+}
```
-connectivity.commands:resetConnectionLogs
+
+### Retrieve connection (piggyback)
+
+```json
+{
+ "targetActorSelection": "/system/sharding/connection",
+ "headers": {
+ "aggregate": false,
+ "is-group-topic": false,
+ "ditto-sudo": true
+ },
+ "piggybackCommand": {
+ "type": "connectivity.commands:retrieveConnection",
+ "connectionId": ""
+ }
+}
```
-## Publishing connection logs
+### Open connection (piggyback)
-In addition to [enable collecting in-memory connection logs](#enable-connection-logs), connection logs may also be
-published to a [Fluentd](https://www.fluentd.org) or [Fluent Bit](https://fluentbit.io) endpoint from where they can be
-forwarded into a logging backend of your choice.
+```json
+{
+ "targetActorSelection": "/system/sharding/connection",
+ "headers": {
+ "aggregate": false,
+ "is-group-topic": false,
+ "ditto-sudo": true
+ },
+ "piggybackCommand": {
+ "type": "connectivity.commands:openConnection",
+ "connectionId": ""
+ }
+}
+```
-This publishing of connection logs is not configured dynamically via a [piggyback helper endpoint](#helper-endpoints),
-instead this must be enabled via [configuration](installation-operating.html#ditto-configuration) in the
-`connectivity.conf` by setting environment variables or overwriting the configuration via system properties.
+### Close connection (piggyback)
-For example, set the following environment variables in the [connectivity](architecture-services-connectivity.html)
-service:
-* `CONNECTIVITY_LOGGER_PUBLISHER_ENABLED` - set to `true` in order to enable connection log publishing, default: `false`
-* `CONNECTIVITY_LOGGER_PUBLISHER_LOG_TAG` - to specify a log tag to use for the published log entries -
- if this is not set, the default `connection:` will be used and the specific connection-id will be injected
-* `CONNECTIVITY_LOGGER_PUBLISHER_FLUENCY_HOST` - the hostname of the Fluentd or Fluent Bit endpoint, default: `"localhost"`
-* `CONNECTIVITY_LOGGER_PUBLISHER_FLUENCY_PORT` - the port of the Fluentd or Fluent Bit endpoint, default `24224`
+```json
+{
+ "targetActorSelection": "/system/sharding/connection",
+ "headers": {
+ "aggregate": false,
+ "is-group-topic": false,
+ "ditto-sudo": true
+ },
+ "piggybackCommand": {
+ "type": "connectivity.commands:closeConnection",
+ "connectionId": ""
+ }
+}
+```
-The contained fields in a single log entry are the following:
-* `connectionId`: ID of the connection which produced the log entry
-* `level`: one of: `"success"|"failure"`
-* `category`: one of: `"connection"|"source"|"target"|"response"`
-* `type`: one of: `"consumed"|"dispatched"|"filtered"|"mapped"|"dropped"|"enforced"|"published"|"acknowledged"`
-* `correlationId`: correlationId of the command/event, if available
-* `address`: address of the Source/Target (e.g. MQTT topic, HTTP path), if available
-* `entityType` : one of: `"thing"|"policy"`
-* `entityId`: ID of the entity for which e.g. an event/message was processed (e.g. the Thing ID)
-* `instanceId`: ID of the connectivity instance which processed the command/event, helpful if clientCount > 1 was configured
-* `message`: the actual log message
+### Delete connection (piggyback)
-Please inspect the other available configuration options in
-[connectivity.conf](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/resources/connectivity.conf)
-at path `ditto.connectivity.monitoring.logger.publisher` to learn about other configuration possibilities.
+```json
+{
+ "targetActorSelection": "/system/sharding/connection",
+ "headers": {
+ "aggregate": false,
+ "is-group-topic": false,
+ "ditto-sudo": true
+ },
+ "piggybackCommand": {
+ "type": "connectivity.commands:deleteConnection",
+ "connectionId": ""
+ }
+}
+```
+### Test connection (piggyback)
-## Payload mapping configuration
+```json
+{
+ "targetActorSelection": "/system/sharding/connection",
+ "headers": {
+ "aggregate": false,
+ "is-group-topic": false,
+ "ditto-sudo": true
+ },
+ "piggybackCommand": {
+ "type": "connectivity.commands:testConnection",
+ "connection": {}
+ }
+}
+```
-To enable a custom [payload mapping](connectivity-mapping.html) for a specific source or target of a connection, you
-have to configure a payload mapping definition in the connection configuration object. The following snippet shows an
-example `mappingDefinitions`. This configuration must be embedded in the connection configuration as shown in the
-[Connections](basic-connections.html) section. These payload mapping definitions are then referenced by its ID
-(the key of the JSON object) in the sources and targets of the connection using the field `payloadMapping`. If no
-payload mapping or definition is provided, the [Ditto message mapping](connectivity-mapping.html#ditto-mapper)
-is used as the default.
+### Retrieve all connection IDs (piggyback)
```json
{
- ...
- "mappingDefinitions": {
- "customJs": {
- // (1)
- "mappingEngine": "JavaScript",
- // (2)
- "options": {
- // (3)
- "incomingScript": "..",
- "outgoingScript": ".."
- }
- }
+ "targetActorSelection": "/user/connectivityRoot/connectionIdsRetrieval",
+ "headers": {
+ "aggregate": false,
+ "is-group-topic": false,
+ "ditto-sudo": true
},
- "sources": [
- {
- "addresses": "source",
- "payloadMapping": [
- "customJs"
- ]
- }
- ]
- ...
+ "piggybackCommand": {
+ "type": "connectivity.commands:retrieveAllConnectionIds"
+ }
}
```
-- (1) This ID can be used in sources and targets of the connection to reference this payload mapping definition.
-- (2) The `mappingEngine` defines the underlying `MessageMapper` implementation.
-- (3) The `options` are used to configure the mapper instance to your needs.
+## Further reading
+
+* [Connections overview](basic-connections.html) -- connection model and configuration
+* [Payload mapping](connectivity-mapping.html) -- transform message payloads
+* [Header mapping](connectivity-header-mapping.html) -- map external headers
+* [Installation and operation](installation-operating.html) -- DevOps configuration
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-mapping.md b/documentation/src/main/resources/pages/ditto/connectivity-mapping.md
index 8f3aa1bb387..1b280dbef6c 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-mapping.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-mapping.md
@@ -1,151 +1,104 @@
---
-title: Payload mapping in connectivity service
+title: Payload Mapping
keywords: mapping, transformation, payload, javascript, mapper, protobuf
tags: [connectivity]
permalink: connectivity-mapping.html
---
-{% include callout.html content="**TL;DR** The payload mapping feature in Ditto's connectivity APIs can be used to
- transform arbitrary payload consumed via the different supported protocols
- to [Ditto Protocol](protocol-overview.html) messages and vice versa." type="primary" %}
+You use payload mapping to transform messages between your device's native format and [Ditto Protocol](protocol-overview.html) JSON.
+{% include callout.html content="**TL;DR**: Payload mapping transforms arbitrary payloads consumed via connections
+ to [Ditto Protocol](protocol-overview.html) messages and vice versa. Use built-in mappers or write custom JavaScript to handle any format." type="primary" %}
-## Motivation
+## Overview
-Eclipse Ditto is about providing access to IoT devices via the [digital twin](intro-digitaltwins.html) pattern.
-In order to provide structured APIs for different heterogeneous devices Ditto defines a lightweight JSON based
-[model](basic-overview.html).
-
-A [Thing](basic-thing.html) might look like in the following example:
-
-```json
-{
- "thingId": "the.namespace:the-thing-id",
- "policyId": "the.namespace:the-policy-id",
- "attributes": {
- "location": "kitchen"
- },
- "features": {
- "transmission": {
- "properties": {
- "cur_speed": 90
- }
- }
- }
-}
-```
-
-Devices in the IoT, may they be brownfield devices or newly produced devices, will probably not send their data to the
-cloud in the structure and [protocol](protocol-overview.html) Ditto requires.
-
-They should not need to be aware of something like Ditto running in the cloud mirroring them as digital twins.
-
-So for example device payload could look like this:
+Devices rarely send data in Ditto Protocol format. A device might send:
```json
-{
- "val": "23.42 °C",
- "ts": 1523946112727
-}
+{"val": "23.42 °C", "ts": 1523946112727}
```
-In case of constrained devices or IoT protocols, even binary payload might be common.
+Or even binary:
```
-0x08BD (hex representation)
+0x08BD
```
-## Builtin mappers
+Payload mapping bridges this gap by converting between device-native formats and the structured
+[Ditto Protocol](protocol-specification.html) that Ditto requires.
-The following message mappers are included in the Ditto codebase:
+## Built-in mappers
-| Mapper Alias | Description | Inbound | Outbound |
-|------------|--------------------------------|---------------------------|---------------------------|
-| [Ditto](#ditto-mapper) | Assumes that inbound/outbound messages are already in [Ditto Protocol](protocol-overview.html) (JSON) format. | ✓ | ✓ |
-| [JavaScript](#javascript-mapper) | Converts arbitrary messages from and to the [Ditto Protocol](protocol-overview.html) format using **custom** JavaScript code executed by Ditto. | ✓ | ✓ |
-| [Normalized](#normalized-mapper) | Transforms the payload of events to a normalized view. | | ✓ |
-| [ConnectionStatus](#connectionstatus-mapper) | This mapper handles messages containing `creation-time` and `ttd` headers by updating a feature of the targeted thing with [definition](basic-feature.html#feature-definition) [ConnectionStatus](https://github.com/eclipse/vorto/tree/development/models/org.eclipse.ditto-ConnectionStatus-1.0.0.fbmodel). | ✓ | |
-| [RawMessage](#rawmessage-mapper) | For outgoing message commands and responses, this mapper extracts the payload for publishing directly into the channel. For incoming messages, this mapper wraps them in a configured message command or response envelope. | ✓ | ✓ |
-| [ImplicitThingCreation](#implicitthingcreation-mapper) | This mapper handles messages for which a Thing should be created automatically based on a defined template. | ✓ | |
-| [UpdateTwinWithLiveResponse](#updatetwinwithliveresponse-mapper) | This mapper creates a [merge Thing command](protocol-specification-things-merge.html) when an indiviudal [retrieve command](protocol-specification-things-retrieve.html) for an single Thing was received via the [live channel](protocol-twinlive.html#live) patching exactly the retrieved "live" data into the twin. | ✓ | |
-| [CloudEvents Mapper](#cloudevents-mapper) | The mapper maps incoming CloudEvent to Ditto Protocol. Supports both Binary and Structured CloudEvent. | ✓ | ✓ |
+| Mapper | Alias | Inbound | Outbound | Description |
+|--------|-------|:---:|:---:|-------------|
+| [Ditto](#ditto-mapper) | `Ditto` | ✔ | ✔ | Messages already in Ditto Protocol format |
+| [JavaScript](#javascript-mapper) | `JavaScript` | ✔ | ✔ | Custom JS scripts for arbitrary formats |
+| [Normalized](#normalized-mapper) | `Normalized` | | ✔ | Transforms events to a normalized JSON view |
+| [ConnectionStatus](#connectionstatus-mapper) | `ConnectionStatus` | ✔ | | Updates a feature based on `ttd`/`creation-time` headers |
+| [RawMessage](#rawmessage-mapper) | `RawMessage` | ✔ | ✔ | Maps message command payloads directly |
+| [ImplicitThingCreation](#implicitthingcreation-mapper) | `ImplicitThingCreation` | ✔ | | Auto-creates things from incoming messages |
+| [UpdateTwinWithLiveResponse](#updatetwinwithliveresponse-mapper) | `UpdateTwinWithLiveResponse` | ✔ | | Patches twin data from live responses |
+| [CloudEvents](#cloudevents-mapper) | `CloudEvents` | ✔ | ✔ | Maps CloudEvent format to Ditto Protocol |
### Ditto mapper
-This is the default [Ditto Protocol](protocol-overview.html) mapper. If you do not specify any payload mapping this
- mapper is used to map inbound and outbound messages. The mapper requires no mandatory options, so its alias can
- be directly used as a mapper reference.
-
-It assumes that received messages are in [Ditto Protocol JSON](protocol-specification.html) and emits outgoing messages
- also in that format.
+The default mapper. Assumes messages are in [Ditto Protocol JSON](protocol-specification.html).
+No configuration required -- use the alias `Ditto` directly.
### JavaScript mapper
-This mapper may be used whenever any inbound messages are not yet in [Ditto Protocol](protocol-overview.html).
-By using the built in [JavaScript mapping engine](#javascript-mapping-engine) (based on Rhino) custom defined
-JavaScript scripts can be executed which are responsible for creating [Ditto Protocol JSON](protocol-specification.html)
-message from arbitrary consumed payload.
+Transforms arbitrary payloads using custom JavaScript scripts executed in a sandboxed
+[Rhino](https://github.com/mozilla/rhino) engine. See the
+[JavaScript mapping engine](#javascript-mapping-engine) section for details.
-The same is possible for outbound messages in order to transform [Ditto Protocol JSON](protocol-specification.html)
-messages (e.g. events or responses) to arbitrary other formats.
+**Options:**
-#### Configuration options
-
-* `incomingScript` (required): the mapping script for incoming messages
-* `outgoingScript` (required): the mapping script for outgoing messages
-* `loadBytebufferJS` (optional, default: `"false"`): whether to load ByteBufferJS library
-* `loadLongJS` (optional, default: `"false"`): whether to load LongJS library
+| Option | Required | Description |
+|--------|----------|-------------|
+| `incomingScript` | Yes | Script for inbound messages |
+| `outgoingScript` | Yes | Script for outbound messages |
+| `loadBytebufferJS` | No | Load ByteBufferJS library (default: `false`) |
+| `loadLongJS` | No | Load LongJS library (default: `false`) |
### Normalized mapper
-This mapper transforms `created`, `modified`, and `deleted` events (other type of messages are dropped) to a normalized view.
-Events are mapped to a nested sparse JSON.
-
-**Note:** By default, only complete thing deletions (`ThingDeleted`) are mapped with a special `_deleted` field. Partial
-deletions like `AttributeDeleted`, `FeatureDeleted`, etc. are dropped unless explicitly enabled (see option below).
+Transforms `created`, `modified`, and `deleted` events to a normalized JSON structure.
+Other message types are dropped.
-#### Configuration options
+**Options:**
-* `includeDeletedFields` (optional, default: `false`): when enabled, partial delete events are mapped and merge-patch
- `null` values are tracked in `_deletedFields`. The `_deletedFields` object mirrors the JSON structure of the deleted
- paths and stores ISO-8601 timestamps at the leaf nodes.
+| Option | Default | Description |
+|--------|---------|-------------|
+| `fields` | all | Comma-separated list of [field selectors](httpapi-concepts.html#field-selectors) |
+| `includeDeletedFields` | `false` | Track partial deletions in `_deletedFields` |
+Example input:
```json
{
"topic": "thing/id/things/twin/events/modified",
- "headers": { "content-type": "application/json" },
- "path": "/features/sensors/properties/temperature/indoor/value",
+ "path": "/features/sensors/properties/temperature/value",
"value": 42
}
```
-would result in the following normalized JSON representation:
-
+Normalized output:
```json
{
"thingId": "thing:id",
"features": {
"sensors": {
"properties": {
- "temperature": {
- "indoor": {
- "value": 42
- }
- }
+ "temperature": { "value": 42 }
}
}
},
"_context": {
"topic": "thing/id/things/twin/events/modified",
- "path": "/features/sensors/properties/temperature/indoor/value",
- "value": 42,
- "headers": {
- "content-type": "application/json"
- }
+ "path": "/features/sensors/properties/temperature/value",
+ "value": 42
}
}
```
-The `_context` field contains the original message content excluding the `value`.
For `deleted` events, the mapper includes a `_deleted` field with the deletion timestamp:
@@ -158,7 +111,7 @@ For `deleted` events, the mapper includes a `_deleted` field with the deletion t
}
```
-would result in the following normalized JSON representation:
+Normalized output:
```json
{
@@ -175,11 +128,9 @@ would result in the following normalized JSON representation:
}
```
-The `_deleted` field contains the timestamp when the thing was deleted in ISO-8601 format.
-
-**Note:** The `_deleted` field is only added for complete thing deletions (when the entire thing is deleted).
+The `_deleted` field contains the ISO-8601 timestamp when the thing was deleted. This field is only added for complete thing deletions.
-When `includeDeletedFields` is enabled, the mapper adds `_deletedFields` for partial deletions and merge-patch deletions:
+When `includeDeletedFields` is enabled, partial deletions and merge-patch deletions are tracked in `_deletedFields`. The `_deletedFields` object mirrors the JSON structure of the deleted paths and stores ISO-8601 timestamps at the leaf nodes:
```json
{
@@ -197,21 +148,24 @@ When `includeDeletedFields` is enabled, the mapper adds `_deletedFields` for par
}
```
-#### Configuration options
-
-* `fields` (optional, default: all fields): comma separated list of fields that are contained in the result (see also
- chapter about [field selectors](httpapi-concepts.html#with-field-selector))
-
### ConnectionStatus mapper
-This mapper transforms the information from the `ttd` and `creation-time` message headers
-(see Eclipse Hono [device notifications](https://www.eclipse.org/hono/docs/concepts/device-notifications/)) into a
-ModifyFeature command that complies with the [Vorto functionblock](https://github.com/eclipse/vorto/tree/development/models/org.eclipse.ditto-ConnectionStatus-1.0.0.fbmodel) `{%raw%}org.eclipse.ditto:ConnectionStatus{%endraw%}`.
-
-The connectivity state of the device is then represented in a Feature.
-It is mostly used in conjunction with another mapper that transforms the payload e.g.:
-`"payloadMapping": [ "Ditto" , "connectionStatus" ]`
-
+
+Transforms `ttd` and `creation-time` headers (from
+[Eclipse Hono device notifications](https://www.eclipse.org/hono/docs/concepts/device-notifications/))
+into a ModifyFeature command that updates a `ConnectionStatus` feature.
+
+Typically used alongside another mapper:
+`"payloadMapping": ["Ditto", "connectionStatus"]`
+
+**Options:**
+
+| Option | Required | Description |
+|--------|----------|-------------|
+| `thingId` | Yes | Thing ID (supports placeholders like `{%raw%}{{ header:device_id }}{%endraw%}`) |
+| `featureId` | No | Feature ID (default: `ConnectionStatus`) |
+
Example of a resulting `ConnectionStatus` feature:
+
```json
{
"thingId": "eclipse:ditto",
@@ -228,25 +182,47 @@ Example of a resulting `ConnectionStatus` feature:
}
}
```
-
-#### Configuration options
-* `thingId` (required): The ID of the Thing that is updated with the connectivity state. It can either be a fixed value
- or a header placeholder (e.g. `{%raw%}{{ header:device_id }}{%endraw%}`).
-* `featureId` (optional, default: `ConnectionStatus`): The ID of the Feature that is updated. It can either be a
- fixed value or resolved from a message header (e.g. `{%raw%}{{ header:feature_id }}{%endraw%}`).
+Use the ConnectionStatus mapper alongside another mapper in a source configuration:
+
+```json
+{
+ "addresses": [""],
+ "authorizationContext": ["ditto:inbound"],
+ "payloadMapping": ["Ditto", "connectionStatus"]
+}
+```
### RawMessage mapper
-This mapper relates the payload in the `"value"` field of message commands and message responses to the payload
-of AMQP, MQTT and Kafka messages and the body of HTTP requests. The encoding of the payload is chosen according to
-the configured content type. The subject, direction, thing ID and feature ID of the envelope for incoming message
-commands and responses need to be configured.
+Maps message command/response payloads directly to/from the external message format. The encoding
+is determined by the content type.
+
+For incoming messages, the mapper wraps the payload in a message command envelope.
+For outgoing messages, the mapper extracts the `"value"` field for publishing.
+
+Messages with `content-type: application/vnd.eclipse.ditto+json` fall through to the Ditto mapper.
-Messages with the Ditto protocol content type `application/vnd.eclipse.ditto+json` or signals that are not message
-commands or responses are mapped by the [Ditto mapper](#ditto-mapper) instead.
+**Options:**
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `outgoingContentType` | `text/plain; charset=UTF-8` | Fallback content type for outgoing messages |
+| `incomingMessageHeaders` | (see below) | Headers for constructing the message envelope |
+
+Key incoming headers (all support placeholders):
+
+| Header | Purpose | Default |
+|--------|---------|---------|
+| `content-type` | Encoding of the payload | `{%raw%}{{ header:content-type \| fn:default('application/octet-stream') }}{%endraw%}` |
+| `ditto-message-subject` | Message subject (required for MQTT 3) | `{%raw%}{{ header:ditto-message-subject }}{%endraw%}` |
+| `ditto-message-thing-id` | Target thing ID (required for MQTT 3) | `{%raw%}{{ header:ditto-message-thing-id }}{%endraw%}` |
+| `ditto-message-direction` | `TO` (inbox) or `FROM` (outbox) | `TO` |
+| `ditto-message-feature-id` | Feature ID (omit for thing-level messages) | `{%raw%}{{ header:ditto-message-feature-id }}{%endraw%}` |
+| `status` | Include for responses, omit for commands | `{%raw%}{{ header:status }}{%endraw%}` |
+
+The mapper maps between a feature message command response like:
-For example, the mapper maps between the feature message command response
```json
{
"topic": "org.eclipse.ditto/smartcoffee/things/live/messages/heatUp",
@@ -256,11 +232,15 @@ For example, the mapper maps between the feature message command response
"status": 200
}
```
-and an AMQP, MQTT 5, Kafka message with payload or an HTTP request with body of 6 bytes
+
+and an AMQP, MQTT 5, or Kafka message with payload of 6 bytes:
+
```
0x01 02 03 04 05 06
```
-and headers
+
+with headers:
+
```
content-type: application/octet-stream
status: 200
@@ -269,11 +249,11 @@ ditto-message-direction: TO
ditto-message-thing-id: org.eclipse.ditto:smartcoffee
ditto-message-feature-id: water-tank
```
-The headers are lost for connection protocols without application headers such as MQTT 3.
-
-#### Configuration options
+
+Headers are lost for connection protocols without application headers such as MQTT 3.
Example configuration:
+
```json
{
"outgoingContentType": "application/octet-stream",
@@ -288,77 +268,43 @@ Example configuration:
}
```
-* `outgoingContentType` (optional): The fallback content-type for outgoing message commands and responses without
- the content-type header. Default to `text/plain; charset=UTF-8`.
-* `incomingMessageHeaders` (optional): A JSON object containing the following headers needed to construct a message
- command or response envelope containing the incoming message as payload in the field `"value"`.
- The following placeholders may be used in the headers:
-
- | Placeholder | Description |
- |-----------------------------------|--------------|
- | `{%raw%}{{ header: }}{%endraw%}` | header value from the external message, e.g. from protocol headers |
- | `{%raw%}{{ request:subjectId }}{%endraw%}` | the first authenticated subjectId which did the request - the one of the connection source in this case |
- | `{%raw%}{{ time:now }}{%endraw%}` | the current timestamp in ISO-8601 format as string in UTC timezone |
- | `{%raw%}{{ time:now_epoch_millis }}{%endraw%}` | the current timestamp in "milliseconds since epoch" formatted as string |
-
- * `content-type` (optional): The content type with which to encode the incoming message as payload.
- Default to `{%raw%}{{ header:content-type | fn:default('application/octet-stream') }}{%endraw%}`.
- If resolved to the Ditto protocol content type `application/vnd.eclipse.ditto+json`, then the entire payload
- is interpreted as a Ditto protocol message instead.
- * `status` (optional): Include for message responses. Exclude for message commands. Default to
- `{%raw%}{{ header:status }}{%endraw%}`.
- * `ditto-message-subject` (mandatory for MQTT 3): Subject of the message. Default to `{%raw%}{{ header:ditto-message-subject }}{%endraw%}`.
- Mapping will fail if not resolvable.
- * `ditto-message-direction` (optional): The message direction. Default to `TO`, which corresponds to `inbox` in
- message commands and responses.
- * `ditto-message-thing-id` (mandatory for MQTT 3): ID of the thing to send the message command or response to.
- Default to `{%raw%}{{ header:ditto-message-thing-id }}{%endraw%}`. Mapping will fail if not resolvable.
- * `ditto-message-feature-id` (optional): Include to send the message or message response to a feature of the thing.
- Exclude to send it to the thing itself. Default to `{%raw%}{{ header:ditto-message-feature-id }}{%endraw%}`.
-
### ImplicitThingCreation mapper
-This mapper implicitly creates a new thing for an incoming message.
-
-The created thing contains the values defined in the template, configured in the `mappingDefinitions` `options`.
+Automatically creates a thing when an incoming message arrives. The thing structure is defined
+in the `thing` option as a JSON template with placeholder support.
-#### Configuration options
+**Options:**
-* `thing` (required): The values of the thing that is created implicitly. It can either contain fixed values
- or header placeholders (e.g. `{%raw%}{{ header:device_id }}{%endraw%}`).
- * the following placeholders may be used inside the `"thing"` JSON:
+* `thing` (required): The values of the thing that is created implicitly. It can contain fixed values
+ or header placeholders (e.g. `{%raw%}{{ header:device_id }}{%endraw%}`).
+ * The following placeholders may be used inside the `"thing"` JSON:
- | Placeholder | Description |
- |------------------------------------|--------------|
- | `{%raw%}{{ header: }}{%endraw%}` | header value from the external message, e.g. from protocol headers |
- | `{%raw%}{{ request:subjectId }}{%endraw%}` | the first authenticated subjectId which did the request - the one of the connection source in this case |
- | `{%raw%}{{ time:now }}{%endraw%}` | the current timestamp in ISO-8601 format as string in UTC timezone |
- | `{%raw%}{{ time:now_epoch_millis }}{%endraw%}` | the current timestamp in "milliseconds since epoch" formatted as string |
+ | Placeholder | Description |
+ |-------------|-------------|
+ | `{%raw%}{{ header: }}{%endraw%}` | Header value from the external message, e.g. from protocol headers |
+ | `{%raw%}{{ request:subjectId }}{%endraw%}` | The first authenticated subjectId which did the request |
+ | `{%raw%}{{ time:now }}{%endraw%}` | The current timestamp in ISO-8601 format as string in UTC timezone |
+ | `{%raw%}{{ time:now_epoch_millis }}{%endraw%}` | The current timestamp in "milliseconds since epoch" formatted as string |
* The `"thing"` JSON may also include:
- * an inline policy: `"_policy"` containing the [Policy JSON](basic-policy.html#model-specification) to create a new policy
- from and link with the thing
+ * an inline policy: `"_policy"` containing the [Policy JSON](basic-policy.html#model-specification)
* a "copy policy from" statement: `"_copyPolicyFrom"` - see also [create Thing alternatives](protocol-examples-creatething.html#alternative-creatething-commands)
* either including a policyId to copy from
- * or containing the link to a thing to copy the policy from in the form: `{% raw %}{{ ref:things//policyId }}{% endraw %}`
-
-* `commandHeaders` (optional, default: `{"If-None-Match": "*"}`): The Ditto headers to use for constructing the "create thing" command for creating the
- twin and to use for creating errors.
- * in this configured headers, the following placeholders may be used:
-
- | Placeholder | Description |
- |-----------------------------------|--------------|
- | `{%raw%}{{ header: }}{%endraw%}` | header value from the external message, e.g. from protocol headers |
- | `{%raw%}{{ request:subjectId }}{%endraw%}` | the first authenticated subjectId which did the request - the one of the connection source in this case |
- | `{%raw%}{{ time:now }}{%endraw%}` | the current timestamp in ISO-8601 format as string in UTC timezone |
- | `{%raw%}{{ time:now_epoch_millis }}{%endraw%}` | the current timestamp in "milliseconds since epoch" formatted as string |
-
-* `allowPolicyLockout` (optional, default: `true`): whether it should be allowed to create policies without having `WRITE`
- permissions in the created policy for the subject which creates the policy
- (the [authorizationContext](connectivity-manage-connections.html#authorization) of the connection source which
- received the message for which a thing should be created implicitly)
-
-Example of a template defined in `options`:
+ * or containing the link to a thing to copy the policy from in the form: `{%raw%}{{ ref:things//policyId }}{%endraw%}`
+
+* `commandHeaders` (optional, default: `{"If-None-Match": "*"}`): The Ditto headers to use for constructing the "create thing" command.
+ * The following placeholders may be used:
+
+ | Placeholder | Description |
+ |-------------|-------------|
+ | `{%raw%}{{ header: }}{%endraw%}` | Header value from the external message, e.g. from protocol headers |
+ | `{%raw%}{{ request:subjectId }}{%endraw%}` | The first authenticated subjectId which did the request |
+ | `{%raw%}{{ time:now }}{%endraw%}` | The current timestamp in ISO-8601 format as string in UTC timezone |
+ | `{%raw%}{{ time:now_epoch_millis }}{%endraw%}` | The current timestamp in "milliseconds since epoch" formatted as string |
+
+* `allowPolicyLockout` (optional, default: `true`): Whether it should be allowed to create policies without having `WRITE`
+ permissions in the created policy for the subject which creates the policy.
+
```json
{
"thing": {
@@ -377,26 +323,25 @@ Example of a template defined in `options`:
### UpdateTwinWithLiveResponse mapper
-This mapper creates a [merge Thing command](protocol-specification-things-merge.html) when a
-[retrieve command](protocol-specification-things-retrieve.html) was received via the
-[live channel](protocol-twinlive.html#live) patching exactly the retrieved "live" data into the twin.
-
-#### Configuration options
+Creates a [merge Thing command](protocol-specification-things-merge.html) from a
+[live retrieve response](protocol-twinlive.html#live-channel), patching the live data into the twin.
-* `dittoHeadersForMerge` (optional): The Ditto headers to use for constructing the "merge thing"
- command for updating the twin, may for example add a condition to apply in order to update the twin
- (default applied Ditto headers if not configured: `"response-required": false`, `"if-match": "*"`).
- * in this configured headers, the following placeholders may be used:
+**Options:**
- | Placeholder | Description |
- |-----------------------------------|--------------|
- | `{%raw%}{{ header: }}{%endraw%}` | header value from the external message, e.g. from protocol headers |
- | `{%raw%}{{ request:subjectId }}{%endraw%}` | the first authenticated subjectId which did the request - the one of the connection source in this case |
- | `{%raw%}{{ time:now }}{%endraw%}` | the current timestamp in ISO-8601 format as string in UTC timezone |
- | `{%raw%}{{ time:now_epoch_millis }}{%endraw%}` | the current timestamp in "milliseconds since epoch" formatted as string |
+* `dittoHeadersForMerge` (optional): The Ditto headers to use for constructing the "merge thing"
+ command, may for example add a condition to apply in order to update the twin
+ (default: `"response-required": false`, `"if-match": "*"`).
+ * The following placeholders may be used:
+ | Placeholder | Description |
+ |-------------|-------------|
+ | `{%raw%}{{ header: }}{%endraw%}` | Header value from the external message, e.g. from protocol headers |
+ | `{%raw%}{{ request:subjectId }}{%endraw%}` | The first authenticated subjectId which did the request |
+ | `{%raw%}{{ time:now }}{%endraw%}` | The current timestamp in ISO-8601 format as string in UTC timezone |
+ | `{%raw%}{{ time:now_epoch_millis }}{%endraw%}` | The current timestamp in "milliseconds since epoch" formatted as string |
Example configuration:
+
```json
{
"dittoHeadersForMerge": {
@@ -411,11 +356,13 @@ Example configuration:
}
}
```
-### CloudEvents Mapper
-This mapper maps incoming [CloudEvent](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md) to Ditto Protocol. It provides support for both Binary CloudEvents as well as Structured CloudEvents.
+### CloudEvents mapper
+
+Maps incoming [CloudEvents](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md) to
+Ditto Protocol. Supports both Binary and Structured CloudEvents.
-**Note**: The mapper supports incoming Structured CloudEvents messages with `content-type:application/cloudevents+json` and Binary CloudEvents message with `content-type:application/vnd.eclipse.ditto+json`
+**Note**: The mapper supports incoming Structured CloudEvents messages with `content-type:application/cloudevents+json` and Binary CloudEvents messages with `content-type:application/vnd.eclipse.ditto+json`.
#### CloudEvents examples
@@ -458,7 +405,6 @@ For example, a Binary CloudEvent for Ditto would look like this:
A Structured CloudEvent for Ditto would look like this:
```
-
headers:
content-type:application/cloudevents+json
```
@@ -493,21 +439,19 @@ headers:
}
```
-## Example connection with multiple mappers
+## Using multiple mappers
-The following example connection defines a `ConnectionStatus` mapping with the ID `status` and references it in a source.
-Messages received via this source will be mapped by the `Ditto` mapping and the `ConnectionStatus` mapping.
-The `Ditto` mapping requires no options to be configured, so you can directly use its alias `Ditto`.
+Reference multiple mappers in a source's `payloadMapping` array. Define custom mappers in
+`mappingDefinitions`:
```json
-{
+{
"name": "exampleConnection",
"sources": [{
- "addresses": [""],
- "authorizationContext": ["ditto:inbound"],
- "payloadMapping": ["Ditto", "status"]
- }
- ],
+ "addresses": [""],
+ "authorizationContext": ["ditto:inbound"],
+ "payloadMapping": ["Ditto", "status"]
+ }],
"mappingDefinitions": {
"status": {
"mappingEngine": "ConnectionStatus",
@@ -519,30 +463,15 @@ The `Ditto` mapping requires no options to be configured, so you can directly us
}
```
-{% include note.html content="Starting aliases with an uppercase character and IDs with a lowercase character is
- encouraged to avoid confusion but this is not enforced. "%}
-
-
+{% include note.html content="Start aliases with an uppercase character and IDs with a lowercase character to distinguish them clearly. This convention is not enforced. "%}
-## Example connection with mapping conditions
+## Mapping conditions
-The following example connection defines `incomingConditions` and `outgoingConditions`for the ConnectionStatus
-mapping engine.
-Optional incomingConditions are validated before the mapping of inbound messages.
-Optional outgoingConditions are validated before the mapping of outbound messages.
-Conditional Mapping can be achieved by using [function expressions](basic-placeholders.html#function-expressions).
-When multiple incoming or outgoing conditions are set for one `mappingEngine`,
-all have to equal true for the mapping to be executed.
+You can add `incomingConditions` and `outgoingConditions` to control when a mapper executes.
+All conditions must evaluate to true for the mapping to run:
```json
-{
- "name": "exampleConnection",
- "sources": [{
- "addresses": [""],
- "authorizationContext": ["ditto:inbound"],
- "payloadMapping": ["status"]
- }
- ],
+{
"mappingDefinitions": {
"status": {
"mappingEngine": "ConnectionStatus",
@@ -560,103 +489,52 @@ all have to equal true for the mapping to be executed.
}
```
-
-
## JavaScript mapping engine
-Ditto utilizes the [Rhino](https://github.com/mozilla/rhino) JavaScript engine for Java for evaluating the JavaScript
-to apply for mapping payloads.
-
-Using Rhino instead of Nashorn, the newer JavaScript engine shipped with Java, has the benefit that sandboxing can be
-applied in a better way.
-
-Sandboxing of different payload scripts is required as Ditto is intended to be run as cloud service where multiple
-connections to different endpoints are managed for different tenants at the same time. This requires the isolation of
-each single script to avoid interference with other scripts and to protect the JVM executing the script against harmful
-code execution.
-
-
-### Constraints
+Ditto uses the [Rhino](https://github.com/mozilla/rhino) JavaScript engine (version `1.7.14`,
+ES6 flag enabled) with strict sandboxing for security.
-Rhino does not fully support EcmaScript 6. Check which language constructs are supported before using
-them in a mapping function. See [https://mozilla.github.io/rhino/compat/engines.html](https://mozilla.github.io/rhino/compat/engines.html).
+### Sandboxing constraints
-Ditto currently includes Rhino version `1.7.14` and has the `VERSION_ES6` flag enabled.
+* No access to Java packages or classes
+* No file access, network calls, or `exit`/`quit`/`print`
+* Endless loops and deep recursion are terminated
+* Script file size is limited
+* No foreign JS library loading (unless included inline)
-#### Sandboxing
-
-For sandboxing/security reasons following restrictions apply:
-
-
-* access to Java packages and classes is not possible
-* using `exit`, `quit`, `print`, etc. is not possible
-* file access is not possible
-* doing remote calls (e.g. to foreign web-servers) is not possible
-* programming an endless-loop will terminate the script
-* programming a recursion will terminate the script
-* the file size of the script is limited
-* no foreign JS libraries can be loaded (unless they fit in the file size limit and are included into the mapping script)
+Check [Rhino compatibility](https://mozilla.github.io/rhino/compat/engines.html) for supported
+ES6 features.
### Helper libraries
-In order to work more conveniently with binary payloads, the following libraries may be loaded for payload transformations:
-
+You can load these libraries via `specificConfig` options:
-* [bytebuffer.js](https://github.com/dcodeIO/bytebuffer.js) a ByteBuffer implementation using ArrayBuffers
-* [long.js](https://github.com/dcodeIO/long.js) for representing a 64-bit two's-complement integer value
+* [bytebuffer.js](https://github.com/dcodeIO/bytebuffer.js) -- `ArrayBuffer` manipulation
+* [long.js](https://github.com/dcodeIO/long.js) -- 64-bit integer support
+### Adding CommonJS modules
-### Adding additional JS libraries
+Configure `CONNECTIVITY_MESSAGE_MAPPING_JS_COMMON_JS_MODULE_PATH` to point to a directory
+containing CommonJS modules (for example, via a Docker volume mount):
-The used [Rhino JS engine](https://github.com/mozilla/rhino) allows making use of "CommonJS" in order to load JS
-modules via `require('')` into the engine.
-This feature is exposed to Ditto, configuring the configuration key `commonJsModulePath` or environment variable
-`CONNECTIVITY_MESSAGE_MAPPING_JS_COMMON_JS_MODULE_PATH` of the connectivity service to a path in the
-connectivity Docker container where to load additional CommonJS modules from - e.g. use a volume mount in order to get
-additional JS modules into the container.
-
-For example, configure this variable to a folder to which you add (our mount) JavaScript libraries:
```
CONNECTIVITY_MESSAGE_MAPPING_JS_COMMON_JS_MODULE_PATH=/opt/commonjs-modules/
```
-Then, for example, put [`pbf.js`](https://www.npmjs.com/package/pbf) (or any other JS library you want to use)
-into that folder.
+Then use `require()` in your scripts:
-Afterwards, the library can be used in your JS snippet using:
```javascript
var Pbf = require('pbf');
```
-
### Helper functions
-Ditto comes with a few helper functions, which makes writing the mapping scripts easier. They are available under the
-`Ditto` scope:
+Ditto provides these functions under the `Ditto` scope:
```javascript
-/**
- * Builds a Ditto Protocol message from the passed parameters.
- * @param {string} namespace - The namespace of the entity in java package notation, e.g.: "org.eclipse.ditto". Or "_"
- * (underscore) for connection announcements.
- * @param {string} name - The name of the entity, e.g.: "device".
- * @param {string} channel - The channel for the signal: "twin"|"live"|"none"
- * @param {string} group - The affected group/entity: "things"|"policies"|"connections".
- * @param {string} criterion - The criterion to apply: "commands"|"events"|"search"|"messages"|"announcements"|"errors".
- * @param {string} action - The action to perform: "create"|"retrieve"|"modify"|"delete". Or the announcement name:
- * "opened"|"closed"|"subjectDeletion". Or the subject of the message.
- * @param {string} path - The path which is affected by the message (e.g.: "/attributes"), or the destination
- * of a message (e.g.: "inbox"|"outbox").
- * @param {Object.} dittoHeaders - The headers Object containing all Ditto Protocol header values.
- * @param {*} [value] - The value to apply / which was applied (e.g. in a "modify" action).
- * @param {number} [status] - The status code that indicates the result of the command. If setting a status code,
- * the Ditto Protocol Message will be interpreted as a response (e.g. content will be ignored when using 204).
- * @param {Object} [extra] - The enriched extra fields when selected via "extraFields" option.
- * @returns {DittoProtocolMessage} dittoProtocolMessage(s) -
- * The mapped Ditto Protocol message or
- * null if the message could/should not be mapped
- */
-function buildDittoProtocolMsg(namespace, name, group, channel, criterion, action, path, dittoHeaders, value, status, extra) {
+// Builds a Ditto Protocol message from the passed parameters.
+function buildDittoProtocolMsg(namespace, name, group, channel,
+ criterion, action, path, dittoHeaders, value, status, extra) {
const topic = buildTopic(namespace, name, group, channel, criterion, action);
return {
@@ -669,34 +547,14 @@ function buildDittoProtocolMsg(namespace, name, group, channel, criterion, actio
};
}
-/**
- * Builds a Ditto Protocol topic from the passed parameters.
- * @param {string} namespace - The namespace of the entity in java package notation, e.g.: "org.eclipse.ditto". Or "_"
- * (underscore) for connection announcements.
- * @param {string} name - The name of the entity, e.g.: "device".
- * @param {string} channel - The channel for the signal: "twin"|"live"|"none"
- * @param {string} group - The affected group/entity: "things"|"policies"|"connections".
- * @param {string} criterion - The criterion to apply: "commands"|"events"|"search"|"messages"|"announcements"|"errors".
- * @param {string} action - The action to perform: "create"|"retrieve"|"modify"|"delete". Or the announcement name:
- * "opened"|"closed"|"subjectDeletion". Or the subject of the message.
- * @returns {string} topic - the topic.
- */
+// Builds a Ditto Protocol topic string.
function buildTopic(namespace, name, group, channel, criterion, action) {
const topicChannel = 'none' === channel ? '' : '/' + channel;
return namespace + "/" + name + "/" + group + topicChannel + "/" + criterion + "/" + action;
}
-/**
- * Builds an external message from the passed parameters.
- * @param {Object.} headers - The external headers Object containing header values
- * @param {string} [textPayload] - The external mapped String
- * @param {ArrayBuffer} [bytePayload] - The external mapped bytes as ArrayBuffer
- * @param {string} [contentType] - The returned Content-Type
- * @returns {ExternalMessage} externalMessage -
- * the mapped external message
- * or null if the message could/should not be mapped
- */
+// Builds an external message from the passed parameters.
function buildExternalMsg(headers, textPayload, bytePayload, contentType) {
return {
@@ -707,24 +565,13 @@ function buildExternalMsg(headers, textPayload, bytePayload, contentType) {
};
}
-/**
- * Transforms the passed ArrayBuffer to a String interpreting the content of the passed arrayBuffer as unsigned 8
- * bit integers.
- *
- * @param {ArrayBuffer} arrayBuffer the ArrayBuffer to transform to a String
- * @returns {String} the transformed String
- */
+// Transforms an ArrayBuffer to a String (unsigned 8-bit integers).
function arrayBufferToString(arrayBuffer) {
return String.fromCharCode.apply(null, new Uint8Array(arrayBuffer));
}
-/**
- * Transforms the passed String to an ArrayBuffer using unsigned 8 bit integers.
- *
- * @param {String} string the String to transform to an ArrayBuffer
- * @returns {ArrayBuffer} the transformed ArrayBuffer
- */
+// Transforms a String to an ArrayBuffer (unsigned 8-bit integers).
function stringToArrayBuffer(string) {
let buf = new ArrayBuffer(string.length);
@@ -735,14 +582,9 @@ function stringToArrayBuffer(string) {
return buf;
}
-/**
- * Transforms the passed ArrayBuffer to a {ByteBuffer} (from bytebuffer.js library which needs to be loaded).
- *
- * @param {ArrayBuffer} arrayBuffer the ArrayBuffer to transform
- * @returns {ByteBuffer} the transformed ByteBuffer
- */
+// Transforms an ArrayBuffer to a ByteBuffer (requires bytebuffer.js).
function asByteBuffer(arrayBuffer) {
-
+
let byteBuffer = new ArrayBuffer(arrayBuffer.byteLength);
new Uint8Array(byteBuffer).set(new Uint8Array(arrayBuffer));
return dcodeIO.ByteBuffer.wrap(byteBuffer);
@@ -751,20 +593,9 @@ function asByteBuffer(arrayBuffer) {
### Mapping incoming messages
-Incoming external messages can be mapped to Ditto Protocol conform messages by implementing the following JavaScript function:
+Implement `mapToDittoProtocolMsg` to convert external payloads to Ditto Protocol:
```javascript
-/**
- * Maps the passed parameters to a Ditto Protocol message.
- * @param {Object.} headers - The headers Object containing all received header values
- * @param {string} [textPayload] - The String to be mapped
- * @param {ArrayBuffer} [bytePayload] - The bytes to be mapped as ArrayBuffer
- * @param {string} [contentType] - The received Content-Type, e.g. "application/json"
- * @returns {(DittoProtocolMessage|Array)} dittoProtocolMessage(s) -
- * the mapped Ditto Protocol message,
- * an array of Ditto Protocol messages or
- * null if the message could/should not be mapped
- */
function mapToDittoProtocolMsg(
headers,
textPayload,
@@ -772,25 +603,23 @@ function mapToDittoProtocolMsg(
contentType
) {
- // ### Insert/adapt your mapping logic here.
- // Use helper function Ditto.buildDittoProtocolMsg to build Ditto protocol message
- // based on incoming payload.
- // See https://websites.eclipseprojects.io/ditto/connectivity-mapping.html#helper-functions for details.
- // ### example code assuming the Ditto protocol content type for incoming messages.
+ // Insert/adapt your mapping logic here.
+ // Use Ditto.buildDittoProtocolMsg to build Ditto Protocol messages from incoming payload.
if (contentType === 'application/vnd.eclipse.ditto+json') {
- // Message is sent as Ditto protocol text payload and can be used directly
+ // Message is already in Ditto Protocol format -- use directly
return JSON.parse(textPayload);
} else if (contentType === 'application/octet-stream') {
- // Message is sent as binary payload; assume Ditto protocol message (JSON).
+ // Binary payload -- assume Ditto Protocol message (JSON)
try {
return JSON.parse(Ditto.arrayBufferToString(bytePayload));
} catch (e) {
- // parsing failed (no JSON document); return null to drop the message
+ // parsing failed (no JSON document); drop the message
return null;
}
} else if (contentType === 'application/json') {
let parsedJson = JSON.parse(textPayload);
- // the following variables would be determined from the "parsedJson" and from the "headers":
+ value = parsedJson.number1 + parsedJson['sub-field']; // access JSON keys with dashes using bracket notation
+ // determine these variables from parsedJson and headers:
let namespace = "";
let name = "";
let group = "things";
@@ -803,128 +632,56 @@ function mapToDittoProtocolMsg(
"a": 1
};
return Ditto.buildDittoProtocolMsg(
- namespace,
- name,
- group,
- channel,
- criterion,
- action,
- path,
- dittoHeaders,
+ namespace,
+ name,
+ group,
+ channel,
+ criterion,
+ action,
+ path,
+ dittoHeaders,
value)
}
- // no mapping logic matched; return null to drop the message
+ // no mapping logic matched; drop the message
return null;
}
```
-The result of the function has to be a JavaScript object in [Ditto Protocol](protocol-overview.html) or an array of
-such JavaScript objects. That's where the helper method `Ditto.buildDittoProtocolMsg` is useful:
-it explicitly defines which parameters are required for the Ditto Protocol message.
+Return a single Ditto Protocol message, an array of messages, or `null` to drop the message.
-There is another JavaScript function which is helpful when access to the complete external message is needed.
-It is possible to define the `mapToDittoProtocolMsgWrapper` in the incoming payload mapping and access the original
-`externalMsg`.
+For full access to the external message object, implement `mapToDittoProtocolMsgWrapper` instead:
-This is the default implementation of `mapToDittoProtocolMsgWrapper`, delegating to `mapToDittoProtocolMsg`:
```javascript
-/**
- * Maps the passed external message to a Ditto Protocol message.
- * @param {ExternalMessage} externalMsg - The external message to map to a Ditto Protocol message
- * @returns {(DittoProtocolMessage|Array)} dittoProtocolMessage(s) -
- * The mapped Ditto Protocol message,
- * an array of Ditto Protocol messages or
- * null if the message could/should not be mapped
- */
function mapToDittoProtocolMsgWrapper(externalMsg) {
-
let headers = externalMsg.headers;
let textPayload = externalMsg.textPayload;
let bytePayload = externalMsg.bytePayload;
let contentType = externalMsg.contentType;
-
return mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType);
}
```
### Mapping outgoing messages
-Outgoing Ditto Protocol messages (e.g. [responses](basic-signals-commandresponse.html) or [events](basic-signals-event.html))
-can be mapped to external messages by implementing the following JavaScript function:
+Implement `mapFromDittoProtocolMsg` to convert Ditto Protocol messages to external format:
```javascript
-/**
- * Maps the passed parameters which originated from a Ditto Protocol message to an external message.
- * @param {string} namespace - The namespace of the entity in java package notation, e.g.: "org.eclipse.ditto". Or "_"
- * (underscore) for connection announcements.
- * @param {string} name - The name of the entity, e.g.: "device".
- * @param {string} group - The affected group/entity: "things"|"policies"|"connections".
- * @param {string} channel - The channel for the signal: "twin"|"live"|"none"
- * @param {string} criterion - The criterion to apply: "commands"|"events"|"search"|"messages"|"announcements"|
- * "errors".
- * @param {string} action - The action to perform: "create"|"retrieve"|"modify"|"delete". Or the announcement name:
- * "opened"|"closed"|"subjectDeletion". Or the subject of the message.
- * @param {string} path - The path which is affected by the message (e.g.: "/attributes"), or the destination
- * of a message (e.g.: "inbox"|"outbox").
- * @param {Object.} dittoHeaders - The headers Object containing all Ditto Protocol header values.
- * @param {*} [value] - The value to apply / which was applied (e.g. in a "modify" action).
- * @param {number} [status] - The status code that indicates the result of the command. When this field is set,
- * it indicates that the Ditto Protocol Message contains a response.
- * @param {Object} [extra] - The enriched extra fields when selected via "extraFields" option.
- * @returns {(ExternalMessage|Array)} externalMessage - The mapped external message, an array of
- * external messages or null if the message could/should not be mapped.
- */
-function mapFromDittoProtocolMsg(
- namespace,
- name,
- group,
- channel,
- criterion,
- action,
- path,
- dittoHeaders,
- value,
- status,
- extra
-) {
-
- // ###
- // Insert your mapping logic here
- // ### example code using the Ditto protocol content type.
+function mapFromDittoProtocolMsg(namespace, name, group, channel,
+ criterion, action, path, dittoHeaders, value, status, extra) {
let headers = dittoHeaders;
- let textPayload = JSON.stringify(Ditto.buildDittoProtocolMsg(namespace, name, group, channel, criterion, action,
- path, dittoHeaders, value, status, extra));
- let bytePayload = null;
- let contentType = 'application/vnd.eclipse.ditto+json';
- return Ditto.buildExternalMsg(
- headers, // The external headers Object containing header values
- textPayload, // The external mapped String
- bytePayload, // The external mapped byte[]
- contentType // The returned Content-Type
+ let textPayload = JSON.stringify(
+ Ditto.buildDittoProtocolMsg(namespace, name, group, channel,
+ criterion, action, path, dittoHeaders, value, status, extra)
);
+ return Ditto.buildExternalMsg(headers, textPayload, null,
+ 'application/vnd.eclipse.ditto+json');
}
```
-The result of the function has to be a JavaScript object or, an array of JavaScript objects with the fields `headers`,
-`textPayload`, `bytePayload` and `contentType`. That's where the helper method `Ditto.buildExternalMsg` is useful:
-it explicitly defines which parameters are required for the external message.
-
-There is another JavaScript function which is helpful when access to the complete Ditto protocol message is needed.
-It is possible to define the `mapFromDittoProtocolMsgWrapper` in the outgoing payload mapping and access the
-original `dittoProtocolMsg`.
-Please refer to the [Ditto Protocol specification](protocol-specification.html#dittoProtocolEnvelope)
-to inspect which JSON fields are available when.
+For access to the full Ditto Protocol message (including `revision`), implement
+`mapFromDittoProtocolMsgWrapper`:
-This is the default implementation of `mapFromDittoProtocolMsgWrapper`, delegating to `mapFromDittoProtocolMsg`:
```javascript
-/**
- * Maps the passed Ditto Protocol message to an external message.
- * @param {DittoProtocolMessage} dittoProtocolMsg - The Ditto Protocol message to map
- * @returns {(ExternalMessage|Array)} externalMessage -
- * The mapped external message,
- * an array of external messages or
- * null if the message could/should not be mapped
- */
function mapFromDittoProtocolMsgWrapper(dittoProtocolMsg) {
let topic = dittoProtocolMsg.topic;
@@ -957,67 +714,26 @@ function mapFromDittoProtocolMsgWrapper(dittoProtocolMsg) {
}
```
-## JavaScript payload types
-
-Both, text payloads and byte payloads may be mapped.
-
-### Text payloads
-
-Working with text payloads is as easy as it gets in JavaScript. For example, for the content-type `application/json`
-structured data may be processed like this:
-
-```javascript
-let value;
-if (contentType === 'application/json') {
- let parsedJson = JSON.parse(textPayload);
- value = parsedJson.number1 + parsedJson['sub-field']; // remember to access JSON keys with dashes in a JS special way
-} else {
- // a script may decide to not map other content-types than application/json
- return null;
-}
-// proceed ...
-```
-
-### Byte payloads
-
-Working with byte payloads is also possible but does require a little bit of knowledge about JavaScript's
-[ArrayBuffer](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer)
-[TypedArrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray) and
-[DataView](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView).
-
-What you get in the mapping scripts is a `bytePayload` of type `ArrayBuffer` which lets you work on the bytes
-in different ways:
+### Working with byte payloads
-#### Typed Arrays
-
-> A TypedArray \[is\] a view into an ArrayBuffer where every item has the same size and type. [source](https://hacks.mozilla.org/2017/01/typedarray-or-dataview-understanding-byte-order/)
-
-With TypedArrays you can simply wrap the `bytePayload` `ArrayBuffer` and work on all the items e.g.
-as unsigned 8-bit integers:
+Use [TypedArrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray)
+or [DataViews](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView)
+to process binary data:
```javascript
+// TypedArray approach
let bytes = new Uint8Array(bytePayload);
-bytes[0]; // access the first byte
-bytes[1]; // access the second byte
-```
-
-#### DataViews
-> The DataView \[is\] another view into an ArrayBuffer, but one which allows items of different size and type in the ArrayBuffer. [source](https://hacks.mozilla.org/2017/01/typedarray-or-dataview-understanding-byte-order/)
-
-```javascript
+// DataView approach (mixed types)
let view = new DataView(bytePayload);
-view.getInt8(0); // access a 8-bit signed integer (byte) on offset=0
-view.getUint16(1); // access a 16-bit unsigned integer (usigned short) on offset=1
-```
-
-DataViews also allow to `set` bytes to an underlying ArrayBuffer conveniently.
-
-#### ByteBuffer.js
+view.getInt8(0); // 8-bit signed integer (byte) at offset 0
+view.getUint16(1); // 16-bit unsigned integer (unsigned short) at offset 1
+let temp = view.getInt16(0) / 100.0; // 16-bit signed int at offset 0
+let pressure = view.getInt16(2); // 16-bit signed int at offset 2
+let humidity = view.getUint8(4); // 8-bit unsigned int at offset 4
+```
-Alternatively, Ditto's JavaScript transformation may be loaded with the [above mentioned](#helper-libraries) libraries,
-e.g. "bytebuffer.js".
-With `ByteBuffer`, the content of an `ArrayBuffer` can be accessed in a buffered way:
+Or use `ByteBuffer.js` (load with `"loadBytebufferJS": "true"`):
```javascript
let byteBuf = Ditto.asByteBuffer(bytePayload);
@@ -1031,225 +747,88 @@ buf.readUTF8String(4); // read 4 characters of UTF-8 encoded string + advances t
buf.remaining(); // gets the number of remaining readable bytes in the buffer
```
-Check the [ByteBuffer API documentation](https://github.com/dcodeIO/bytebuffer.js/wiki/API) to find out what is possible
-with that helper.
+Check the [ByteBuffer API documentation](https://github.com/dcodeIO/bytebuffer.js/wiki/API) for the full list of operations.
-
-## JavaScript Examples
+## JavaScript examples
### Text payload example
-Let's assume your device sends telemetry data via [Eclipse Hono's](https://www.eclipse.org/hono/) MQTT adapter
-into the cloud. And, that an example payload of your device is:
+Device sends JSON telemetry:
```json
-{
- "temp": "23.42 °C",
- "hum": 78,
- "pres": {
- "value": 760,
- "unit": "mmHg"
- }
-}
-```
-
-We want to map a single message of this device containing updates for all 3 values to a Thing in the following structure:
-
-```json
-{
- "thingId": "the.namespace:the-thing-id",
- "policyId": "the.namespace:the-policy-id",
- "features": {
- "temperature": {
- "properties": {
- "value": 23.42
- }
- },
- "pressure": {
- "properties": {
- "value": 760
- }
- },
- "humidity": {
- "properties": {
- "value": 78
- }
- }
- }
-}
+{"temp": "23.42 °C", "hum": 78, "pres": {"value": 760, "unit": "mmHg"}}
```
-Therefore, we define following `incoming` mapping function:
+Mapping to update thing features:
```javascript
-function mapToDittoProtocolMsg(
- headers,
- textPayload,
- bytePayload,
- contentType
-) {
-
- if (contentType !== 'application/json') {
- return null; // only handle messages with content-type application/json
- }
-
- let jsonData = JSON.parse(textPayload);
-
- let value = {
- temperature: {
- properties: {
- value: jsonData.temp.split(" ")[0] // omit the unit
- }
- },
- pressure: {
- properties: {
- value: jsonData.pres.value
- }
- },
- humidity: {
- properties: {
- value: jsonData.hum
- }
- }
- };
+function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {
+ if (contentType !== 'application/json') return null;
+
+ let jsonData = JSON.parse(textPayload);
+ let value = {
+ temperature: { properties: { value: parseFloat(jsonData.temp.split(" ")[0]) } },
+ pressure: { properties: { value: jsonData.pres.value } },
+ humidity: { properties: { value: jsonData.hum } }
+ };
- return Ditto.buildDittoProtocolMsg(
- 'org.eclipse.ditto', // in this example always the same
- headers['device_id'], // Eclipse Hono sets the authenticated device_id as AMQP 1.0 header
- 'things', // we deal with a Thing
- 'twin', // we want to update the twin
- 'commands', // we want to create a command to update a twin
- 'modify', // modify the twin
- '/features', // modify all features at once
- headers, // pass through the headers from AMQP 1.0
- value
- );
+ return Ditto.buildDittoProtocolMsg(
+ 'org.eclipse.ditto', headers['device_id'],
+ 'things', 'twin', 'commands', 'modify',
+ '/features', headers, value
+ );
}
```
-When your device now sends its payload via the MQTT adapter of Eclipse Hono:
+Send this payload via Eclipse Hono's MQTT adapter:
```bash
mosquitto_pub -u 'sensor1@DEFAULT_TENANT' -P hono-secret -t telemetry -m '{"temp": "23.42 °C","hum": 78,"pres": {"value": 760,"unit": "mmHg"}}'
```
-Your digital twin is updated by applying the specified script and extracting the relevant values from the passed `textPayload`.
+The digital twin is updated by applying the script and extracting the relevant values from the `textPayload`.
+### Binary payload example
-### Bytes payload example
-
-For this example, let's assume your device sends telemetry data via [Eclipse Hono's](https://www.eclipse.org/hono/)
-HTTP adapter into the cloud. An example payload of your device - displayed as hexadecimal - is:
-
-```
-0x09EF03F72A
-```
-
-Let us now also assume that
-
-* the first 2 bytes `09 EF` represent
- * the temperature as 16bit signed integer (thus, may also be negative)
- * this is not a float in oder to save space (as float needs at least 32 bit)
-* the second 2 bytes `03 F7` represent the pressure as 16bit signed integer
-* the last byte `2A` represents the humidity as 8bit unsigned integer of our device.
-
-We want to map a single message of this device containing updates for all 3 values to a Thing in the following structure:
-
-```json
-{
- "thingId": "the.namespace:the-thing-id",
- "policyId": "the.namespace:the-policy-id",
- "features": {
- "temperature": {
- "properties": {
- "value": 25.43
- }
- },
- "pressure": {
- "properties": {
- "value": 1015
- }
- },
- "humidity": {
- "properties": {
- "value": 42
- }
- }
- }
-}
-```
+Device sends 5 bytes as hexadecimal `0x09EF03F72A`:
-Therefore, we define following `incoming` mapping function:
+* the first 2 bytes `09 EF` represent the temperature as 16-bit signed integer (not a float, to save space)
+* the next 2 bytes `03 F7` represent the pressure as 16-bit signed integer
+* the last byte `2A` represents the humidity as 8-bit unsigned integer
```javascript
-function mapToDittoProtocolMsg(
- headers,
- textPayload,
- bytePayload,
- contentType
-) {
-
- if (contentType !== 'application/octet-stream') {
- return null; // only handle messages with content-type application/octet-stream
- }
-
- let view = new DataView(bytePayload);
-
- let value = {
- temperature: {
- properties: {
- // interpret the first 2 bytes (16 bit) as signed int and divide through 100.0:
- value: view.getInt16(0) / 100.0
- }
- },
- pressure: {
- properties: {
- // interpret the next 2 bytes (16 bit) as signed int:
- value: view.getInt16(2)
- }
- },
- humidity: {
- properties: {
- // interpret the next 1 bytes (8 bit) as unsigned int:
- value: view.getUint8(4)
- }
- }
- };
+function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {
+ if (contentType !== 'application/octet-stream') return null;
+
+ let view = new DataView(bytePayload);
+ let value = {
+ temperature: { properties: { value: view.getInt16(0) / 100.0 } },
+ pressure: { properties: { value: view.getInt16(2) } },
+ humidity: { properties: { value: view.getUint8(4) } }
+ };
- return Ditto.buildDittoProtocolMsg(
- 'org.eclipse.ditto', // in this example always the same
- headers['device_id'], // Eclipse Hono sets the authenticated device_id as AMQP 1.0 header
- 'things', // we deal with a Thing
- 'twin', // we want to update the twin
- 'commands', // we want to create a command to update a twin
- 'modify', // modify the twin
- '/features', // modify all features at once
- headers, // pass through the headers from AMQP 1.0
- value
- );
+ return Ditto.buildDittoProtocolMsg(
+ 'org.eclipse.ditto', headers['device_id'],
+ 'things', 'twin', 'commands', 'modify',
+ '/features', headers, value
+ );
}
```
-When your device now sends its payload via the HTTP adapter of Eclipse Hono:
+Send this payload via Eclipse Hono's HTTP adapter:
```bash
echo -e $((0x09EF03F72A)) | curl -i -X POST -u sensor1@DEFAULT_TENANT:hono-secret -H 'Content-Type: application/octet-stream' --data-binary @- http://127.0.0.1:8080/telemetry
```
-Your digital twin is updated by applying the specified script and extracting the relevant values from the passed `bytePayload`.
-
+The digital twin is updated by applying the script and extracting the relevant values from the `bytePayload`.
-## Custom Java based implementation
+## Custom Java mapper
-Beside the JavaScript based mapping - which can be configured/changed at runtime without the need of restarting the
-connectivity service - there is also the possibility to implement a custom Java based mapper.
+For advanced use cases, implement a custom Java-based mapper by extending
+[`AbstractMessageMapper`](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/mapping/AbstractMessageMapper.java).
-The interface to be implemented is
-[`MessageMapper`](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/mapping/MessageMapper.java))
-and there is an abstract class [`AbstractMessageMapper`](https://github.com/eclipse-ditto/ditto/blob/master/connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/mapping/AbstractMessageMapper.java)
-which eases implementation of a custom mapper.
-
-Simply extend from `AbstractMessageMapper` to provide a custom mapper:
+Extend `AbstractMessageMapper` to provide a custom mapper:
```java
public final class FooMapper extends AbstractMessageMapper {
@@ -1304,41 +883,28 @@ public final class FooMapper extends AbstractMessageMapper {
}
```
-After instantiation of the custom `MessageMapper`, the `doConfigure` method is called with all the *options* which were
-provided to the mapper in the [configured connection](connectivity-manage-connections.html#create-connection).
-Use them in order to pass in configurations, thresholds, etc.
-
-Then, simply implement both of the `map` methods:
-
-* `List map(ExternalMessage message)` maps from an incoming external message to
- * an empty list of `Adaptable`s if the incoming message should be dropped
- * a list of one or many [Ditto Protocol](protocol-overview.html) `Adaptable`s
-* `List map(Adaptable adaptable)` maps from an outgoing [Ditto Protocol](protocol-overview.html) `Adaptable` to
- * an empty list of `ExternalMessage`s if the outgoing message should be dropped
- * a list of one or many external messages
-
-In order to use this custom Java based mapper implementation, the following steps are required:
-
-* the alias has to be defined via the implemented `getAlias()` method - it must be unique and *should* start with an uppercase letter
-* if the custom mapper requires mandatory options then implement `isConfigurationMandatory()` to return `true`
-* the mapper class needs to be on the classpath of the [connectivity](architecture-services-connectivity.html)
- microservice in order to be loaded.
- Follow the instructions of
- [how to extend Ditto](installation-extending.html#providing-additional-functionality-by-adding-jars-to-the-classpath)
- to achieve that.
-* the mapper needs to be registered via configuration in the connectivity service,
- [extend the configuration](installation-extending.html#adjusting-configuration-of-ditto) or add the mapper via
- [system properties](installation-operating.html#ditto-configuration) configuration
-* when creating a new connection you have to specify the alias of your mapper as the `mappingEngine` in the
- connection's `mappingDefinitions` and reference the ID of your mapper in a source or a target
+Key methods to implement:
+
+* `List map(ExternalMessage message)` -- inbound mapping (return empty list to drop)
+* `List map(Adaptable adaptable)` -- outbound mapping (return empty list to drop)
+* `String getAlias()` -- unique mapper alias (must start with uppercase)
+
+To deploy:
+
+1. Add the mapper JAR to the connectivity service classpath
+ ([extending Ditto](installation-extending.html#adding-jars-to-the-classpath))
+2. Register the alias in [connectivity configuration](installation-extending.html#adjusting-service-configuration)
+3. Reference the alias in your connection's `mappingDefinitions`
{% include tip.html content="If your mapper does not require any options (`isConfigurationMandatory() = true`), you can
- directly reference the alias in a source or a target without first defining it inside `mappingDefinitions`." %}
+ directly reference the alias in a source or a target without first defining it inside `mappingDefinitions`." %}
-### Example for Custom Java based mapper
+For a complete example, see the
+[custom-ditto-java-payload-mapper](https://github.com/eclipse-ditto/ditto-examples/tree/master/custom-ditto-java-payload-mapper)
+project.
-Please have a look at the following Ditto example project:
-* [custom-ditto-java-payload-mapper](https://github.com/eclipse-ditto/ditto-examples/tree/master/custom-ditto-java-payload-mapper)
+## Further reading
-This shows how to implement, add and configure a custom, Protobuf based, Java payload mapper for Ditto to use in the
-connectivity service for mapping a custom domain specific Protbuf encoded payload.
+* [Connections overview](basic-connections.html) -- connection model and configuration
+* [Header mapping](connectivity-header-mapping.html) -- map external headers
+* [Ditto Protocol](protocol-overview.html) -- message format specification
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-overview.md b/documentation/src/main/resources/pages/ditto/connectivity-overview.md
index b74f219c4d1..3d8da61217e 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-overview.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-overview.md
@@ -1,28 +1,18 @@
---
-title: Connectivity API overview
-keywords:
+title: Connectivity overview
+keywords: connectivity, connections, overview, AMQP, MQTT, HTTP, Kafka, Hono
tags: [connectivity]
permalink: connectivity-overview.html
---
-The Connectivity API is a bare management API for Ditto's [Connectivity Service](architecture-services-connectivity.html).
-It is available:
-* via [DevOps Commands](installation-operating.html#devops-commands)
-* via HTTP API with a specific authentication - for details see [Manage connections](connectivity-manage-connections.html)
+Ditto's Connectivity service lets you integrate with external messaging systems and backends using protocols like AMQP, MQTT, HTTP, and Kafka.
-Use it to manage client [connections](basic-connections.html) to remote systems and to exchange
-[Ditto Protocol](protocol-specification.html) messages with those.
-If a remote system is unable to send messages in the necessary format, there is the option
-to configure custom [payload mapping logic](connectivity-mapping.html) to adapt to almost any message format and
-encoding.
+For the connection data model and core concepts, see [Connections](basic-connections.html).
-The following connection types are supported:
-
-
-* [AMQP 0.9.1](connectivity-protocol-bindings-amqp091.html)
-* [AMQP 1.0](connectivity-protocol-bindings-amqp10.html)
-* [MQTT 3.1.1](connectivity-protocol-bindings-mqtt.html)
-* [MQTT 5](connectivity-protocol-bindings-mqtt5.html)
-* [HTTP 1.1](connectivity-protocol-bindings-http.html)
-* [Kafka 2.x](connectivity-protocol-bindings-kafka2.html)
+This section covers the operational aspects of connectivity:
+* **[Manage Connections](connectivity-manage-connections.html)** — Create, modify, retrieve, and delete connections via the HTTP API
+* **[Piggyback Commands](connectivity-manage-connections-piggyback.html)** — Manage connections via DevOps commands
+* **Protocol Bindings** — Protocol-specific configuration for [AMQP 0.9.1](connectivity-protocol-bindings-amqp091.html), [AMQP 1.0](connectivity-protocol-bindings-amqp10.html), [MQTT 3.1.1](connectivity-protocol-bindings-mqtt.html), [MQTT 5](connectivity-protocol-bindings-mqtt5.html), [HTTP 1.1](connectivity-protocol-bindings-http.html), [Kafka 2.x](connectivity-protocol-bindings-kafka2.html), and [Eclipse Hono](connectivity-protocol-bindings-hono.html)
+* **Data Transformation** — [Payload mapping](connectivity-mapping.html) and [header mapping](connectivity-header-mapping.html)
+* **Security** — [TLS certificates](connectivity-tls-certificates.html), [SSH tunneling](connectivity-ssh-tunneling.html), and [HMAC signing](connectivity-hmac-signing.html)
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-amqp091.md b/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-amqp091.md
index 8d6c996cc98..c6d5cdcee45 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-amqp091.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-amqp091.md
@@ -5,84 +5,67 @@ tags: [protocol, connectivity, rql]
permalink: connectivity-protocol-bindings-amqp091.html
---
-Consume messages from AMQP 0.9.1 brokers via [sources](#source-format) and send messages to AMQP 0.9.1 brokers via
-[targets](#target-format).
+You use the AMQP 0.9.1 binding to connect Ditto with message brokers like RabbitMQ for consuming and publishing messages.
-## Content-type
+{% include callout.html content="**TL;DR**: Configure an AMQP 0.9.1 connection with `connectionType: \"amqp-091\"`. Source addresses are queue names, and target addresses use the format `exchange_name/routing_key`." type="primary" %}
-When messages are sent in [Ditto Protocol](protocol-overview.html) (as `UTF-8` encoded String payload),
-the `content-type` of AMQP 0.9.1 messages must be set to:
+## Overview
+
+The AMQP 0.9.1 protocol binding lets you consume messages from AMQP 0.9.1 brokers via
+[sources](#source-configuration) and publish messages via [targets](#target-configuration).
+
+When you send messages in [Ditto Protocol](protocol-overview.html) format (`UTF-8` encoded strings),
+set the `content-type` to:
```
application/vnd.eclipse.ditto+json
```
-If messages, which are not in Ditto Protocol, should be processed, a [payload mapping](connectivity-mapping.html) must
-be configured for the AMQP 0.9.1 connection in order to transform the messages.
+For other payload formats, configure a [payload mapping](connectivity-mapping.html).
-## AMQP 0.9.1 properties
+### AMQP 0.9.1 properties
-Supported AMQP 0.9.1 properties which are interpreted in a specific way are:
+Ditto interprets these AMQP 0.9.1 properties:
-* `content-type`: for defining the Ditto Protocol content-type
-* `correlation-id`: for correlating request messages to responses
+* `content-type` -- defines the Ditto Protocol content type
+* `correlation-id` -- correlates request messages to responses
-## Specific connection configuration
+## Connection URI format
-The common configuration for connections in [Connections > Sources](basic-connections.html#sources) and
-[Connections > Targets](basic-connections.html#targets) applies here as well.
-Following are some specifics for AMQP 0.9.1 connections:
+```
+amqp://user:password@hostname:5672/vhost
+```
-### Source format
+Use `amqps://` for TLS-secured connections.
-An AMQP 0.9.1 connection requires the protocol configuration source object to have an `addresses` property with a list
-of queue names, and `authorizationContext` array that contains the authorization subjects, in whose context
-incoming messages are processed. These subjects may contain placeholders, see
-[placeholders](basic-connections.html#placeholder-for-source-authorization-subjects) section for more information.
+## Source configuration
+The common [source configuration](basic-connections.html#sources) applies. Source `addresses` are
+AMQP 0.9.1 queue names:
```json
{
- "addresses": [
- "",
- "..."
- ],
- "authorizationContext": ["ditto:inbound-auth-subject", "..."]
+ "addresses": ["queueName"],
+ "authorizationContext": ["ditto:inbound-auth-subject"]
}
```
-#### Source acknowledgement handling
-
-For AMQP 0.9.1 sources, when configuring
-[acknowledgement requests](basic-connections.html#source-acknowledgement-requests), consumed messages from the AMQP 0.9.1
-broker are treated in the following way:
-
-For Ditto acknowledgements with successful [status](protocol-specification-acks.html#combined-status-code):
-* Acknowledges a single AMQP 0.9.1 message with an `Ack` message for the received `deliveryTag`
+### Source acknowledgement handling
-For Ditto acknowledgements with mixed successful/failed [status](protocol-specification-acks.html#combined-status-code):
-* If some of the aggregated [acknowledgements](basic-acknowledgements.html#acknowledgements-acks) require redelivery (e.g. based on a timeout):
- * Negatively acknowledges the AMQP 0.9.1 message with a `Nack` message for the received `deliveryTag` and setting `requeue` to `true`
-* If none of the aggregated [acknowledgements](basic-acknowledgements.html#acknowledgements-acks) require redelivery:
- * Negatively acknowledges the AMQP 0.9.1 message with a `Nack` message for the received `deliveryTag` and setting `requeue` to `false` preventing redelivery by the AMQP 0.9.1 broker
+When you configure [acknowledgement requests](basic-connections.html#source-acknowledgement-requests):
+* **Successful acknowledgements** -- Ditto sends an `Ack` for the received `deliveryTag`
+* **Failed with redelivery needed** -- Ditto sends a `Nack` with `requeue: true`
+* **Failed without redelivery** -- Ditto sends a `Nack` with `requeue: false`
-### Target format
-
-An AMQP 0.9.1 connection requires the protocol configuration target object to have an `address` property with a combined
-value of the `exchange_name` and `routing_key`. The target address may contain placeholders; see
-[placeholders](basic-connections.html#placeholder-for-target-addresses) section for more information.
-
-Further, `"topics"` is a list of strings, each list entry representing a subscription of
-[Ditto protocol topics](protocol-specification-topic.html).
-
-Outbound messages are published to the configured target address if one of the subjects in `"authorizationContext"`
-has READ permission on the thing, which is associated with a message.
+## Target configuration
+The common [target configuration](basic-connections.html#targets) applies. The target `address`
+combines the exchange name and routing key:
```json
{
- "address": "/",
+ "address": "exchangeName/routingKey",
"topics": [
"_/_/things/twin/events",
"_/_/things/live/messages"
@@ -91,62 +74,49 @@ has READ permission on the thing, which is associated with a message.
}
```
-#### Target acknowledgement handling
-
-For AMQP 0.9.1 targets, when configuring
-[automatically issued acknowledgement labels](basic-connections.html#target-issued-acknowledgement-label), requested
-acknowledgements are produced in the following way:
+The target address supports [placeholders](basic-connections.html#placeholder-for-target-addresses).
-Once the AMQP 0.9.1 client signals that the message was acknowledged by the AMQP 0.9.1 broker, the following information
-is mapped to the automatically created [acknowledgement](protocol-specification-acks.html#acknowledgement):
-* Acknowledgement.status:
- * will be `200`, if the message was successfully ACKed by the AMQP 0.9.1 broker
- * will be `400`, if the AMQP 0.9.1 broker does not support publisher confirms
- * will be `503`, if the AMQP 0.9.1 broker negatively confirmed receiving a message
-* Acknowledgement.value:
- * will be missing, for status `200`
- * will contain more information, in case that an error `status` was set
+### Target acknowledgement handling
-### Specific configuration properties
+When you configure [issued acknowledgement labels](basic-connections.html#target-issued-acknowledgement-label):
-There are no specific configuration properties available for this type of connection.
+| Status | Condition |
+|--------|-----------|
+| `200` | Message successfully ACKed by the broker |
+| `400` | Broker does not support publisher confirms |
+| `503` | Broker negatively confirmed the message |
-## Establishing connecting to an AMQP 0.9.1 endpoint
+## Specific configuration options
-Ditto's [Connectivity service](architecture-services-connectivity.html) is responsible for creating new and managing
-existing connections.
+There are no AMQP 0.9.1-specific configuration properties.
-This can be done dynamically at runtime without the need to restart any microservice using a
-[Ditto DevOps command](installation-operating.html#devops-commands).
-
-Example connection configuration to create a new AMQP 0.9.1 connection (e.g. in order to connect to a RabbitMQ):
+## Example connection JSON
```json
{
- "connection": {
- "id": "rabbit-example-connection-123",
- "connectionType": "amqp-091",
- "connectionStatus": "open",
- "failoverEnabled": true,
- "uri": "amqp://user:password@localhost:5672/vhost",
- "sources": [
- {
- "addresses": [
- "queueName"
- ],
- "authorizationContext": ["ditto:inbound-auth-subject", "..."]
- }
+ "id": "rabbit-example-connection-123",
+ "connectionType": "amqp-091",
+ "connectionStatus": "open",
+ "failoverEnabled": true,
+ "uri": "amqp://user:password@localhost:5672/vhost",
+ "sources": [{
+ "addresses": ["queueName"],
+ "authorizationContext": ["ditto:inbound-auth-subject"]
+ }],
+ "targets": [{
+ "address": "exchangeName/routingKey",
+ "topics": [
+ "_/_/things/twin/events",
+ "_/_/things/live/messages"
],
- "targets": [
- {
- "address": "exchangeName/routingKey",
- "topics": [
- "_/_/things/twin/events",
- "_/_/things/live/messages"
- ],
- "authorizationContext": ["ditto:outbound-auth-subject", "..."]
- }
- ]
- }
+ "authorizationContext": ["ditto:outbound-auth-subject"]
+ }]
}
```
+
+## Further reading
+
+* [Connections overview](basic-connections.html) -- connection model and configuration
+* [Payload mapping](connectivity-mapping.html) -- transform message payloads
+* [Header mapping](connectivity-header-mapping.html) -- map external headers
+* [TLS certificates](connectivity-tls-certificates.html) -- secure connections with TLS
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-amqp10.md b/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-amqp10.md
index 89a2857bb70..c0cd9813431 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-amqp10.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-amqp10.md
@@ -5,44 +5,36 @@ tags: [protocol, connectivity, rql]
permalink: connectivity-protocol-bindings-amqp10.html
---
-Consume messages from AMQP 1.0 endpoints via [sources](#source-format) and send messages to AMQP 1.0 endpoints via
-[targets](#target-format).
+You use the AMQP 1.0 binding to connect Ditto with AMQP 1.0 endpoints such as Eclipse Hono or Azure Service Bus.
-## Content-type
+{% include callout.html content="**TL;DR**: Configure an AMQP 1.0 connection with `connectionType: \"amqp-10\"`. Source addresses are AMQP link names, and target addresses can be queues (`queue://`) or topics (`topic://`)." type="primary" %}
-When messages are sent in [Ditto Protocol](protocol-overview.html) (as `UTF-8` encoded String payload),
-the `content-type` of AMQP 1.0 messages must be set to:
+## Overview
+
+The AMQP 1.0 protocol binding lets you consume messages from AMQP 1.0 endpoints via
+[sources](#source-configuration) and publish messages via [targets](#target-configuration).
+
+When you send messages in [Ditto Protocol](protocol-overview.html) format (`UTF-8` encoded strings),
+set the `content-type` to:
```
application/vnd.eclipse.ditto+json
```
-If messages, which are not in Ditto Protocol, should be processed, a [payload mapping](connectivity-mapping.html) must
-be configured for the AMQP 1.0 connection in order to transform the messages.
-
-## AMQP 1.0 properties, application properties and message annotations
-
-When set as external headers by outgoing payload or header mapping, the properties defined by AMQP 1.0 specification are
-set to the corresponding header value. Conversely, the values of AMQP 1.0 properties are available for incoming payload
-and header mapping as external headers. The supported AMQP 1.0 properties are:
-
-* `message-id`
-* `user-id`
-* `to`
-* `subject`
-* `reply-to`
-* `correlation-id`
-* `content-type`
-* `absolute-expiry-time`
-* `creation-time`
-* `group-id`
-* `group-sequence`
-* `reply-to-group-id`
-
-External headers not on this list are mapped to AMQP application properties.
-To set an application property whose name is identical to an AMQP 1.0 property, prefix it by
-`amqp.application.property:`. The following [target header mapping](basic-connections.html#target-header-mapping) sets
-the application property `to` to the value of the Ditto protocol header `reply-to`:
+For other payload formats, configure a [payload mapping](connectivity-mapping.html).
+
+### AMQP 1.0 properties and headers
+
+Ditto maps these standard AMQP 1.0 properties:
+
+`message-id`, `user-id`, `to`, `subject`, `reply-to`, `correlation-id`, `content-type`,
+`absolute-expiry-time`, `creation-time`, `group-id`, `group-sequence`, `reply-to-group-id`
+
+Headers not in this list are mapped as AMQP application properties. To set an application property
+whose name collides with a standard property, prefix it with `amqp.application.property:`.
+
+**Target** -- set the application property `to` to the Ditto protocol header `reply-to`:
+
```json
{
"headerMapping": {
@@ -51,9 +43,8 @@ the application property `to` to the value of the Ditto protocol header `reply-t
}
```
-To read an application property whose name is identical to an AMQP 1.0 property, prefix it by
-`amqp.application.property:`. The following [source header mapping](basic-connections.html#source-header-mapping) sets
-the Ditto protocol header `reply-to` to the value of the application property `to`:
+**Source** -- read the application property `to` into the Ditto protocol header `reply-to`:
+
```json
{
"headerMapping": {
@@ -62,9 +53,10 @@ the Ditto protocol header `reply-to` to the value of the application property `t
}
```
-To set a message annotation, prefix it by `amqp.message.annotation:`.
-The following [target header mapping](basic-connections.html#target-header-mapping) sets
-the message annotation `to` to the value of the Ditto protocol header `reply-to`:
+To set or read a message annotation, prefix with `amqp.message.annotation:`.
+
+**Target** -- set the message annotation `to`:
+
```json
{
"headerMapping": {
@@ -73,9 +65,8 @@ the message annotation `to` to the value of the Ditto protocol header `reply-to`
}
```
-To read a message annotation, prefix it by `amqp.message.annotation:`.
-The following [source header mapping](basic-connections.html#source-header-mapping) sets
-the Ditto protocol header `reply-to` to the value of the message annotation `to`:
+**Source** -- read the message annotation `to`:
+
```json
{
"headerMapping": {
@@ -86,116 +77,76 @@ the Ditto protocol header `reply-to` to the value of the message annotation `to`
{% include note.html content="For now, setting or reading the AMQP 1.0 property 'content-encoding' is impossible." %}
-## Specific connection configuration
+## Connection URI format
+
+```
+amqps://user:password@hostname:5671
+```
-The common configuration for connections in [Connections > Sources](basic-connections.html#sources) and
-[Connections > Targets](basic-connections.html#targets) applies here as well.
-Following are some specifics for AMQP 1.0 connections:
+Use `amqp://` for unencrypted connections (port 5672 typically).
-### Source format
+## Source configuration
-Any `source` item defines an `addresses` array of source identifiers (e.g. Eclipse Hono's
-Telemetry API) to consume messages from,
-and `authorizationContext` array that contains the authorization subjects, in whose context
-inbound messages are processed. These subjects may contain placeholders, see
-[placeholders](basic-connections.html#placeholder-for-source-authorization-subjects) section for more information.
+The common [source configuration](basic-connections.html#sources) applies. Source `addresses` are
+AMQP 1.0 link names:
```json
{
- "addresses": [
- "",
- "..."
- ],
- "authorizationContext": ["ditto:inbound-auth-subject", "..."]
+ "addresses": ["telemetry/FOO"],
+ "authorizationContext": ["ditto:inbound-auth-subject"]
}
```
-#### Source acknowledgement handling
-
-For AMQP 1.0 sources, when configuring
-[acknowledgement requests](basic-connections.html#source-acknowledgement-requests), consumed messages from the AMQP 1.0
-endpoint are treated in the following way:
-
-For Ditto acknowledgements with successful [status](protocol-specification-acks.html#combined-status-code):
-* Acknowledges the AMQP 1.0 message with `accepted` outcome (see [AMQP 1.0 spec: 3.4.2 Accepted](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-accepted))
+### Source acknowledgement handling
-For Ditto acknowledgements with mixed successful/failed [status](protocol-specification-acks.html#combined-status-code):
-* If some of the aggregated [acknowledgements](basic-acknowledgements.html#acknowledgements-acks) require redelivery (e.g. based on a timeout):
- * Negatively acknowledges the AMQP 1.0 message with `modified[delivery-failed]` outcome (see [AMQP 1.0 spec: 3.4.5 Modified](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-modified))
-* If none of the aggregated [acknowledgements](basic-acknowledgements.html#acknowledgements-acks) require redelivery:
- * Negatively acknowledges the AMQP 1.0 message with `rejected` outcome (see [AMQP 1.0 spec: 3.4.3 Rejected](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-rejected)) preventing redelivery by the AMQP 1.0 endpoint
+When you configure [acknowledgement requests](basic-connections.html#source-acknowledgement-requests):
-### Target format
+* **Successful** -- message acknowledged with `accepted` outcome
+* **Failed with redelivery needed** -- message rejected with `modified[delivery-failed]` outcome
+* **Failed without redelivery** -- message rejected with `rejected` outcome
-An AMQP 1.0 connection requires the protocol configuration target object to have an `address` property with a source
-identifier. The target address may contain placeholders; see
-[placeholders](basic-connections.html#placeholder-for-target-addresses) section for more
-information.
+## Target configuration
-Target addresses for AMQP 1.0 are by default handled as AMQP 1.0 "queues". There is however the possibility to also
-configure AMQP 1.0 "topics" as well. In fact, the following formats for the `address` are
-supported:
-* `the-queue-name` (when configuring w/o prefix, the `address` is handled as "queue")
-* `queue://the-queue-name`
-* `topic://the-topic-name`
+The common [target configuration](basic-connections.html#targets) applies. Target addresses support
+these formats:
-Further, `"topics"` is a list of strings, each list entry representing a subscription of
-[Ditto protocol topics](protocol-specification-topic.html), see
-[target topics and filtering](basic-connections.html#target-topics-and-filtering) for more information on that.
-
-Outbound messages are published to the configured target address if one of the subjects in `"authorizationContext"`
-has READ permission on the thing, which is associated with a message.
+* `the-queue-name` -- handled as a queue (default)
+* `queue://the-queue-name` -- explicit queue
+* `topic://the-topic-name` -- AMQP 1.0 topic
```json
{
- "address": "",
+ "address": "events/twin",
"topics": [
"_/_/things/twin/events",
"_/_/things/live/messages"
],
- "authorizationContext": ["ditto:outbound-auth-subject", "..."]
+ "authorizationContext": ["ditto:outbound-auth-subject"]
}
```
-#### Target acknowledgement handling
-
-For AMQP 1.0 targets, when configuring
-[automatically issued acknowledgement labels](basic-connections.html#target-issued-acknowledgement-label), requested
-acknowledgements are produced in the following way:
-
-Once the AMQP 1.0 client signals that the message was acknowledged by the AMQP 1.0 endpoint, the following information
-is mapped to the automatically created [acknowledgement](protocol-specification-acks.html#acknowledgement):
+The target address supports [placeholders](basic-connections.html#placeholder-for-target-addresses).
-* Acknowledgement.status:
- * will be `200`, if the message was successfully consumed by the AMQP 1.0 endpoint
- * will be `5xx`, if the AMQP 1.0 endpoint failed in consuming the message, retrying sending the message is feasible
-* Acknowledgement.value:
- * will be missing, for status `200`
- * will contain more information, in case that an error `status` was set
+### Target acknowledgement handling
+When you configure [issued acknowledgement labels](basic-connections.html#target-issued-acknowledgement-label):
-### Specific configuration properties
+| Status | Condition |
+|--------|-----------|
+| `200` | Message successfully consumed by the endpoint |
+| `5xx` | Endpoint failed to consume the message (retry feasible) |
-The specific configuration properties are interpreted as
-[JMS Configuration options](https://qpid.apache.org/releases/qpid-jms-0.40.0/docs/index.html#jms-configuration-options).
-Use these to customize and tweak your connection as needed.
+## Specific configuration options
+AMQP 1.0 specific configuration properties are interpreted as
+[JMS Configuration options](https://qpid.apache.org/releases/qpid-jms-0.40.0/docs/index.html#jms-configuration-options).
### HMAC request signing
-Ditto supports HMAC request signing for AMQP 1.0 connections. Find detailed information on this in
-[Connectivity API > HMAC request signing](connectivity-hmac-signing.html).
+Ditto supports HMAC request signing for AMQP 1.0 connections. See
+[HMAC request signing](connectivity-hmac-signing.html) for details.
-
-## Establishing connecting to an AMQP 1.0 endpoint
-
-Ditto's [Connectivity service](architecture-services-connectivity.html) is responsible for creating new and managing
-existing connections.
-
-This can be done dynamically at runtime without the need to restart any microservice using a
-[Ditto DevOps command](installation-operating.html#devops-commands).
-
-Example connection configuration to create a new AMQP 1.0 connection:
+## Example connection JSON
```json
{
@@ -204,22 +155,22 @@ Example connection configuration to create a new AMQP 1.0 connection:
"connectionStatus": "open",
"failoverEnabled": true,
"uri": "amqps://user:password@hono.eclipse.org:5671",
- "sources": [
- {
- "addresses": [
- "telemetry/FOO"
- ],
- "authorizationContext": ["ditto:inbound-auth-subject"]
- }
- ],
- "targets": [
- {
- "address": "events/twin",
- "topics": [
- "_/_/things/twin/events"
- ],
- "authorizationContext": ["ditto:outbound-auth-subject"]
- }
- ]
+ "sources": [{
+ "addresses": ["telemetry/FOO"],
+ "authorizationContext": ["ditto:inbound-auth-subject"]
+ }],
+ "targets": [{
+ "address": "events/twin",
+ "topics": ["_/_/things/twin/events"],
+ "authorizationContext": ["ditto:outbound-auth-subject"]
+ }]
}
```
+
+## Further reading
+
+* [Connections overview](basic-connections.html) -- connection model and configuration
+* [Payload mapping](connectivity-mapping.html) -- transform message payloads
+* [Header mapping](connectivity-header-mapping.html) -- map external headers
+* [HMAC signing](connectivity-hmac-signing.html) -- HMAC-based authentication
+* [TLS certificates](connectivity-tls-certificates.html) -- secure connections with TLS
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-hono.md b/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-hono.md
index 522c9d5cf7b..4114d705fa2 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-hono.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-hono.md
@@ -5,75 +5,71 @@ tags: [connectivity]
permalink: connectivity-protocol-bindings-hono.html
---
-Consume messages from Eclipse Hono through Apache Kafka brokers and send messages to
-Eclipse Hono in the same manner as the [Kafka connection](connectivity-protocol-bindings-kafka2.html) does.
+You use the Eclipse Hono binding to integrate Ditto with Eclipse Hono without manually configuring Kafka topics, header mappings, or SASL settings.
-The Hono connection type is implemented just for convenience - to avoid the need for the user to be aware of the specific
-header mappings, address formats and Kafka `specificConfig` settings, which are required to connect to Eclipse Hono.
-These specifics are applied automatically at runtime for the connections of type Hono.
+{% include callout.html content="**TL;DR**: Configure a Hono connection with `connectionType: \"hono\"`. Source addresses use aliases (`event`, `telemetry`, `command_response`) that resolve to Kafka topics automatically. One Hono connection maps to one Hono tenant." type="primary" %}
-The Hono connection is based on the Kafka connection and uses it behind the scenes, so most of the
-[Kafka connection documentation](connectivity-protocol-bindings-kafka2.html) is valid for the Hono connection too,
-but with some exceptions, as described below.
+## Overview
+The Hono connection type is a convenience wrapper around the
+[Kafka connection](connectivity-protocol-bindings-kafka2.html). It automatically applies the
+correct header mappings, address formats, and Kafka `specificConfig` settings required to
+communicate with Eclipse Hono.
+
+Most of the [Kafka connection documentation](connectivity-protocol-bindings-kafka2.html) applies,
+with the exceptions described below.
{% include note.html
- content="A Hono connection is associated with _one_ Hono tenant. That means for each Hono tenant a separate Hono connection
- needs to be created. The tenant ID is used in the source and target connection addresses, representing the Kafka
- topics used by Hono for sending and receiving messages for this tenant. See below sections [Source addresses](#source-addresses),
- [Source reply target](#source-reply-target) and [Target Address](#target-address). The Hono tenant ID for the connection is defined in the
- [specific config](#specific-configuration-properties) `honoTenantId` property."
+ content="A Hono connection is associated with _one_ Hono tenant. Create a separate connection for
+ each tenant. The tenant ID is configured via `specificConfig.honoTenantId`."
%}
-## Specific Hono connection configuration
+## Connection URI format
+
+You do **not** specify the `uri` in a Hono connection -- it is generated automatically from the
+connectivity service configuration:
+
+* `ditto.connectivity.hono.base-uri` -- protocol, host, and port
+* `ditto.connectivity.hono.username` -- Kafka username
+* `ditto.connectivity.hono.password` -- Kafka password
+
+The resulting URI follows the format `tcp://username:password@host:port`.
-### Connection URI
-In the Hono connection definition, the `uri` property should not be specified (any specified value will be ignored).
-The connection URI and credentials are common for all Hono connections and are derived from the [configuration](installation-operating.html#ditto-configuration) of the connectivity service.
-`uri` will be automatically generated, based on the values of 3 configuration properties of the connectivity service -
-`ditto.connectivity.hono.base-uri`, `ditto.connectivity.hono.username` and `ditto.connectivity.hono.password`.
-The connectivity service property `base-uri` must specify protocol, host and port number (see the [example below](#connectivity-configuration-example)).
-In order to connect to Kafka brokers, `username` and `password` values will be inserted at runtime between the
-protocol identifier and the host name parts of `base-uri`, resulting in a connection URI of the form `tcp://username:password@host.name:port`.
+You must restart the service after changing these properties.
-Note: If any of these parameters has to be changed, the service must be restarted to apply the new values.
+## Source configuration
-### Source format
-#### Source addresses
-For a Hono connection, source "addresses" are specified as aliases, which are resolved at runtime to Kafka topics to subscribe to.
-Valid source addresses (aliases) are `event`, `telemetry` and `command_response`.
-At runtime, these are resolved as follows:
-* `event` -> `{%raw%}hono.event.{%endraw%}`
-* `telemetry` -> `{%raw%}hono.telemetry.{%endraw%}`
-* `command_response` -> `{%raw%}hono.command_response.{%endraw%}`
+### Source addresses
-`{%raw%}{%endraw%}` will be replaced by the value of `specificConfig.honoTenantId` or, if not set,
-by the connection id.
+Source `addresses` use aliases that resolve to Kafka topics at runtime:
-#### Source reply target
-Similar to source addresses, the reply target `address` is an alias as well. The single valid value for it is `command`.
-It is resolved to Kafka topic/key like this:
-* `command` -> `{%raw%}hono.command./{%endraw%}`
+| Alias | Resolved Kafka topic |
+|-------|---------------------|
+| `event` | `hono.event.` |
+| `telemetry` | `hono.telemetry.` |
+| `command_response` | `hono.command_response.` |
-`{%raw%}{%endraw%}` will be replaced by the value of `specificConfig.honoTenantId` or, if not set,
-by the connection id. `{%raw%}{%endraw%}` is substituted by the thing ID value.
+If `specificConfig.honoTenantId` is not set, the connection ID is used instead.
-The needed header mappings for the `replyTarget` are also populated automatically at runtime and there is
-no need to specify them in the connection definition. Any of the following specified value will be substituted (i.e. ignored).
-Actually the `headerMapping` subsection is not required and could be omitted completely (in the context of `replyTarget`).
+### Source reply target
+
+The reply target `address` alias `command` resolves to:
+
+* `hono.command./`
+
+Header mappings for the reply target are auto-generated. For `telemetry` and `event` sources:
-For addresses `telemetry` and `event`, the following header mappings will be automatically applied:
* `device_id`: `{%raw%}{{ thing:id }}{%endraw%}`
-* `subject`: `{%raw%}{{ header:subject \| fn:default(topic:action-subject) \| fn:default(topic:criterion) }}{%endraw%}-response`
+* `subject`: `{%raw%}{{ header:subject | fn:default(topic:action-subject) | fn:default(topic:criterion) }}{%endraw%}-response`
* `correlation-id`: `{%raw%}{{ header:correlation-id }}{%endraw%}`
-For address `command_response`, the following header mappings will be automatically applied:
+For `command_response` sources:
+
* `correlation-id`: `{%raw%}{{ header:correlation-id }}{%endraw%}`
* `status`: `{%raw%}{{ header:status }}{%endraw%}`
-Note: Any other header mappings defined manually will be merged with the auto-generated ones.
+Any additional manually defined header mappings are merged with the auto-generated ones.
-The following example shows a valid Hono-connection source:
```json
{
"addresses": ["event"],
@@ -97,34 +93,32 @@ The following example shows a valid Hono-connection source:
"declaredAcks": []
}
```
-#### Source header mapping
-
-The Hono connection does not need any header mapping for sources. Nevertheless, the header mappings documented for
-[Kafka connection](connectivity-protocol-bindings-kafka2.html) are still available.
-See [Source header mapping](connectivity-protocol-bindings-kafka2.html#source-header-mapping) in Kafka protocol bindings
-and [Header mapping for connections](connectivity-header-mapping.html).
-
-### Target format
-#### Target address
-The target `address` is specified as an alias and the only valid alias is `command`.
-It is automatically resolved at runtime to the following Kafka topic/key:
-* `command` -> `{%raw%}hono.command./{%endraw%}`
-
-`{%raw%}{%endraw%}` will be replaced by the value of `specificConfig.honoTenantId` or, if not set,
-by the connection id. `{%raw%}{%endraw%}` is substituted by the thing ID value.
-
-#### Target header mapping
-The target `headerMapping` section is also populated automatically at runtime and there is
-no need to specify it the connection definition i.e. could be omitted.
-If any of the following keys are specified in the connection, they will be ignored and automatically substituted as follows:
+
+### Source header mapping
+
+No source header mappings are required for Hono connections. The header mappings documented for
+[Kafka connections](connectivity-protocol-bindings-kafka2.html#source-header-mapping) are still
+available if needed.
+
+## Target configuration
+
+### Target address
+
+The only valid target `address` alias is `command`, which resolves to:
+
+* `hono.command./`
+
+### Target header mapping
+
+Target header mappings are auto-generated:
+
* `device_id`: `{%raw%}{{ thing:id }}{%endraw%}`
-* `subject`: `{%raw%}{{ header:subject \| fn:default(topic:action-subject) }}{%endraw%}`
+* `subject`: `{%raw%}{{ header:subject | fn:default(topic:action-subject) }}{%endraw%}`
* `response-required`: `{%raw%}{{ header:response-required }}{%endraw%}`
* `correlation-id`: `{%raw%}{{ header:correlation-id }}{%endraw%}`
-Note: Any other header mappings defined manually will be merged with the auto-generated ones.
+Additional manually defined mappings are merged with the auto-generated ones.
-The following example shows a valid Hono-connection target:
```json
{
"address": "command",
@@ -136,78 +130,68 @@ The following example shows a valid Hono-connection target:
}
```
-### Specific configuration properties
+## Specific configuration options
-In the `specificConfig` section, the Hono tenant of the connection is specified in the `honoTenantId` property.
-If that property is not set, the connection ID will be taken as Hono tenant ID.
+| Property | Description | Default |
+|----------|-------------|---------|
+| `honoTenantId` | Hono tenant ID for this connection | connection ID |
+| `groupId` | Kafka consumer group ID | connection ID |
+| `debugEnabled` | Include debug information in acknowledgements | `false` |
-The following Kafka connection related properties in the `specificConfig` section will be automatically added at runtime
-to the connection. Any manually specified definition of `bootstrapServers` and `saslMechanism` will be ignored, but `groupId` will not.
-* `bootstrapServers` The value will be taken from the configuration property `ditto.connectivity.hono.bootstrap-servers` of the connectivity service.
-It must contain a comma separated list of Kafka bootstrap servers to use for connecting to (in addition to the automatically added connection uri).
-* `saslMechanism` The value will be taken from configuration property `ditto.connectivity.hono.sasl-mechanism`.
-The value must be one of `SaslMechanism` enum values to select the SASL mechanisms to use for authentication at Kafka:
- * `PLAIN`
- * `SCRAM-SHA-256`
- * `SCRAM-SHA-512`
-* `groupId`: could be specified by the user, but is not required. If omitted, the value of the connection ID will be automatically used.
+The following properties are set automatically from the connectivity service configuration and
+cannot be overridden:
-Hono connection still allows to manually specify additional properties (like `debugEnabled`), which will be merged with the auto-generated ones.
+* `bootstrapServers` -- from `ditto.connectivity.hono.bootstrap-servers`
+* `saslMechanism` -- from `ditto.connectivity.hono.sasl-mechanism`
-### Certificate validation
-The connection property `validateCertificates` is also set automatically. The value is taken from the `ditto.connectivity.hono.validate-certificates` property.
-For more details see [Connection configuration](connectivity-tls-certificates.html).
+The `validateCertificates` connection property is also set automatically from
+`ditto.connectivity.hono.validate-certificates`.
+
+## Example connection JSON
-## Examples
-### Example of Hono connection
```json
{
- "connection": {
- "id": "connection-for-hono-example-tenant",
- "connectionType": "hono",
- "connectionStatus": "open",
- "failoverEnabled": true,
- "specificConfig": {
- "honoTenantId": "example-tenant"
+ "id": "connection-for-hono-example-tenant",
+ "connectionType": "hono",
+ "connectionStatus": "open",
+ "failoverEnabled": true,
+ "specificConfig": {
+ "honoTenantId": "example-tenant"
+ },
+ "sources": [{
+ "addresses": ["event"],
+ "consumerCount": 1,
+ "qos": 1,
+ "authorizationContext": ["ditto:inbound-auth-subject"],
+ "enforcement": {
+ "input": "{%raw%}{{ header:device_id }}{%endraw%}",
+ "filters": ["{%raw%}{{ entity:id }}{%endraw%}"]
},
- "sources": [
- {
- "addresses": ["event"],
- "consumerCount": 1,
- "qos": 1,
- "authorizationContext": ["ditto:inbound-auth-subject"],
- "enforcement": {
- "input": "{%raw%}{{ header:device_id }}{%endraw%}",
- "filters": ["{%raw%}{{ entity:id }}{%endraw%}"]
- },
- "headerMapping": {},
- "payloadMapping": ["Ditto"],
- "replyTarget": {
- "enabled": true,
- "address": "command",
- "expectedResponseTypes": ["response", "error", "nack"]
- },
- "acknowledgementRequests": {
- "includes": []
- },
- "declaredAcks": []
- }
+ "headerMapping": {},
+ "payloadMapping": ["Ditto"],
+ "replyTarget": {
+ "enabled": true,
+ "address": "command",
+ "expectedResponseTypes": ["response", "error", "nack"]
+ },
+ "acknowledgementRequests": {
+ "includes": []
+ },
+ "declaredAcks": []
+ }],
+ "targets": [{
+ "address": "command",
+ "topics": [
+ "_/_/things/twin/events",
+ "_/_/things/live/messages"
],
- "targets": [
- {
- "address": "command",
- "topics": [
- "_/_/things/twin/events",
- "_/_/things/live/messages"
- ],
- "authorizationContext": ["ditto:outbound-auth-subject"]
- }
- ]
- }
+ "authorizationContext": ["ditto:outbound-auth-subject"]
+ }]
}
```
-### Connectivity configuration example
-Here is an example with all the configuration options of the connectivity service that are needed by Hono connections:
+
+### Connectivity service configuration example
+
```
ditto {
connection {
@@ -217,13 +201,21 @@ ditto {
password = "honoPassword"
sasl-mechanism = "PLAIN"
bootstrap-servers = "localhost:9092"
- validateCertificates = true,
+ validateCertificates = true
ca = "-----BEGIN CERTIFICATE-----\n\n-----END CERTIFICATE-----"
}
}
}
```
-## Troubleshooting Hono connection configuration
-To help the troubleshooting, a separate Piggyback command `retrieveHonoConnection` is implemented.
-It is valid only for Hono connections. It returns the "real" Hono connection after all its properties being resolved or auto-generated.
-The returned value could be used for inspection, but not for example to create a new Hono connection using it.
+
+## Troubleshooting
+
+Use the `retrieveHonoConnection` piggyback command to inspect the fully resolved Hono connection
+with all auto-generated properties. This is useful for debugging but the output cannot be used
+directly to create a new connection.
+
+## Further reading
+
+* [Kafka 2.x binding](connectivity-protocol-bindings-kafka2.html) -- underlying Kafka connection details
+* [Connections overview](basic-connections.html) -- connection model and configuration
+* [Payload mapping](connectivity-mapping.html) -- transform message payloads
diff --git a/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-http.md b/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-http.md
index 2ac270c3816..24dd6155208 100644
--- a/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-http.md
+++ b/documentation/src/main/resources/pages/ditto/connectivity-protocol-bindings-http.md
@@ -5,50 +5,47 @@ tags: [protocol, connectivity, rql]
permalink: connectivity-protocol-bindings-http.html
---
-Perform HTTP request (with verbs GET, POST, PUT, PATCH) to HTTP endpoints via [targets](#target-format).
+You use the HTTP 1.1 binding to push data from Ditto to external HTTP endpoints using GET, POST, PUT, or PATCH requests.
-## Specific connection configuration
+{% include callout.html content="**TL;DR**: Configure an HTTP connection with `connectionType: \"http-push\"`. Target addresses use the format `VERB:/path`. HTTP connections are outbound-only -- they do not support sources." type="primary" %}
-The common configuration for connections in [Connections > Targets](basic-connections.html#targets) applies here
-as well. Following are some specifics for HTTP connections:
+## Overview
-### Source format
+The HTTP 1.1 protocol binding lets you perform HTTP requests to external endpoints via
+[targets](#target-configuration). This is useful for forwarding events, sending messages,
+or integrating with REST APIs.
-{% include note.html content="HTTP connections currently don't support sources - the HTTP integration may only be used for pushing data out." %}
+{% include note.html content="HTTP connections do not support sources -- the HTTP integration is outbound-only." %}
-### Target format
+## Connection URI format
-A HTTP connection requires the protocol configuration target object to have an `address` property.
-This property has the following format: `:`
+```
+http://user:password@hostname:80
+```
-The supported HTTP `` values are:
-* GET
-* POST
-* PUT
-* PATCH
+Use `https://` for TLS-secured connections.
-The specified `` contains the path - including optionally potential query parameters - to be appended to the
-configured `uri` of the connection.
+## Source configuration
-The body of the HTTP request is either the outgoing [Ditto Protocol](protocol-specification.html) message (e.g. an event)
-or - if a [payload mapping](connectivity-mapping.html) was specified in the connection - a transformed body.
+HTTP connections do not support sources.
-The target address may contain placeholders; see
-[placeholders](basic-connections.html#placeholder-for-target-addresses) section for more information.
+## Target configuration
-The target may define a [header mapping](connectivity-header-mapping.html) specifying which additional HTTP headers to
-send along with the performed HTTP requests.
+The common [target configuration](basic-connections.html#targets) applies. The target `address`
+combines an HTTP verb and path:
-Further, `"topics"` is a list of strings, each list entry representing a subscription of
-[Ditto protocol topics](protocol-specification-topic.html), see
-[target topics and filtering](basic-connections.html#target-topics-and-filtering) for more information on that.
+```
+:
+```
-Outbound messages are published to the configured target address if one of the subjects in `"authorizationContext"`
-has READ permission on the thing, which is associated with a message.
+Supported HTTP verbs: `GET`, `POST`, `PUT`, `PATCH`.
+
+The `