From cd54c1cced8c0087f0596af0e4e6bb549fb04462 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 16 Apr 2026 08:03:26 -0500 Subject: [PATCH 1/3] docs(v2): document storage-compact-throughput configuration parameter - Add storage-compact-throughput to config options reference - Document environment variable and configuration file settings - Note that CLI flag will be available in a future release - Default value: 50331648 bytes per second - Controls rate limit for TSM compaction disk writes Closes #7107 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../influxdb/v2/reference/config-options.md | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 4f2d83d1e2..1b3f1dc236 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -170,6 +170,7 @@ To configure InfluxDB, use the following configuration options when starting the - [storage-cache-snapshot-memory-size](#storage-cache-snapshot-memory-size) - [storage-cache-snapshot-write-cold-duration](#storage-cache-snapshot-write-cold-duration) - [storage-compact-full-write-cold-duration](#storage-compact-full-write-cold-duration) +- [storage-compact-throughput](#storage-compact-throughput) - [storage-compact-throughput-burst](#storage-compact-throughput-burst) - [storage-max-concurrent-compactions](#storage-max-concurrent-compactions) - [storage-max-index-log-file-size](#storage-max-index-log-file-size) @@ -2190,6 +2191,51 @@ storage-compact-full-write-cold-duration = "4h0m0s" --- +### storage-compact-throughput +Rate limit (in bytes per second) that TSM compactions can write to disk. + +> [!Note] +> The CLI flag for this option will be available in a future release. + +**Default:** `50331648` + +| influxd flag | Environment variable | Configuration key | +| :----------- | :------------------- | :---------------- | +| _Not yet available_ | `INFLUXD_STORAGE_COMPACT_THROUGHPUT` | `storage-compact-throughput` | + +###### Environment variable +```sh +export INFLUXD_STORAGE_COMPACT_THROUGHPUT=50331648 +``` + +###### Configuration file +{{< code-tabs-wrapper >}} +{{% code-tabs %}} +[YAML](#) +[TOML](#) +[JSON](#) +{{% /code-tabs %}} +{{% code-tab-content %}} +```yml +storage-compact-throughput: 50331648 +``` +{{% /code-tab-content %}} +{{% code-tab-content %}} +```toml +storage-compact-throughput = 50331648 +``` +{{% /code-tab-content %}} +{{% code-tab-content %}} +```json +{ + "storage-compact-throughput": 50331648 +} +``` +{{% /code-tab-content %}} +{{< /code-tabs-wrapper >}} + +--- + ### storage-compact-throughput-burst Rate limit (in bytes per second) that TSM compactions can write to disk. From 42a2fb8138fdcb8cec191c3bae03101e39824bcd Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 16 Apr 2026 09:22:59 -0500 Subject: [PATCH 2/3] Update content/influxdb/v2/reference/config-options.md --- content/influxdb/v2/reference/config-options.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 1b3f1dc236..7e1c92fc7b 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -2192,7 +2192,7 @@ storage-compact-full-write-cold-duration = "4h0m0s" --- ### storage-compact-throughput -Rate limit (in bytes per second) that TSM compactions can write to disk. +Rate (in bytes per second) that TSM compactions can write to disk. > [!Note] > The CLI flag for this option will be available in a future release. From 8287a5efca823d7a521c5ed3499fca88cd1656ac Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Fri, 17 Apr 2026 12:59:56 -0500 Subject: [PATCH 3/3] docs(v2): clarify compaction throughput tuning --- .../upgrade/v1-to-v2/manual-upgrade.md | 2 +- .../influxdb/v2/reference/config-options.md | 35 ++++++++++++++++--- .../influxdb/v2/reference/internals/shards.md | 4 ++- 3 files changed, 34 insertions(+), 7 deletions(-) diff --git a/content/influxdb/v2/install/upgrade/v1-to-v2/manual-upgrade.md b/content/influxdb/v2/install/upgrade/v1-to-v2/manual-upgrade.md index 6d2ee101af..fe0051d578 100644 --- a/content/influxdb/v2/install/upgrade/v1-to-v2/manual-upgrade.md +++ b/content/influxdb/v2/install/upgrade/v1-to-v2/manual-upgrade.md @@ -61,7 +61,7 @@ If you're using custom configuration settings in your InfluxDB 1.x instance, do | cache-snapshot-write-cold-duration | [storage-cache-snapshot-write-cold-duration](/influxdb/v2/reference/config-options/#storage-cache-snapshot-write-cold-duration) | | compact-full-write-cold-duration | [storage-compact-full-write-cold-duration](/influxdb/v2/reference/config-options/#storage-compact-full-write-cold-duration) | | max-concurrent-compactions | [storage-max-concurrent-compactions](/influxdb/v2/reference/config-options/#storage-max-concurrent-compactions) | -| compact-throughput | | +| compact-throughput | (not exposed as a separate option; see `storage-compact-throughput` notes) | | compact-throughput-burst | [storage-compact-throughput-burst](/influxdb/v2/reference/config-options/#storage-compact-throughput-burst) | | tsm-use-madv-willneed | [storage-tsm-use-madv-willneed](/influxdb/v2/reference/config-options/#storage-tsm-use-madv-willneed) | | max-series-per-database | | diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 7e1c92fc7b..9b1bbb24ca 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -2192,16 +2192,23 @@ storage-compact-full-write-cold-duration = "4h0m0s" --- ### storage-compact-throughput -Rate (in bytes per second) that TSM compactions can write to disk. +Sustained rate limit (in bytes per second) that TSM compactions can write to disk. -> [!Note] -> The CLI flag for this option will be available in a future release. +InfluxDB also supports a separate burst limit (`storage-compact-throughput-burst`). +In InfluxDB OSS v2, the sustained and burst compaction throughput values are currently the same. +To tune compaction write throughput, set `storage-compact-throughput-burst` and verify the effective values in the `Compaction settings` log entry at startup. + +Example log entry: + +```text +Compaction settings {"max_concurrent_compactions": 7, "throughput_bytes_per_second": 50331648, "throughput_bytes_per_second_burst": 50331648} +``` **Default:** `50331648` | influxd flag | Environment variable | Configuration key | | :----------- | :------------------- | :---------------- | -| _Not yet available_ | `INFLUXD_STORAGE_COMPACT_THROUGHPUT` | `storage-compact-throughput` | +| _Not available_ | `INFLUXD_STORAGE_COMPACT_THROUGHPUT` | `storage-compact-throughput` | ###### Environment variable ```sh @@ -2237,7 +2244,25 @@ storage-compact-throughput = 50331648 --- ### storage-compact-throughput-burst -Rate limit (in bytes per second) that TSM compactions can write to disk. +Maximum rate limit (in bytes per second) that TSM compactions can write to disk. + +In InfluxDB OSS v2, this setting also effectively controls the sustained compaction throughput. + +#### Scale compaction throughput + +If you scale to larger machines and faster storage, increase this value to let compactions keep up with ingest. +If you set this too high, compactions can compete with writes and queries for disk I/O and hurt performance. + +To identify a compaction bottleneck, monitor the following: + +- `storage_compactions_queued` and `storage_compactions_active` from the `/metrics` endpoint +- Disk I/O utilization and throughput from your OS or infrastructure metrics + +Example values: + +- `50331648` (default, about 48 MiB/s) +- `100663296` (about 96 MiB/s) +- `201326592` (about 192 MiB/s) **Default:** `50331648` diff --git a/content/influxdb/v2/reference/internals/shards.md b/content/influxdb/v2/reference/internals/shards.md index 3b5c93987b..25a0e1319b 100644 --- a/content/influxdb/v2/reference/internals/shards.md +++ b/content/influxdb/v2/reference/internals/shards.md @@ -134,7 +134,9 @@ The following configuration settings are especially beneficial for systems with - [`storage-max-concurrent-compactions`](/influxdb/v2/reference/config-options/#storage-max-concurrent-compactions) - [`storage-max-index-log-file-size`](/influxdb/v2/reference/config-options/#storage-max-index-log-file-size) - [`storage-series-file-max-concurrent-snapshot-compactions`](/influxdb/v2/reference/config-options/#storage-series-file-max-concurrent-snapshot-compactions) -- [`storage-series-file-max-concurrent-snapshot-compactions`](/influxdb/v2/reference/config-options/#storage-series-file-max-concurrent-snapshot-compactions) + +In InfluxDB OSS v2, `storage-compact-throughput-burst` effectively controls both the sustained and burst compaction throughput. +Use it to scale compaction write throughput as you scale CPU and disk. In systems with stable loads, if compactions interfere with other operations, typically, the system is undersized for its load, and configuration changes won't help much.