Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ If you're using custom configuration settings in your InfluxDB 1.x instance, do
| cache-snapshot-write-cold-duration | [storage-cache-snapshot-write-cold-duration](/influxdb/v2/reference/config-options/#storage-cache-snapshot-write-cold-duration) |
| compact-full-write-cold-duration | [storage-compact-full-write-cold-duration](/influxdb/v2/reference/config-options/#storage-compact-full-write-cold-duration) |
| max-concurrent-compactions | [storage-max-concurrent-compactions](/influxdb/v2/reference/config-options/#storage-max-concurrent-compactions) |
| compact-throughput | |
| compact-throughput | (not exposed as a separate option; see `storage-compact-throughput` notes) |
| compact-throughput-burst | [storage-compact-throughput-burst](/influxdb/v2/reference/config-options/#storage-compact-throughput-burst) |
| tsm-use-madv-willneed | [storage-tsm-use-madv-willneed](/influxdb/v2/reference/config-options/#storage-tsm-use-madv-willneed) |
| max-series-per-database | |
Expand Down
73 changes: 72 additions & 1 deletion content/influxdb/v2/reference/config-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ To configure InfluxDB, use the following configuration options when starting the
- [storage-cache-snapshot-memory-size](#storage-cache-snapshot-memory-size)
- [storage-cache-snapshot-write-cold-duration](#storage-cache-snapshot-write-cold-duration)
- [storage-compact-full-write-cold-duration](#storage-compact-full-write-cold-duration)
- [storage-compact-throughput](#storage-compact-throughput)
- [storage-compact-throughput-burst](#storage-compact-throughput-burst)
- [storage-max-concurrent-compactions](#storage-max-concurrent-compactions)
- [storage-max-index-log-file-size](#storage-max-index-log-file-size)
Expand Down Expand Up @@ -2190,8 +2191,78 @@ storage-compact-full-write-cold-duration = "4h0m0s"

---

### storage-compact-throughput
Sustained rate limit (in bytes per second) that TSM compactions can write to disk.

InfluxDB also supports a separate burst limit (`storage-compact-throughput-burst`).
In InfluxDB OSS v2, the sustained and burst compaction throughput values are currently the same.
To tune compaction write throughput, set `storage-compact-throughput-burst` and verify the effective values in the `Compaction settings` log entry at startup.

Example log entry:

```text
Compaction settings {"max_concurrent_compactions": 7, "throughput_bytes_per_second": 50331648, "throughput_bytes_per_second_burst": 50331648}
```

**Default:** `50331648`

| influxd flag | Environment variable | Configuration key |
| :----------- | :------------------- | :---------------- |
| _Not available_ | `INFLUXD_STORAGE_COMPACT_THROUGHPUT` | `storage-compact-throughput` |

###### Environment variable
```sh
export INFLUXD_STORAGE_COMPACT_THROUGHPUT=50331648
```

###### Configuration file
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[YAML](#)
[TOML](#)
[JSON](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```yml
storage-compact-throughput: 50331648
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```toml
storage-compact-throughput = 50331648
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```json
{
"storage-compact-throughput": 50331648
}
```
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}

---

### storage-compact-throughput-burst
Rate limit (in bytes per second) that TSM compactions can write to disk.
Maximum rate limit (in bytes per second) that TSM compactions can write to disk.

In InfluxDB OSS v2, this setting also effectively controls the sustained compaction throughput.

#### Scale compaction throughput

If you scale to larger machines and faster storage, increase this value to let compactions keep up with ingest.
If you set this too high, compactions can compete with writes and queries for disk I/O and hurt performance.

To identify a compaction bottleneck, monitor the following:

- `storage_compactions_queued` and `storage_compactions_active` from the `/metrics` endpoint
- Disk I/O utilization and throughput from your OS or infrastructure metrics

Example values:

- `50331648` (default, about 48 MiB/s)
- `100663296` (about 96 MiB/s)
- `201326592` (about 192 MiB/s)

**Default:** `50331648`

Expand Down
4 changes: 3 additions & 1 deletion content/influxdb/v2/reference/internals/shards.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,9 @@ The following configuration settings are especially beneficial for systems with
- [`storage-max-concurrent-compactions`](/influxdb/v2/reference/config-options/#storage-max-concurrent-compactions)
- [`storage-max-index-log-file-size`](/influxdb/v2/reference/config-options/#storage-max-index-log-file-size)
- [`storage-series-file-max-concurrent-snapshot-compactions`](/influxdb/v2/reference/config-options/#storage-series-file-max-concurrent-snapshot-compactions)
- [`storage-series-file-max-concurrent-snapshot-compactions`](/influxdb/v2/reference/config-options/#storage-series-file-max-concurrent-snapshot-compactions)

In InfluxDB OSS v2, `storage-compact-throughput-burst` effectively controls both the sustained and burst compaction throughput.
Use it to scale compaction write throughput as you scale CPU and disk.

In systems with stable loads, if compactions interfere with other operations, typically, the system is undersized for its load, and configuration changes won't help much.

Expand Down
Loading