Skip to content

fix(point): validate newlines on write_line_protocol - #49

Open
an-jello wants to merge 4 commits into
InfluxCommunity:mainfrom
an-jello:fix/point/validate-linebreaks
Open

fix(point): validate newlines on write_line_protocol#49
an-jello wants to merge 4 commits into
InfluxCommunity:mainfrom
an-jello:fix/point/validate-linebreaks

Conversation

@an-jello

@an-jello an-jello commented Aug 13, 2026

Copy link
Copy Markdown

Proposed Changes

In line with InfluxDB 3's Line Protocol reference, this PR adds an extra validation on Point::write_line_protocol() where fields containing newlines (\n) will throw out an error.

Checklist

  • CHANGELOG.md updated
  • Rebased/mergeable
  • Commit messages are conventional
  • Sign CLA (if not already signed)

@an-jello
an-jello force-pushed the fix/point/validate-linebreaks branch from 6100a9a to 63f4539 Compare August 13, 2026 02:53
@bednar
bednar requested a review from karel-rehor August 27, 2026 07:33
@bednar

bednar commented Aug 27, 2026

Copy link
Copy Markdown
Member

@karel-rehor please take a look to this

@alespour

alespour commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

For consideration - AI (5.6-Sol & Opus 5) converged on these potential issues:

  1. Medium — Measurement is not validated. The new guard checks tag keys/values and field keys/string values at src/point.rs:262, but the measurement is written unchecked at src/point.rs:294. measurement_needs_escape handles only commas and spaces at src/point.rs:395, so Point::new("a\nb").field(...) still emits a raw newline and creates a bogus additional line - the corruption this PR is intended to prevent.

  2. Medium - default_tags are not validated. The new validation examines only self.tags(). Default tags are merged later at src/point.rs:313 and written unchecked at src/point.rs:344. Because write_escaped_tag_key and write_escaped_tag_value do not escape newlines, a newline in a WriteOptions default-tag key or value corrupts every serialized point using that option.

  3. Medium - The new validation has no regression tests. The PR changes only src/point.rs and CHANGELOG.md; it does not extend the existing serialization tests in tests/point_tests.rs:4. Add table-driven rejection cases for measurement, point-tag key/value, default-tag key/value, field key, and string-field value, plus a valid no-newline control. Without these cases, the intended behavior and the two bypasses above are not protected against regression.

  4. Low - The new changelog entry is misplaced and inaccurate. CHANGELOG.md:20 uses ## Bug Fixes, making it a sibling of ## 0.3.0 [unreleased] and placing the entry outside that release. It should be ### Bug Fixes, alongside ### Performance, ### Dependencies, and ### CI. The text also says only fields are validated even though tags are covered, and it names the crate-private Point::write_line_protocol() instead of the public Point::to_line_protocol() and client write paths.

@an-jello
an-jello force-pushed the fix/point/validate-linebreaks branch from 63f4539 to 105d09f Compare August 27, 2026 10:15
@an-jello

Copy link
Copy Markdown
Author

thanks for the replies; rebased to latest on main, added tests & adjusted changelog

@alespour point 1 & 2 highlighted by the LLM is not necessarily relevant in the scope of the PR.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.44%. Comparing base (b98caa8) to head (ef2e8f4).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #49      +/-   ##
==========================================
+ Coverage   82.74%   83.44%   +0.69%     
==========================================
  Files          10       10              
  Lines        1971     1999      +28     
==========================================
+ Hits         1631     1668      +37     
+ Misses        340      331       -9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@idclark
idclark requested review from idclark and removed request for karel-rehor August 27, 2026 17:35
@idclark

idclark commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Hi @an-jello

Thanks for catching this, the underlying bug is real and worse than the description suggests. On main a newline anywhere in a point splits it into two lines of line protocol, so a user-controlled tag value can inject an entirely separate point into the batch. Worth fixing.

That said, I'd like to go with escaping rather than rejecting, to match the rest of the client family:

Rust is currently the only client doing nothing at all, so we're the outlier, but erroring would also make us the outlier in the other direction, and it's import that users have a similar experience across influxdb clients.

Concretely, I'd suggest adding \n / \r / \t to escape_with in src/point.rs (extend needs_escape and emit \\ + 'n' instead of \\ + the char). That approach also closes some gaps in the current patch, which I confirmed against this branch:

measurement      -> OK  "me\nas v=1i"       [2 lines]  # measurement isn't validated
CR in tag value  -> OK  "m,host=a\rb v=1i"             # \r not covered
TAB in tag value -> OK  "m,host=a\tb v=1i"             # \t not covered

Plus two more the validation loop can't reach:

  • WriteOptions::default_tags are merged in after the check, so default_tags = {"env": "prod\ninjected,x=1 y=2"} still emits an injected second point.
  • The polars path (write_dataframe.rs) never constructs a Point — it calls escape_tag / escape_measurement directly, so it gets no protection here. That's the bulk-ingest path where a stray newline in a string column is most likely.

Because Point and write_dataframe share those escape helpers, fixing it there covers every path at once, including measurement names and default tags.

Two smaller notes if you'd like to keep iterating on this:

  • Error::Config is meant for client misconfiguration; invalid point data probably wants its own variant. The message also interpolates the raw newline, which splits log lines too.
  • The tests use #[should_panic] on .unwrap(), which asserts on a panic message rather than the error itself. assert!(matches!(result, Err(Error::...))) would be more direct.

@bednar

bednar commented Aug 28, 2026

Copy link
Copy Markdown
Member

Let’s align all clients on the same solution: escaping characters, as already implemented in the Python, Java, C#, and JS clients.

@an-jello, would you like to proceed with the same approach in Rust? If it’s too much work on your side, we can take care of it.

@bednar

bednar commented Aug 28, 2026

Copy link
Copy Markdown
Member

PR for GO v3 je ready - InfluxCommunity/influxdb3-go#287

@an-jello

an-jello commented Aug 28, 2026

Copy link
Copy Markdown
Author

I'd probably yield this PR for your team to take care of it; It's a little bit too much for my plate right now but I'm happy that y'all finally get back to this 🙏

There are quite a few thought that occurred and manifested in me since writing the initial PR. If you don't mind some potential noise / my personal 2-cents:

Details

This bug might be CVE-minor worthy after all; I can imagine an SQL injection-like attack but for data appends or data point alteration. (If filing for CVE / changelog, I would be happy if it could mention me or this PR 😛 /halfjoking).

As an outsider to the InfluxDB internals and someone who's not familiar with InfluxDB's ecosystem, my uneducated judgement that's solely based on the Line Protocol reference doc would be that I would be against escaping under the basis of correctness;

I do understand that the behavior suggested is based on familiarity and reproducibility with other clients, but Familiarity is not necessarily Correct: I view escaping \n into an \\n to be an incorrect implementation of InfluxDB client / implementation of the Line Protocol. As Rust has an ecosystem with a focus towards correctness, I would prefer the Rust client to be correct and throw an explicit error for it instead of escaping it, producing a potentially undocumented behavior.

You've also mentioned concerns on the TAB character (\t) and the Carriage Return character (\r) but according to the Line Protocol reference, those characters can exist just fine as a part of a valid Line Protocol data point.

Respectfully, y'all might have a bigger issue with correctness within the ecosystem :(

On the usage of Error::Config for my changes, I simply copied and pasted the error rejected on the line above (https://github.com/an-jello/fork.InfluxCommunity.influxdb3-rust/blob/fix%2Fpoint%2Fvalidate-linebreaks/src/point.rs#L256-L260). I am personally unsure on the extent of Error::Config but the code comment stated "missing required field", so I thought it'll make sense for the error to also throw in that type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants