feat(kafka): support transaction isolation levels#933
Open
mbuckley2000 wants to merge 2 commits into
Open
Conversation
jem-davies
reviewed
Jul 15, 2026
jem-davies
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for the PR 🙏 - yeah seems we aren't exposing this config option at the moment.
Only thing I would say is that we could get rid of a lot of stuff:
- The
NewStringAnnotatedEnumFieldwill throw lint errors if the value isn't one of the options, anyway. (I do realise it doesn't do this using the .ParseYAML() like in your test) Perhaps this could be simplified such we only have one switch in the components constructor i.e.newFranzKafkaReaderFromConfig- we add a default here that throws an error - like withkafka'ssasl.mechanism - The new unit tests are somewhat redundant - we could keep them - but they largely seem to just be checking configField spec & parsing - not much else.
- I would move this field down such it appears later in the config for doc readability.
- I would just duplicate the field across the two components personally
This review is nit-picky but then we would want to be consistent with the existing patterns in the code imo.
| }). | ||
| Description("Controls the isolation level used for Kafka fetch requests."). | ||
| Default(string(transactionIsolationLevelReadUncommitted)). | ||
| Advanced() |
Collaborator
There was a problem hiding this comment.
Suggested change
| Advanced() | |
| Advanced(). | |
| Version("1.20.0") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add an advanced
transaction_isolation_levelfield to both Kafka inputs:read_uncommitted(the backward-compatible default)read_committedThe legacy
kafkainput maps it to Sarama's consumer isolation level.kafka_franzmaps it to franz-go's fetch isolation level. Both inputs share one field definition and defensive parser, and their generated documentation describes the behavior.Closes #932.
Why
Both clients support Kafka transactions, but Bento previously offered no way to hide aborted records from consumers of transactional producers such as Kafka Streams exactly-once topologies.
The field name and values follow the existing Redpanda Connect franz-go input convention.
Tests
go vet ./internal/impl/kafkagolangci-lint -j 1 run -c .golangci.yml ./internal/impl/kafka(0 issues)go test ./internal/impl/kafka -run TestIntegrationKafkaTransactionIsolation -count=1 -v -timeout 2mmake docsgenerated the changed Kafka pages successfully; its repository-wide lint subsequently failed on pre-existing DuckDB pages because the normal non-CGO documentation build cannot load the DuckDB driver.