chore(deps): update dependency karafka to v2.6.1 - #746
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
2.6.0→2.6.1Release Notes
karafka/karafka (karafka)
v2.6.1Compare Source
config.encryption.mode = :envelope, requires the openssl gem>= 3.0). It removes the RSA payload size ceiling of the default:directmode (~245 bytes for 2048-bit keys) by encrypting each message with a one-time RSA-wrapped AES-256-GCM key. Both formats are recognized automatically on decryption, so data already at rest stays readable with no migration. Deployment: upgrade all consuming processes before enabling - older versions cannot decrypt envelope payloads, while upgraded consumers read both formats and keep producing:directuntil switched, so the rollout can be staged safely. The default will switch to:envelopein a future release with prior notice; the:directformat stays decryptable indefinitely as data at rest never expires. Cipher key material is now pre-parsed (warmed) during setup - this introduces no new boot failure modes, as the configuration contract has always parsed and validated all configured keys at boot.Karafka::Errors::UnsupportedCaseErrorinstead of aNoMethodError.1flush_batch_size, the target's chunk could confirm and evict the key before the tombstone chunk ran, leaving the schedule non-tombstoned in Kafka (and re-dispatched after a restart/reload) if that later chunk then failed.Karafka::Server#stopspecs from leaking a sub-secondshutdown_timeoutinto the global config. The#stopexamples mutateKarafka::App.config.shutdown_timeout(down to500ms, below the defaultmax_wait_timeof1_000ms) but never restored it, so with a random run order the invalid value bled into unrelated specs and made them fail contract validation withshutdown_timeout must be more than max_wait_time. The original value is now captured and restored (and the memoized class-level value cleared) in the example teardown.schedule_target_epochset implausibly far in the future (e.g. a milliseconds value passed as seconds) at publish time instead of silently dropping it on the consumer.Karafka::Errors::UnsupportedCaseErrorinstead of aNameError(wrong constant namespace) when a custom DLQ strategy returns an unsupported flow, in both the default and virtual-partitions DLQ strategies.Karafka::Constraintsregistered in two phases::load(require time) and:config(during setup, after contract validation), so feature-specific environment requirements are verified in one place.keyandheadersdeserialization results were not cached when the deserializer returnednilorfalse(e.g. keyless messages), re-running the deserializer on every access. Falsy results are now cached like truthy ones, matching the payload behavior and the documented contract.Pro::Iteratorstored-offsets latch between runs.#mark_as_consumedset an internal@stored_offsetsflag that was never cleared, so once a reused iterator marked anything, the teardown guard fired a spurious blocking synccommit_offsetson the end of every subsequent#each- each running on a brand-new consumer with no stored offsets. The flag is now reset alongside the other per-run state in the#eachensureblock, so it reflects only the current run's marking activity (benign no-op commit removed; no correctness impact).ConsumerGroup#topic=appended unconditionally, so the shared consumer group accumulated up tomultiplex_factorduplicateTopicobjects per discovered topic (polluting the routing tree and slowing linear topic scans, though the per-subscription-group consume path stayed correct). The detector now registers each discovered topic in the shared group only once (under its existing mutex) while still giving every subscription group its ownTopicinstance, mirroring how the multiplexing subscription groups builder dups topics per group at boot.shutdown_timeoutvalues collapsing to a zero-iteration supervision loop. The graceful-shutdown wait computed its iteration count with integer division (shutdown_timeout / 1_000), so any configuredshutdown_timeoutbelow 1000ms (valid config, as long as it exceedsmax_wait_time) floored to zero iterations, skipping the grace period entirely and forcing an immediate forceful shutdown (killing in-flight jobs before their offset commits). The count is now computed with float math andceil, guaranteeing at least one supervision check for any positive timeout. The swarm supervisor shared the same formula (masked by the added grace period) and was fixed the same way.config.internal.processing.*paths (coordinator_class,errors_tracker_class,partitioner_class,strategy_selector,expansions_selector,executor_class,jobs_builder) resolving after the 2.6.0 move toconfig.internal.processing.consumer_groups.*. The shim was shipped by accident; the nesting was announced as having no backward compatibility. Use the nestedconfig.internal.processing.consumer_groups.*paths instead. Requireskarafka-testing >= 2.6.2, which reads these settings from their nested location.PausesManager#@pausesfrom growing unbounded across rebalances.CoordinatorsBuffer#revokereset the pause tracker's retry attempt count on revocation but never removed it, and nothing else pruned the hash outside of critical-error recovery, so aPause(and itsMutex) was retained forever for every distinct topic-partition ever assigned - harmless for static routing (Topicobjects are stable and get reused on reassignment) but unbounded under regex pattern subscriptions with ephemeral, per-discovery topic names.#revokenow removes a tracker outright when it is not currently paused (addedPausesManager#delete), since it carries no state worth preserving; a tracker that is paused is still reset-and-kept, unchanged, because its remaining backoff may need to be honoured again if the same partition is reclaimed (routine under eager rebalancing, where every previously owned partition is revoked and reassigned even when nothing has actually changed for it).TOPIC_ALREADY_EXISTSbroker warning inunexpected_patterns_loop_spec.rb, the same pre-existing broker-side race already allow-listed for other pattern-matched-topic specs, unrelated to this release's other fixes.Pro::Processing::JobsQueue's per-group semaphore accumulating unconsumed#ticksignals under non-blocking/LRJ and async-locking workloads.Pro::Processing::JobsQueue#unlockfrom decrementing@statistics[:waiting]before confirming the job was actually tracked. It unconditionally decremented first and only afterwards checked whether the job was still present in@in_waiting, so a job unlocked after its subscription group had already been reset via#clear(which decrementswaitingitself as part of recovery) both over-decremented the counter a second time and raisedJobsQueueSynchronizationError- the counter corruption was silent while the (still legitimate) synchronization error masked it.#unlocknow only decrements when the job is found, mirroring#unlock_async's existing check-then-act pattern; both raise consistently when the group has already been cleared.Swarm::Node#signal(used by#stop/#terminate/#quiet) could send a signal to a reaped node's stale@pid, which the OS may have already reassigned to an unrelated process. It now returnsfalsewithout signaling if the node is already known to be dead.Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate CLI.