Staging-Prod DataStream with manual Pruning - #6870
Merged
kajalshah-da merged 2 commits intoAug 20, 2026
Merged
Conversation
…prod tables (#6542) * added a new data stream and code to create datatransfer jobs to load prod tables Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * Added code to stop and start datastreams from config and intergrated PR comments Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * added index file changes Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * Addressed PR comments and refactured the code Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * made extra edits to bigquery.ts Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * added new boundary condition per PR comments Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * restored index and envr.envrs file Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * removed the watermark update criteria as it would always fail being less than the crrent value Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * Addresed PR comments Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * formatting Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * formatting and deleting extra coments Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * Apply suggestions from code review Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com> Signed-off-by: kajalshah-da <kajal.shah@digitalasset.com> Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * minor edits Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * minor edits- removed IAM function Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * Merged with main and fixed all diffs[ci] Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * added partition in publication per issue #6730 Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * added partition in publication per issue #6730 [ci] Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> * update expected Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> --------- Signed-off-by: Kajal Shah <kajal.shah@digitalasset.com> Signed-off-by: kajalshah-da <kajal.shah@digitalasset.com> Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com> (cherry picked from commit f56f841) Signed-off-by: Kajal <kajal.shah@digitalasset.com>
… expiry (#6813) * added purge datatransfer job for manually deleting records after 7 days from staging tables Signed-off-by: Kajal <kajal.shah@digitalasset.com> * Addressed PR comments [ci] Signed-off-by: Kajal <kajal.shah@digitalasset.com> * Addressed PR comments - sharing IAMMember [ci] Signed-off-by: Kajal <kajal.shah@digitalasset.com> * Addressed PR comments - sharing IAMMember with defaul removed for 7 days [ci] Signed-off-by: Kajal <kajal.shah@digitalasset.com> * Addressed PR comments - sharing IAMMember with defaul removed for 7 days [ci] new Signed-off-by: Kajal <kajal.shah@digitalasset.com> * fixed zod error messages [ci] Signed-off-by: Kajal <kajal.shah@digitalasset.com> * npm run fix changes [[ci] Signed-off-by: Kajal <kajal.shah@digitalasset.com> * formatting cahnges to cloudArmor.ts [ci] Signed-off-by: Kajal <kajal.shah@digitalasset.com> * [ci] Signed-off-by: Kajal <kajal.shah@digitalasset.com> --------- Signed-off-by: Kajal <kajal.shah@digitalasset.com> (cherry picked from commit a860953) Signed-off-by: Kajal <kajal.shah@digitalasset.com>
Contributor
[backport] ReminderPlease consider backporting to the following branches:
And your PR is currently against base branch: release-line-0.7.3. Note: Any PR comment containing [backport] will be considered for auto-backporting upon merge, |
stephencompall-DA
approved these changes
Aug 19, 2026
stephencompall-DA
left a comment
stephencompall-DA
left a comment
Contributor
There was a problem hiding this comment.
Just a cherry-pick backport 👍
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This back ports #6542 and #6813 to 0.7.3
Part of DACH-NY/canton-network-internal#6409
Pull Request Checklist
Cluster Testing
/cluster_teston this PR to request it, and ping someone with access to the DA-internal system to approve it./upgrade_teston this PR to request it, and ping someone with access to the DA-internal system to approve it./hdm_teston this PR to request it, and ping someone with access to the DA-internal system to approve it./lsu_teston this PR to request it, and ping someone with access to the DA-internal system to approve it.PR Guidelines
Fixes #n, and mention issues worked on using#nMerge Guidelines