Skip to content

feat(transformations): add OneLake external data source data classes #2714

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Khyat-Cognite wants to merge 23 commits into
base: master
Choose a base branch
from
Draft

feat(transformations): add OneLake external data source data classes #2714

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
7635005
feat(transformations): add OneLake external data source data classes
Khyat-Cognite Jul 9, 2026
bebeafb
fix(transformations): restore MeteringData exports in __init__.py
Khyat-Cognite Jul 9, 2026
ad1e459
fix(ci): restore streamlit pyodide cryptography workaround
Khyat-Cognite Jul 9, 2026
fb8e853
refactor(transformations): simplify external data dump methods
Khyat-Cognite Jul 9, 2026
66dcea8
docs(transformations): clarify OneLake Write models and fix pyodide p…
Khyat-Cognite Jul 9, 2026
6bbc545
Merge branch 'master' into feat/onelake-edc-data-model
Khyat-Cognite Jul 13, 2026
2ec6da3
refactor(transformations): drop Read suffix on OneLake read models
Khyat-Cognite Jul 15, 2026
3192b42
Update cognite/client/data_classes/transformations/external_data.py
Khyat-Cognite Jul 15, 2026
21384d7
Update cognite/client/data_classes/transformations/external_data.py
Khyat-Cognite Jul 15, 2026
64eed27
Update cognite/client/data_classes/transformations/external_data.py
Khyat-Cognite Jul 15, 2026
2d6ead6
refactor(transformations): address haakonvt review on external data m…
Khyat-Cognite Jul 15, 2026
a05483c
fix(transformations): satisfy docstring lint for as_write methods
Khyat-Cognite Jul 15, 2026
d84bb6d
test(transformations): pass client_secret in base as_write tests
Khyat-Cognite Jul 15, 2026
27abac5
test(transformations): fix mypy on ExternalDataSource as_write tests
Khyat-Cognite Jul 15, 2026
b69a069
refactor(transformations): address haakonvt round-2 on external data
Khyat-Cognite Jul 16, 2026
5487e7b
style(transformations): collapse external data re-exports
Khyat-Cognite Jul 16, 2026
9a80848
refactor(transformations): add UnknownExternalDataSource and drop Core
Khyat-Cognite Jul 16, 2026
f7976e8
refactor(transformations): move OneLake settings off base read model
Khyat-Cognite Jul 16, 2026
45c96c8
fix(transformations): satisfy mypy on external data refactor
Khyat-Cognite Jul 16, 2026
2ef4b6a
fix(transformations): return ExternalDataSource from _load dispatch
Khyat-Cognite Jul 16, 2026
4e3151f
fix(transformations): annotate UnknownExternalDataSource.as_write NoR…
Khyat-Cognite Jul 16, 2026
f7888df
fix(transformations): abstract read base, drop with_settings
Khyat-Cognite Jul 16, 2026
41b1cba
refactor(transformations): settings-only OneLakeExternalDataSourceWrite
Khyat-Cognite Jul 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions cognite/client/data_classes/__init__.py
View file
Open in desktop
Comment thread
Khyat-Cognite marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,18 @@
TransformationBlockedInfo,
TransformationDestination,
)
from cognite.client.data_classes.transformations.external_data import (
ExternalDataSource,
ExternalDataSourceList,
ExternalDataSourceUsability,
ExternalDataSourceWrite,
ExternalDataSourceWriteList,
OneLakeCredentialsRead,
OneLakeCredentialsWrite,
OneLakeDataSourceSettingsRead,
OneLakeDataSourceSettingsWrite,
OneLakeLocationDescription,
)
from cognite.client.data_classes.transformations.jobs import (
TransformationJob,
TransformationJobFilter,
Expand Down Expand Up @@ -384,6 +396,11 @@
"EventUpdate",
"EventWrite",
"EventWriteList",
"ExternalDataSource",
"ExternalDataSourceList",
"ExternalDataSourceUsability",
"ExternalDataSourceWrite",
"ExternalDataSourceWriteList",
"ExtractionPipeline",
"ExtractionPipelineConfig",
"ExtractionPipelineConfigRevision",
Expand Down Expand Up @@ -459,6 +476,11 @@
"MeteringDataList",
"MeteringDataPoint",
"OidcCredentials",
"OneLakeCredentialsRead",
"OneLakeCredentialsWrite",
"OneLakeDataSourceSettingsRead",
"OneLakeDataSourceSettingsWrite",
"OneLakeLocationDescription",
"RawTable",
"Relationship",
"RelationshipFilter",
Expand Down
16 changes: 16 additions & 0 deletions cognite/client/data_classes/capabilities.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1104,6 +1104,22 @@ class Scope:
DataSet = DataSetScope


@dataclass
class TransformationsExternalDataSourcesAcl(Capability):
Comment thread
Khyat-Cognite marked this conversation as resolved.
_capability_name = "transformationsExternalDataSourcesAcl"
actions: Sequence[Action]
scope: AllScope | DataSetScope

class Action(Capability.Action): # type: ignore [misc]
Read = "READ"
Write = "WRITE"
Use = "USE"

class Scope:
All = AllScope
DataSet = DataSetScope


@dataclass
class TypesAcl(Capability):
_capability_name = "typesAcl"
Expand Down
Loading
Loading