-
Notifications
You must be signed in to change notification settings - Fork 37
feat(records): aggregate endpoint #2705
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 9 commits
66221d4
dd98ac8
6653552
389eee2
119f28d
b353db4
ce4149c
fb9c35e
d46279c
a607d83
12524ec
6c37d01
100c3c1
23d7777
bed39fb
4b3d786
4b4bf0a
a209fb3
aedf329
9935753
04309fe
a0f7ab7
26cb596
6385ad3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,7 @@ | ||
| from __future__ import annotations | ||
|
|
||
| import asyncio | ||
| from collections.abc import Sequence | ||
| from collections.abc import Mapping, Sequence | ||
| from typing import TYPE_CHECKING, Any, Literal | ||
|
|
||
| from cognite.client._api_client import APIClient | ||
|
|
@@ -11,13 +11,15 @@ | |
| RecordId, | ||
| RecordIdSequence, | ||
| RecordList, | ||
| RecordsAggregation, | ||
| RecordSourceSelector, | ||
| RecordTargetUnit, | ||
| RecordTargetUnits, | ||
| RecordWrite, | ||
| SyncRecord, | ||
| SyncRecordList, | ||
| TimeRange, | ||
| _dump_aggregate_value, | ||
| ) | ||
| from cognite.client.data_classes.filters import Filter | ||
| from cognite.client.utils._experimental import FeaturePreviewWarning | ||
|
|
@@ -235,6 +237,106 @@ async def upsert( | |
| no_response=True, | ||
| ) | ||
|
|
||
| async def aggregate( | ||
| self, | ||
| aggregates: Mapping[str, Any], | ||
| *, | ||
| stream_id: str, | ||
| last_updated_time: TimeRange | None = None, | ||
| filter: Filter | dict[str, Any] | None = None, | ||
| target_units: RecordTargetUnits | Sequence[RecordTargetUnit] | None = None, | ||
| include_typing: bool = False, | ||
| ) -> RecordsAggregation: | ||
| """`Aggregate records from a stream <https://api-docs.cognite.com/20230101/tag/Records/operation/aggregateRecords>`_. | ||
|
|
||
| Args: | ||
| aggregates (Mapping[str, Any]): Aggregate request tree keyed by client-defined aggregate IDs. | ||
| stream_id (str): External ID of the stream to aggregate from. | ||
| last_updated_time (TimeRange | None): Filter records by last-updated time. | ||
| **Required** for immutable streams (must include a lower bound). | ||
| filter (Filter | dict[str, Any] | None): Filter expression. | ||
| target_units (RecordTargetUnits | Sequence[RecordTargetUnit] | None): Unit conversion specification. | ||
| include_typing (bool): Include property type metadata in the response. | ||
|
|
||
| Returns: | ||
| RecordsAggregation: Aggregate results keyed by the requested aggregate IDs. | ||
|
|
||
| Examples: | ||
|
|
||
| Aggregate average temperature using typed helpers: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I also wonder a bit about these examples - they feel like examples of what kind of data that belongs in the Datapoints service, not the Records service. Could you ask Claude to come up with different examples? 😅
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. re-used the examples from the cognite api docs |
||
|
|
||
| >>> from cognite.client import CogniteClient | ||
| >>> from cognite.client.data_classes.data_modeling.records import Avg | ||
| >>> client = CogniteClient() | ||
| >>> res = client.data_modeling.records.aggregate( | ||
| ... stream_id="my-stream", | ||
| ... aggregates={ | ||
| ... "avg_temp": Avg(["my-space", "sensor", "temperature"]), | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's make it more clear that this is a property reference. Ref docs:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. specified the property key in the examples |
||
| ... }, | ||
| ... ) | ||
|
|
||
| Count records with a time range filter: | ||
|
|
||
| >>> from cognite.client.data_classes.data_modeling.records import Count, TimeRange | ||
| >>> res = client.data_modeling.records.aggregate( | ||
| ... stream_id="my-stream", | ||
| ... aggregates={"total": Count()}, | ||
| ... last_updated_time=TimeRange(gte=1705341600000), | ||
| ... ) | ||
|
|
||
| Daily time histogram with nested metric aggregates: | ||
|
|
||
| >>> from cognite.client.data_classes.data_modeling.records import ( | ||
| ... Max, | ||
| ... Min, | ||
| ... TimeHistogram, | ||
| ... ) | ||
| >>> res = client.data_modeling.records.aggregate( | ||
| ... stream_id="my-stream", | ||
| ... aggregates={ | ||
| ... "by_day": TimeHistogram( | ||
| ... ["my-space", "sensor", "timestamp"], | ||
| ... calendar_interval="1d", | ||
| ... aggregates={ | ||
| ... "min_temp": Min(["my-space", "sensor", "temperature"]), | ||
| ... "max_temp": Max(["my-space", "sensor", "temperature"]), | ||
| ... }, | ||
| ... ), | ||
| ... }, | ||
| ... ) | ||
|
|
||
| Unique values with sub-aggregation: | ||
|
|
||
| >>> from cognite.client.data_classes.data_modeling.records import Sum, UniqueValues | ||
| >>> res = client.data_modeling.records.aggregate( | ||
| ... stream_id="my-stream", | ||
| ... aggregates={ | ||
| ... "by_region": UniqueValues( | ||
| ... ["my-space", "sensor", "region"], | ||
| ... aggregates={"total_output": Sum(["my-space", "sensor", "output"])}, | ||
| ... size=10, | ||
| ... ), | ||
| ... }, | ||
| ... ) | ||
| """ | ||
| self._warning.warn() | ||
| body: dict[str, Any] = {"aggregates": _dump_aggregate_value(aggregates)} | ||
| if last_updated_time is not None: | ||
| body["lastUpdatedTime"] = last_updated_time.dump() | ||
| if filter is not None: | ||
| body["filter"] = filter.dump() if isinstance(filter, Filter) else filter | ||
| if target_units is not None: | ||
| body["targetUnits"] = self._dump_target_units(target_units) | ||
| if include_typing: | ||
| body["includeTyping"] = True | ||
|
|
||
| res = await self._post( | ||
| url_path=self._records_url(stream_id, "/aggregate"), | ||
| json=body, | ||
| semaphore=self._get_semaphore("read"), | ||
| ) | ||
| return RecordsAggregation._load(res.json()) | ||
|
|
||
| async def filter( | ||
| self, | ||
| stream_id: str, | ||
|
|
||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we should keep these out of
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. just a heads up it will be inconsistent with the other data classes that are accessible there. Do you want purely aggregate related classes, or all records/streams classes to be in their own module?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've prefixed the response agg types with records, and then i moved the builder types into the records module. Do you want me to move the response types into records as well? |
Uh oh!
There was an error while loading. Please reload this page.