example(connector_sdk) : re-raised new connector#39
example(connector_sdk) : re-raised new connector#39fivetran-JenasVimal wants to merge 20 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new people_ai Connector SDK example (ported from an older repo) that authenticates to the People.ai API via OAuth2 client credentials, paginates through activities endpoints, and upserts results into destination tables.
Changes:
- Added
people_ai/connector.pyimplementing OAuth token acquisition, retry/backoff, pagination, and upserts. - Added
people_ai/README.mddocumenting usage, configuration, and produced tables. - Added
people_ai/configuration.jsontemplate plus a (currently empty)people_ai/requirements.txt.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 22 comments.
| File | Description |
|---|---|
| people_ai/connector.py | Implements the People.ai API sync logic (auth, pagination, retries, upserts, checkpointing). |
| people_ai/README.md | Documents connector behavior, setup, configuration parameters, and tables. |
| people_ai/configuration.json | Provides placeholder configuration keys for API credentials. |
| people_ai/requirements.txt | Added file intended for dependencies (currently empty). |
| except requests.exceptions.HTTPError as e: | ||
| # The retry logic is now inside get_page, | ||
| # so an error here means it failed permanently | ||
| log.severe(f"Permanent failure fetching base activities" f" at offset {offset}: {e}") |
fivetran-sahilkhirwal
left a comment
There was a problem hiding this comment.
changes look good
For tests, please rebase after this is merged: #30
0523275 to
8c3fc1b
Compare
3e46920 to
fd7caa1
Compare
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
Was unable to test this as I dont have credentials . Verified the People.ai endpoints are valid/routable. The activity endpoints return |
| Returns: | ||
| int: The total number of records successfully processed (upserted). | ||
| """ | ||
| offset = 0 |
| if len(page) < limit: | ||
| break | ||
|
|
||
| offset += limit |
| state["activity_offset"] = offset + len(page) | ||
| op.checkpoint(state) | ||
|
|
||
| if len(page) < limit: | ||
| break | ||
|
|
||
| offset += limit |
| except Exception as e: | ||
| log.error(f"Failed to refresh token: {e}") | ||
| raise e # If re-auth fails, raise the exception |
| # Save the progress by checkpointing the state. This is important for ensuring that the sync process can resume | ||
| # from the correct position in case of next sync or interruptions. | ||
| # Learn more about how and where to checkpoint by reading our best practices documentation | ||
| # (https://fivetran.com/docs/connectors/connector-sdk/best-practices#largedatasetrecommendation). | ||
| state["activity_offset"] = offset + len(page) | ||
| op.checkpoint(state) |
| required_configs = ["api_key", "api_secret"] | ||
| for key in required_configs: | ||
| if key not in configuration: | ||
| raise ValueError(f"Missing required configuration value: {key}") |
| try: | ||
| access_token = get_access_token(api_key, api_secret) | ||
| except requests.exceptions.RequestException as e: | ||
| log.error(f"FATAL: Initial authentication failed: {e}") | ||
| return # Stop the sync |
| - **[oura_ring](https://github.com/fivetran/community_connectors/tree/main/oura_ring)** - This example shows how to sync health and wellness data from the Oura Ring API v2 using Connector SDK. It syncs daily activity, sleep, readiness, stress, and heart rate data with incremental syncing, cursor-based pagination, automatic flattening of nested contributor objects, and date-range chunking for high-volume heart rate data. You need to provide your Oura Personal Access Token for this example to work. | ||
| - **[owasp_api_vulns](https://github.com/fivetran/community_connectors/tree/main/owasp_api_vulns)** - Sync OWASP API vulnerability data from NVD 2.0 | ||
| - **[partech](https://github.com/fivetran/community_connectors/tree/main/partech)** - Sync POS data from Partech (formerly Punchh) | ||
| - **[people_ai](https://github.com/fivetran/community_connectors/tree/main/people_ai)** - Sync activity and participant data from People.ai using OAuth2 client credentials |
| try: | ||
| access_token = get_access_token(api_key, api_secret) | ||
| except requests.exceptions.RequestException as e: | ||
| log.error(f"FATAL: Initial authentication failed: {e}") |
There was a problem hiding this comment.
Why are we silently skipping here? We need to fail if we are unable to authenticate. On line 188, we call this method and assign the return value to : current_token = reauth_func()
Instead should we directly fail?
| the specified activity type. | ||
| """ | ||
| state_offset_key = f"{activity_type}_offset" | ||
| offset = state.get(state_offset_key, 0) |
There was a problem hiding this comment.
The key read here is different than the key that is being checkpointed on line 357, how is it consistent?
|
@codex review |
fivetran-satvikpatil
left a comment
There was a problem hiding this comment.
Please address all the comments given by the copilot/codex.
| - `schema()` defines the `participants` table with primary key `uid`, `email`. | ||
| - `sync_base_activities()` copies each base activity record and renames a `subject` field to `api_subject` when present. | ||
| - `sync_activity_type()` writes participant records to the `participants` table. | ||
| - The connector stores the most recently processed page offsets in `state["activity_offset"]` and `state["participants_offset"]` after each successful page write. |
There was a problem hiding this comment.
Where are we using state["participants_offset"]?
Description of Change
Re-raising this connector from old repo
RD-1237871
Testing
This is actively being used by a customer. Also, tested locally this morning here

Checklist
Some tips and links to help validate your PR:
fivetran debugcommand.