Skip to content

feature(connectors/bright_data_serp): add Bright Data SERP connector#38

Open
adomhamza wants to merge 2 commits into
fivetran:mainfrom
adomhamza:feat/bright_data_serp
Open

feature(connectors/bright_data_serp): add Bright Data SERP connector#38
adomhamza wants to merge 2 commits into
fivetran:mainfrom
adomhamza:feat/bright_data_serp

Conversation

@adomhamza

@adomhamza adomhamza commented Jul 2, 2026

Copy link
Copy Markdown

Description of Change

`This is my first contribution to the Fivetran Connector SDK repository. It adds a Bright Data SERP connector that syncs search engine results from Bright Data's SERP REST API into a Fivetran destination.

The connector supports Google, Bing, and Yandex, accepts multiple search_query formats (single value, comma-separated, newline-separated, JSON array string), flattens nested JSON responses, and upserts rows to a search_results table keyed by query and result_index.

What's included

  • connector.py — schema(), update(), checkpointing, and query parsing
  • helpers/search.py — Bright Data API calls with retry/backoff
  • helpers/data_processing.py — flattening, field discovery, and upsert logic
  • helpers/validation.py — configuration validation
  • README.md — connector documentation per the repo template
  • configuration.json — placeholder values for required config fields

Design decisions (aligned with SDK guidelines)

  • Primary keys and types declared in schema(); remaining columns inferred from data
  • No fields.yaml / PyYAML — field discovery happens at runtime via collect_all_fields()
  • No extra requirements.txt dependencies (uses pre-installed fivetran_connector_sdk and requests)
  • Required connector log at start of update(): log.warning("Example: Connectors : Bright Data SERP")`

Testing

test1 test2 test3

Checklist

Some tips and links to help validate your PR:

  • Tested the connector with fivetran debug command.
  • Added/Updated example-specific README.md file, see the README template for the required structure and guidelines.
  • Followed Python Coding Standards, refer here

@adomhamza adomhamza marked this pull request as draft July 2, 2026 10:21
@adomhamza adomhamza marked this pull request as ready for review July 2, 2026 10:24
@adomhamza

Copy link
Copy Markdown
Author

Hi @fivetran-JenasVimal @fivetran-anushkaparashar,
As requested the PR in the new Community Connectors repository.
Thanks.

@fivetran-JenasVimal

Copy link
Copy Markdown
Contributor

Hi @adomhamza,

Could you please rebase your PR so that the latest checks can run and pass?

Thank you!

@adomhamza

Copy link
Copy Markdown
Author

Hi @adomhamza,

Could you please rebase your PR so that the latest checks can run and pass?

Thank you!

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants