Problem
The aws_s3 input, when SQS-driven, throttles empty-queue polling with a hardcoded 500ms between ReceiveMessage calls (internal/impl/aws/s3/input.go, sqsTargetReader.Pop). With short polling that is ~2 requests/s/instance on an idle queue; even with wait_time_seconds: 20 (the SQS maximum) the floor is ~1 request / 20.5s. There is no way to back off further.
For deployments that poll SQS from outside the AWS account (e.g. across a NAT gateway), this idle traffic adds avoidable request volume and NAT data-processing at fleet scale.
Proposal
Add an optional sqs.idle_poll_period duration field. When set (> 0) it replaces the 500ms throttle applied after an empty receive; the default 0s preserves current behaviour exactly. It stacks after wait_time_seconds long-polling and trades first-message latency for fewer requests — suited to batch S3 ingestion.
This mirrors the idle-sleep pattern common in hand-rolled SQS consumers.
Problem
The
aws_s3input, when SQS-driven, throttles empty-queue polling with a hardcoded500msbetweenReceiveMessagecalls (internal/impl/aws/s3/input.go,sqsTargetReader.Pop). With short polling that is ~2 requests/s/instance on an idle queue; even withwait_time_seconds: 20(the SQS maximum) the floor is ~1 request / 20.5s. There is no way to back off further.For deployments that poll SQS from outside the AWS account (e.g. across a NAT gateway), this idle traffic adds avoidable request volume and NAT data-processing at fleet scale.
Proposal
Add an optional
sqs.idle_poll_periodduration field. When set (> 0) it replaces the 500ms throttle applied after an empty receive; the default0spreserves current behaviour exactly. It stacks afterwait_time_secondslong-polling and trades first-message latency for fewer requests — suited to batch S3 ingestion.This mirrors the idle-sleep pattern common in hand-rolled SQS consumers.