Adaptive Frequency

The generic_scheduler recomputes each task's run interval after every cycle based on how many consecutive idle cycles it has seen. The function used is identified by the adaptive_frequency column on the row.

Definitions live in syncs-schedulers/generic_scheduler/frequency_computation.py.

Column naming (misleading)

Column	Actually means
frequency	Interval in seconds between runs (lower = faster)
adaptive_frequency	The function ID (int) used to recompute frequency after each cycle

Function ladder

Functions 5–12 share the same warm-up shape — they differ only in the idle cap:

inactive_processes	Returned frequency
<= 1	1 (every 1s)
<= 3	5 (every 5s)
<= 7	60 (every 1 min)
else	idle cap (see below)

Fn ID	Idle cap	Notes
0 / null	(no change)	static, no adaptation
1	(no change)	legacy, unused
2	15s	specialized for read_ping (different ladder)
4	60s (1 min)
5	300s (5 min)
6	10 — bug, comment says "10 minutes" (600)	avoid until fixed
7	1800s (30 min)
8	3600s (1 hr)	webhook-backed pings (e.g. Airtable webhook_ping)
9	7200s (2 hr)
10	21600s (6 hr)
11	43200s (12 hr)
12	86400s (1 day)	dormant background pings

How inactive_processes is updated

In generic_scheduler/scheduler.py:

inactive_processes = 0 if process_was_active else inactive_processes + 1

• process_was_active = response_data.get("active", False) — connectors set active=True only when they did real work that cycle.
• Thresholds (<= 1, <= 3, <= 7) are cycle counts, not seconds.

Concurrency-safe writeback

frequency = CASE WHEN t.frequency = v.current_frequency
                 THEN v.new_frequency
                 ELSE t.frequency
            END

The new frequency writes back only if the row hasn't changed since the scheduler read it. External services (e.g. event-forward bumping frequency to 1 on webhook activity) are not clobbered by stale ramp-ups.

Escape hatches in the task response

Mechanism	Effect
freeze_frequency_until (timestamp)	Skip frequency recomputation until that time
max_records_sync_delay_in_seconds_to_adjust_frequency_and_adaptive_frequency	Adjust frequency to honor a max-delay SLA
minimum_frequency / minimum_adaptive_frequency_function_id (in row data)	Floor for the above adjustment

Picking a function for a new task

Requirement	Pick
Webhook fallback poll, dead webhook detected within ≤ 1 hr	8
Polling-only, freshness matters	5 or 7
Dormant background, ≤ 1 day delay acceptable	12

Observability

metric_distribution("task_frequency", new_frequency, unit="second", region=region_name)

Emitted every cycle. Graph the distribution per type to confirm tasks settle at the expected idle cap.

Migrating an existing row to a different function

Changing the function ID in the insert site only affects new rows. To retroactively cap an existing row:

UPDATE schedule_generic
SET adaptive_frequency = 8,
    frequency          = LEAST(frequency, 3600)
WHERE type = 'webhook_ping'
  AND adaptive_frequency = 12
  AND custom_unique_constraint LIKE 'whp_%';