Skip to content

HubSpot Associations Incremental Sync — Development Log

Progress

Task Status Notes
Full scan method ✅ Done Iterates all parent IDs
Incremental method (rollup-based) ✅ Done Searches parent lastmodifieddate
Rollup field detection in setup ✅ Done Alphabetical prefix matching
Rollup field health check (hourly) ✅ Done schedule_generic, 1h frequency
Alphabetical ordering fix ✅ Done PR #1507, 2026-04-06
Custom object CDC Boost support ⚠ Risk Naming convention uses object type IDs (e.g., 2-4029593) — impractical for users

Alphabetical Ordering Fix (2026-04-06)

Problem

Our docs tell users to create rollup fields on the alphabetically first object (e.g., companies before contacts). But setup.py used matched_table['external_table_id'] as the parent, which depends on which table was matched first during setup — not alphabetical order.

For contacts↔companies:

  • Docs say: create on companies with prefix sync_associations_companies_contacts_label_*
  • Code searched: contacts properties with prefix sync_associations_contacts_companies_label_*
  • Result: rollup fields not detected, supports_roll_up = false

Fix

PR #1507 (fc9e68ed) — Two files changed:

setup.py: Sort the two object IDs alphabetically before building the prefix and querying HubSpot properties:

obj_id_1 = matched_table['external_table_id']
obj_id_2 = table["table_options"]["object_type_id_2"]
rollup_parent_id, rollup_child_id = sorted([obj_id_1, obj_id_2])

Also stores rollup_fields_object_id in table_options so the health check knows which object to query.

rollup_fields.py: Health check uses rollup_fields_object_id (with fallback to association_object_external_table_id for old syncs):

parent_external_table_id = table['table_options'].get(
    'rollup_fields_object_id',
    table['table_options'].get('association_object_external_table_id')
)

Why it's safe

For all previously working tables, the alphabetical sort produces the same result:

Association Old parent Alphabetically first Same?
companies↔deals companies companies Yes
contacts↔deals contacts contacts Yes
deals↔products deals deals Yes
contacts↔companies contacts companies No — this is the fix

What was NOT changed

  • association_object_external_table_id — unchanged, still used by incremental read pings
  • associations_incremental.py — no changes
  • switch_app.py — no changes
  • read_ping.py routing — no changes

Client Debugging: GPF (2026-04-06)

Context

Client Walter Goins enabled CDC Boost on UAT and Prod HubSpot environments. We verified the setup.

How we verified

  1. Queried HubSpot Properties API to see which rollup fields exist:
curl -s "https://api.hubapi.com/crm/v3/properties/companies" \
  -H "Authorization: Bearer $TOKEN" \
  | jq '[.results[] | select(.label | startswith("sync_associations_companies"))] | .[].label'

Repeat for contacts and deals.

  1. Compared against table_options.rollup_field_labels in the base schema.

  2. Found mismatches:

    • companies↔contacts: rollup fields existed in HubSpot but not detected (alphabetical bug)
    • contacts_deals_label_plaintiff and companies_deals_label_provider: new labels added after last sync save
    • deals↔line_items: rollup field existed but not detected (never re-saved)

Note

The client also hit orphaned Postgres CDC triggers and stale record locks during this session — those are general Postgres connector issues, not specific to association CDC Boost. See Client Problem Solving for details.

Custom Object Limitations

Custom objects (e.g., 2-4029593 for "batch") are technically eligible for CDC Boost — they're not engagement objects. But the naming convention becomes impractical:

  • The rollup field would need to be named sync_associations_companies_2-4029593_label_none
  • Users see numeric IDs, not friendly names
  • Not documented, not tested with clients

Engagement objects (calls, emails, meetings, notes, tasks, communications, postal_mail) cannot use CDC Boost — HubSpot doesn't allow rollup properties on them.

  • fc9e68ed (2026-04-06): "Refactor rollup field handling in HubSpot setup and verification" — alphabetical ordering fix
  • d0109c05 (2026-04-06): Merge PR #1507
  • 47bef0c5 (2025-12-15): "Associations read incremental" — initial implementation