On this page
Integrate sources and consumers
Compare Golden web, REST, batch, configured delivery, and hybrid integration approaches by latency, volume, authority, and recovery.
Choose a Golden integration shape from the business interaction, not from the availability of an endpoint. Latency, volume, authority, consistency, and recovery ownership determine the right public interface.
Decision table
| Need | Primary approach | Completion evidence | Main trade-off |
|---|---|---|---|
| A person finds or reviews data | Web application | Visible record or confirmed stewardship result | Human throughput and policy consistency |
| An application looks up a subject | Synchronous record search API | Search response and caller decision | Golden becomes a runtime dependency |
| An application creates or corrects one record | Controlled single-record upsert | Write response, then index status where applicable | Conflict and ambiguous-retry handling |
| A system supplies a large extract | Configured source or table load | Returned task reaches a terminal state; records are verified | Eventual rather than request-time completion |
| Golden supplies mastered data | Configured destination | Product event state and receiver-side verification | Receiver must tolerate safe redelivery where supported |
| Automation changes configuration | REST API under controlled administration | Read-back plus any resulting task | Broad role and change impact |
Synchronous lookup
Use record-shaped search when an application needs a decision during an interactive or service flow. The application must define:
- timeout and latency budget;
- fail-open, fail-closed, or review behavior;
- the approved search fields and options;
- how it interprets zero, one, or several results; and
- whether the returned data may be cached.
Do not turn an empty result into an unconditional create without considering search limitations and eventual indexing.
Controlled individual writes
Use single-record upsert for individual application interactions. Use a configured source and table load for batch intake. Keep source identity stable, apply least privilege, validate conflicts as data-state decisions, and check index status before diagnosing a new record as missing from search.
A timeout during a write can leave the caller uncertain whether the change was applied. Use stable identifiers and read-back reconciliation before retrying a non-idempotent operation.
Batch intake and synchronization
Use configured sources, files, table loads, transformations, and entity synchronization for extracts or larger sets. The starting request normally creates work; the task’s terminal state reports completion.
Define:
- extraction scope or watermark;
- behavior for late, repeated, or removed source records;
- representative-data validation before production;
- whether partial record-quality issues are retained or rejected;
- schedule ownership and overlap prevention; and
- restart or cleanup after a failed task.
Managed downstream delivery
Use configured HTTP, Kafka, JDBC, or Golden-table destinations when Golden should deliver managed results. Treat delivery as its own reliability boundary.
The receiver should use stable record identity and tolerate a safe repeat where the protocol and business operation allow it. Product events expose Golden’s customer-visible delivery state; the receiver remains authoritative for whether it applied the result.
Web application and human decisions
Use the web application for stewardship and customer administration. Human review is the correct architectural choice when evidence is contextual, false merges are costly, or policy is still being validated.
Human workflow still needs an integration contract: queue ownership, access, expected decision time, comment policy, downstream impact, and recovery path. See Work with data.
Hybrid patterns
Most production designs combine approaches. A common flow is:
scheduled source load → task completion → entity synchronization
→ automatic classification
→ human review for uncertain candidates
→ configured mastered-data delivery
→ synchronous lookup by applications
For a hybrid design, identify the authoritative completion point for each arrow. Avoid one global “Golden succeeded” status that hides ingestion, indexing, stewardship, and delivery as separate states.
Interface and credential selection
| Caller | Credential | Typical minimum role |
|---|---|---|
| Human reader | User session | VIEWER |
| Human steward | User session | STEWARD |
| Search integration | Dedicated access token | VIEWER |
| Record-write integration | Dedicated access token | STEWARD |
| Configuration or synchronization automation | Dedicated access token | ADMIN |
Roles are global to the deployment. Use separate tokens for routines with different privilege and lifecycle rather than giving one integration a shared administrator credential.
Use Golden integration patterns for representative API exchanges and the generated API reference for exact operations.