Components and data flow

operational and reference sources
              ↓
  contract, quality, and mastering
              ↓
       governed golden records
              ↓
 applications, analytics, and delivery targets

Logical components

The logical model connects Golden configuration, record processing, and customer-visible operations.

Data intake

Data enters through a configured source, a supported file workflow, table operation, or a record API. Credentials are reusable configuration and should be separated from resource definitions and ordinary application logs.

Choose intake according to volume and ownership:

• a record API for controlled individual interactions;
• a configured source and table operation for extracts or larger sets; and
• the web application for authorized human record work.

Data contract and preparation

A dataset defines fields, types, identity, validation, and nested structure. A transformation maps one record shape to another. A pipeline applies an ordered processing flow. These are separate architectural concerns even when a small implementation uses only a dataset.

The target table stores records following the dataset. Its configuration also controls customer-visible behavior such as editability and optional history or audit. A completed load means the operation completed; it does not prove that every input value was accepted unchanged.

Entity

An entity is the managed context for a business subject. It connects the table to the capabilities enabled for that subject. Entity types can provide storage, search, duplicate resolution, or automatic stewardship.

Treat an entity identifier as an integration contract. Keep it stable after callers or destinations begin depending on it. Treat entity state as runtime evidence: a request that starts synchronization is not proof that the entity is ready.

Search and data quality

Search uses the entity’s record contract and configured index behavior. It accepts structured record criteria or free text through their respective endpoints; neither is a general SQL query interface.

Data-quality metadata reports how the stored record relates to configured validation rules. Consumers should use quality facts to understand actionable conditions instead of reverse-engineering the aggregate score.

Resolution policy

Resolution resources divide responsibility:

These resources express approved behavior. Their private algorithms are not a public integration contract.

Candidate groups and stewardship

Candidate groups expose records and comparison evidence as buckets or logical clusters. Depending on classification and policy, Golden can require a human decision or apply configured automatic stewardship.

A steward action is a data-governance decision, not merely a user-interface state. It can change mastered records and therefore affect downstream systems. The architecture must define acceptable evidence, required comments, recovery, and escalation.

Golden records and delivery

A golden record is the mastered outcome with a stable record context and contributing lineage. Consumers can read it through supported record APIs or receive records through configured destinations.

Delivery is a separate boundary from mastering. A successful mastering action does not prove every downstream target has accepted the result. Use destination events and destination-side verification where the business process requires confirmation.

Cross-cutting control and evidence

• Users and tokens identify human and unattended callers.
• Roles authorize public capabilities; data grants restrict entities and rows.
• Tasks expose progress for asynchronous work.
• Schedules define future task starts.
• Product events expose pending or failed configured delivery.
• Audit, history, and trace identifiers provide different kinds of evidence and must not be treated as interchangeable.

See Identity, quality, and authority for design decisions across quality and mastering workflows.

System context

Golden connects source systems with applications and people that consume validated or mastered records.

Public interfaces

• The web application supports stewardship and administration tasks.
• The REST API supports authenticated application and automation flows.
• Configured sources and destinations move records through supported data flows.
• Tasks and product events expose asynchronous progress and delivery outcomes.

Trust boundaries

Every integration crosses a trust boundary and should answer:

• Which identity or token is used?
• Which role provides the minimum required capability?
• Which fields leave the source system and which destination receives them?
• How are failed tasks or deliveries detected and resolved?
• Which system remains authoritative before and after mastering?

See Golden security and Golden scenarios before choosing an integration pattern.

Resources and asynchronous work

Resources describe reusable product configuration. Tasks represent work that does not complete within one interactive request.

Resources

An entity can refer to resources that define different parts of its data flow:

Resources are referenced by identifier. Administrators should treat resource changes as changes to every entity or data flow that depends on them. Validate a resource before attaching it to a production workflow, and review its dependencies before renaming or deleting it.

Tasks

Entity synchronization, data movement, and other long-running operations are represented as tasks. A task can move through these statuses:

PENDING → RUNNING → SUCCEEDED
                  ↘ FAILED
          CANCELLING → CANCELLED

A run can also be SKIPPED. See the full status reference. Cancellation is a request. The final state tells you whether the task stopped. Applications should follow the task rather than assume the request that started it means the work has completed.

Schedules

Some task definitions can run on a schedule or be triggered immediately by an administrator. A schedule and a task instance are different:

• The schedule defines when future runs should start.
• A task instance records one execution and its outcome.

Learn how to follow tasks and product events.