On this page
Source configuration
Configure the supported file, JDBC, and Golden-table sources that supply records to Trazadera Golden data flows.
A source describes where a Golden table load reads records. Golden supports file, JDBC, and existing-table sources.
File sources
source-file supports CSV and JSON formats.
{
"type": "source-file",
"_id": "customer_csv_source",
"description": "Customer CSV extracts",
"dataset": "customer_dataset",
"inputPattern": "customers*.csv",
"format": "CSV",
"separator": ",",
"header": true,
"ignoreQuotes": false
}
| Property | Purpose |
|---|---|
dataset | Structure produced by the source |
inputPattern | Stored filename or name pattern for files uploaded to Golden |
format | CSV or JSON |
separator | CSV separator; default ; |
header | Whether CSV includes a header row |
ignoreQuotes | Controls CSV quote handling; default true |
inputPattern is matched against complete stored filenames, not a filesystem
path. * matches any run of characters and ? matches exactly one character.
For example, customers*.csv matches customers-01.csv but not
a_customers.csv. Golden reads every matching file in turn as one record
sequence. Upload the files before running the source.
CSV files without a header map values to dataset columns by position. With a header, names must match dataset column keys; undeclared columns are dropped. CSV cannot represent nested dataset columns, so use JSON when the dataset has them. JSON input must be one array of objects; validate representative file sizes in the target environment before adopting it for a large exchange.
JDBC sources
{
"type": "source-jdbc",
"_id": "orders_jdbc_source",
"description": "Orders database",
"dataset": "order_dataset",
"url": "jdbc:postgresql://db.example.com:5432/orders",
"credentials": "orders_credentials",
"table": "public.orders",
"timestampFilteringFlag": true,
"timestampFilteringColumn": "updated_at",
"readTimestampFlag": false,
"readIdFlag": false,
"customScripts": false,
"properties": [{"key": "ssl", "value": "true"}]
}
| Property group | Purpose |
|---|---|
url, credentials, properties | Connection and driver settings |
table, count | Source table and optional expected count behavior |
timestampFilteringFlag, timestampFilteringColumn | Timestamp-based filtering support |
readTimestampFlag, readTimestampColumn | Read a timestamp value from the source |
readIdFlag, readIdColumn | Read a source identifier column |
customScripts, queryScript, countQueryScript | Use approved custom read queries |
Reference a credentials resource rather than embedding a user name or password. Connection properties are name/value pairs passed to the JDBC driver; the driver determines which names it accepts. See Configure connection credentials for the supported credential-type mapping. When timestamp filtering is enabled, verify that the selected source column changes for every source update you expect Golden to load.
Golden-table sources
{
"type": "source-table",
"_id": "staging_customer_source",
"description": "Customer staging table",
"table": "staging_customers"
}
A table source reads a table already managed by Golden. Its dataset comes from
that table, so it does not declare a separate dataset property.
Verify a source
Test the resource before saving it, then run a controlled table load and confirm the returned record count and field shape. Source configuration alone does not start ingestion; an administrator invokes or schedules the associated data operation.