Skip to main content
A synthetic dataset is a dataset filled with generated rows that stand in for real records. This guide shows how to build one with a single workflow. The workflow creates the dataset, fills it with 500 fake customers of a fictional retailer, Summit & Pine Outfitters, takes a sample, and maps the columns to Rosetta Stone. Every run leaves the same rows behind. Use this pattern when you need data that looks real without touching real data, such as:
  • A demo of a product flow, so the audience sees mapped, queryable data
  • A partner trial, before the partner has shared any data of their own
  • A test of an audience builder, a connector, or a downstream workflow

Prerequisites

How the rows are generated

The generator is a single NQL SELECT that reads no source table. It builds a sequence of row numbers, then works out every column from the row number.

A sequence with no source table

Three VALUES lists of digits, joined with CROSS JOIN, produce every combination of a units digit, a tens digit, and a hundreds digit: 10 × 10 × 5 = 500 rows. The expression h.d * 100 + t.d * 10 + u.d + 1 turns each combination into a row number n from 1 to 500:
To change the row count, change the hundreds list. (0), (1) gives 200 rows, and adding a fourth list of digits gives thousands.

Columns worked out from the row number

Each column is a function of n, built from MOD, CASE, and LPAD:
  • LPAD(CAST(n AS VARCHAR), 4, '0') gives a zero-padded key such as 0042, which feeds the customer ID and the email address.
  • MOD(n - 1, 5) picks one of five places, and a CASE turns the place into a state, a ZIP code prefix, and an area code.
  • MOD(n * 61, 100) gives a roll from 0 to 99 that spreads customers across loyalty tiers: 45% bronze, 30% silver, 17% gold, 8% platinum.
  • MOD(n * 97, 540) spreads purchase dates over the 540 days before a fixed date.
Nothing in the query is random, and the dates count back from a fixed timestamp rather than CURRENT_TIMESTAMP. A rerun therefore produces the same rows. The only column that changes is _nio_last_modified_at, which Narrative sets on every write.

Identifiers that are fake by construction

Every identifier comes from a range reserved for fiction, so none of them belongs to a real person: The generator hashes both with SHA2(..., 256) in NQL, so the dataset holds the same shape of identifier a real customer file would. The hash has to be taken over the form the Rosetta Stone attribute expects, or it won’t join: The phone hash in particular is easy to get wrong. A hash of 14155550100 (no +) or 4155550100 (no country code) belongs to a different attribute and won’t join against sha256_hashed_phone_number. For the normalization rules in general, see Hashing PII for Upload.

Step 1: Look up the attribute IDs

Mappings name attributes by ID. Look each ID up with GET /attributes rather than copying it from this page:
Repeat for each attribute you map. Check the attribute’s description before you use it: it says what input the attribute expects. This guide maps these attributes: See GET /attributes for the other search parameters.

Step 2: Validate the generator query

Validate the SELECT on its own before you put it in a workflow. Validation parses and type-checks the query without reading data or starting a job, so a typo shows up in seconds, not halfway through a workflow run. Send the query to POST /nql/validate, or ask your AI assistant to run narrative_nql_validate on it:
The five area codes and the 100 numbers from 0100 to 0199 give 500 distinct phone numbers, one for each row. If you add rows, add area codes and widen MOD(seq.n - 1, 5) and FLOOR((base.n - 1) / 5) to match, or the phone numbers repeat.

Step 3: Write the workflow

The workflow runs five tasks in order: Save this as demo-sample-crm.yaml, replacing <DATA_PLANE_ID> and <COMPUTE_POOL_ID> with your own IDs:

Why the tasks run in this order

The clear step comes before the insert. INSERT appends, so without clearCrm a second run would leave 1,000 rows, and a third would leave 1,500. With the clear step first, every run leaves the same 500 rows. The delete is keyed on a prefix. NQL rejects a DELETE that has no WHERE clause, with the error DELETE can not be used without a WHERE condition. Every generated customer_id starts with c, so LIKE 'c%' matches every row the generator wrote. Pick a prefix that no row you want to keep would share. The sample comes before the mappings. A new mapping is checked against the dataset’s sample, and without one the mapping fails with no sample available. See Mapping Schemas. sampleCrm captures the freshly inserted rows, so mapCrm has rows to check its mappings against. The mapping task tolerates reruns. When a mapping already exists, CreateRosettaStoneMappingsIfNotExist reports it under conflictMappings and leaves it unchanged. It doesn’t fail the task. Each hashed identifier maps to the value property of an object attribute, and the mappings set nothing else. Rosetta Stone derives a hashed_email mapping from sha256_hashed_email on its own, with type filled in. For when to map to an attribute that derives others instead of to a derived one, see Mapping Schemas.

Step 4: Build a second dataset from the first

Once the customer dataset exists, INSERT INTO … SELECT … WHERE can build other datasets from it, such as an audience segment. The source must be a dataset or view your company owns. See INSERT. Append these tasks to the do list to build a segment of high-value customers who bought in the last six months:
With the generator above, the segment holds 39 customers. It follows the same create, clear, fill, sample, and map order as the customer dataset. To add another segment to the same dataset, add another ExecuteDml insert with a different segment_name and WHERE clause before sampleSegments.

Step 5: Create and run the workflow

The workflow’s YAML goes in the specification field of a JSON request body. Set trigger_immediately to start the first run as soon as the workflow is created:
The response’s id is the workflow ID, and because the request triggered a run, it also carries a run_id. See Create a workflow for the full request and response. With the MCP server, ask your assistant to call narrative_workflows_create with the YAML as specification, your data plane ID as data_plane_id, and trigger_immediately: true. To refresh the data later, trigger the same workflow again rather than creating a new one. A workflow name can only be used by one workflow, so a second create with the same specification is rejected. See Trigger a workflow, or use narrative_workflows_trigger.

Step 6: Follow the run

Check the run’s status until it reaches completed or failed:
Each task enqueues its own job. List the run’s jobs to see which task is running and what each finished task returned:
With the MCP server, the same steps are narrative_workflow_runs_list and narrative_jobs_search with workflow_run_id. The sample job’s result reports row_count, the number of rows the sample captured. A sample holds up to 1,000 rows, so for a 500-row dataset it should read 500. After the run completes, the dataset’s mappings list the six attributes from mapCrm, along with the hashed_email mapping Rosetta Stone derives from sha256_hashed_email.

Limitations

Fake identifiers match nothing. No identity graph or partner dataset contains these hashes, so match, expansion, and enrichment results against a synthetic dataset come back empty. Use it to show how a flow works, not what it returns.
  • Runs stop at the first failure. Tasks run one at a time, and a failed task skips everything after it. If fillCrm fails, clearCrm has already emptied the dataset, and it stays empty until a run succeeds. See Handling errors.
  • The dataset’s settings are fixed when it’s created. When the dataset already exists, CreateDatasetIfNotExists returns it unchanged, so a change to tags, description, or schema in the YAML has no effect on a rerun. No workflow task changes the tags of an existing dataset. To change them, edit the dataset in the app or with Update a dataset.

Automating Multi-Step Pipelines

Create, trigger, schedule, and monitor workflows

Task Reference

Parameters and output for every workflow task

Workflow Specification Syntax

The YAML format for documents, tasks, and exports

Mapping Schemas

How mappings connect dataset columns to Rosetta Stone attributes