Creating Derivation Rules

This guide shows how to create a derivation rule through the API, check that it produces the derived mappings you expect, and take it out of use. For what rules are and how queries use them, see Attribute Derivations.

Prerequisites

An API token with an admin grant on attribute_derivations. Reading rules needs only read. See Permissions.
The ids of the source and target attributes.
map access on the target attribute for your company.

The examples build a rule that computes sha256_hashed_email from normalized_email. The attribute ids, company id, and dataset id are placeholders. Replace them with your own.

Check what already exists

Only one of your company’s rules can connect a given source and target, and a rule shared with you may already do the job. List the rules between the two attributes before creating one:

curl -H "Authorization: Bearer $TOKEN" \
  "https://api.narrative.io/derivations?source_attribute_id=291&target_attribute_id=76"

An empty records array means no rule you can use connects them.

Write the transformation

The transformation is a mapping whose expressions read the source attribute through $source. Which form you write depends on the types of the two attributes. Fetch both to see their shapes:

curl -H "Authorization: Bearer $TOKEN" \
  "https://api.narrative.io/attributes/normalized_email"

Source attribute	Target attribute	Write
Primitive	Primitive	A `value_mapping` that reads `$source`
Object	Primitive	A `value_mapping` that reads properties as `$source.<property>`
Primitive or object	Object	An `object_mapping` with one expression per target property, or a `value_mapping` whose single expression produces the whole object

A value_mapping for two string attributes:

{
  "type": "value_mapping",
  "expression": "SHA2($source, 256)"
}

Keep these rules in mind:

$source must come first in a reference. $source.value is valid; an identifier with $source anywhere after the first position is rejected.
An object_mapping must cover every required property of the target and must not name properties the target does not have.
cached_mapping is not accepted.
You can use any transformation function, including CURRENT_DATE and CURRENT_TIMESTAMP. The expression runs each time a query uses the rule.

A rule that reads $source.<property> only applies to datasets that map the source attribute with an object_mapping. A rule that reads bare $source only applies to datasets that map it with a value_mapping. Match the form that providers of the source attribute actually use.

Create the rule

Send the rule to POST /derivations. Set lossy when the transformation discards information and imprecise when it is approximate, and use fidelity_note to explain either flag to people deciding whether to accept derived data.

curl -X POST "https://api.narrative.io/derivations" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "normalized_email_to_sha256_hashed_email",
    "description": "Hashes the normalized email address with SHA-256.",
    "source_attribute_id": 291,
    "target_attribute_id": 76,
    "company_id": 42,
    "mapping": {
      "type": "value_mapping",
      "expression": "SHA2($source, 256)"
    },
    "lossy": true,
    "imprecise": false,
    "fidelity_note": "Hashing is one-way, so the original address cannot be recovered.",
    "collaborators": { "use": { "type": "none" } }
  }'

company_id must be your own company. A 201 response returns the rule with its id. The transformation is validated before the rule is written. A 400 lists every problem found, so you can fix them in one pass. A 409 means your company already has a rule with that name or between those two attributes. See the Derivations API reference for the full list. If the response includes a cyclic_derivation warning, the rule was still created. The warning tells you the new rule closes a loop with existing rules, which is allowed. See Why cycles are allowed. A new rule with "type": "none" applies only to your own company’s queries. To let other companies use it, update collaborators with their ids:

curl -X PATCH "https://api.narrative.io/derivations/57" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "collaborators": { "use": { "type": "inclusion", "company_ids": [17, 92] } }
  }'

Sharing with every company is reserved for rules owned by Narrative.

Confirm the rule works

Check that the rule appears where you expect in the derivation graph:

curl -H "Authorization: Bearer $TOKEN" \
  "https://api.narrative.io/attributes/normalized_email/derivation-graph?direction=descendants"

The target attribute should appear in nodes with "relationship": "descendant", and your rule should appear in edges. Then check that a dataset mapped to the source attribute gains a derived mapping. Ask for the dataset with only its derived mappings:

curl -H "Authorization: Bearer $TOKEN" \
  "https://api.narrative.io/datasets/123?mapping_source=derived"

Look in mappings for a row whose attribute_id is the target attribute and your rule’s id in provenance.derivation_path. Its mapping.expression shows the dataset’s own expression substituted for $source. Finally, query the target attribute:

SELECT
  company_data."123"._rosetta_stone.sha256_hashed_email
FROM company_data."123"

If no derived mapping appears

Check	Why it matters
The dataset has an active stored mapping to the rule’s source attribute	A rule applies to stored mappings only. It does not chain from another derived mapping
The dataset has no stored mapping to the target attribute	A stored mapping always takes the place of a derived one
The stored mapping is not a `cached_mapping`	A cached mapping has no expression to substitute for `$source`
The rule’s `$source` form matches the stored mapping’s type	`$source.<property>` needs an `object_mapping`; bare `$source` needs a `value_mapping`
The stored `object_mapping` maps every property the rule needs for required target properties	A missing property that feeds a required target property stops the rule applying to that dataset
The rule is `active` and your company owns it or is listed in its collaborators	Inactive and unshared rules are never applied
Another rule with a lower `cost` reaches the same target	The cheapest rule wins

Change a rule

PATCH /derivations/{derivation_id} changes only the fields you send. A changed mapping is validated the same way as on create. You cannot change source_attribute_id or target_attribute_id. Create a new rule instead. Because derived mappings are computed when they are needed, a change applies to every query that runs after it. There are no stored mappings to rebuild.

Retire a rule

To stop a rule being applied while keeping it, set it inactive:

curl -X PATCH "https://api.narrative.io/derivations/57" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "active": false }'

To remove it permanently, delete it:

curl -X DELETE "https://api.narrative.io/derivations/57" \
  -H "Authorization: Bearer $TOKEN"

Either action removes the derived mappings the rule produced, for your company and every company it was shared with. Queries for the target attribute stop returning rows from datasets that relied on the rule.

Attribute Derivations

How rules become mappings and which mapping wins

Derivations API

Every field, permission, and error response

Mapping Schemas

Create the stored mappings that rules build on

Transformation Functions

Functions available inside rule expressions

Overview

Platform

Data Ingestion

Data Planes

Querying with NQL

Data Collaboration

Identity Resolution

Data Activation

Audience Studio

Connectors

Compliance

Rosetta Stone

Account Settings

Workflows

Webhooks

Data Collaboration MCP Server

Tools

SDKs

Prerequisites

Check what already exists

Write the transformation

Create the rule

Confirm the rule works

If no derived mapping appears

Change a rule

Retire a rule

Attribute Derivations

Derivations API

Mapping Schemas

Transformation Functions

​Prerequisites

​Check what already exists

​Write the transformation

​Create the rule

​Share the rule

​Confirm the rule works

​If no derived mapping appears

​Change a rule

​Retire a rule

​Related content

Attribute Derivations

Derivations API

Mapping Schemas

Transformation Functions

Prerequisites

Check what already exists

Write the transformation

Create the rule

Share the rule

Confirm the rule works

If no derived mapping appears

Change a rule

Retire a rule

Related content