> ## Documentation Index > Fetch the complete documentation index at: https://docs.narrative.io/llms.txt > Use this file to discover all available pages before exploring further. # Upsert retention policies for a dataset > Accepts either a legacy payload (snapshot-only) or the unified v1 policy object. The response is always returned in the unified v1 shape with a `compat.mode` indicating whether the server interpreted the request as legacy or v1. **Row TTL clock default:** If `row_ttl.enabled=true` and `row_ttl.clock` is omitted, the server will default to the configured Rosetta Stone attribute (nio_last_modified) **if that mapping exists**; otherwise it returns HTTP 400. ## OpenAPI ````yaml https://docs-cdn.narrative.io/api-reference/main/openapi.json put /datasets/{dataset_id}/retention-policy openapi: 3.1.1 info: contact: email: support@narrative.io name: Narrative Support url: https://www.narrative.io termsOfService: https://www.narrative.io/legal/terms-of-service x-logo: url: >- https://cdn.narrative.io/images/company-logos/prod/narrative-logo-text-white.svg backgroundColor: rgb(9, 34, 166) altText: Narrative Logo description: >- The [Narrative Data Collaboration Platform](https://app.narrative.io) API is organized around REST. Our API has predictable resource-oriented URLs, accepts form-encoded request bodies, returns JSON-encoded responses, and uses standard HTTP response codes, authentication, and verbs. The current version is a pre-release beta. It may result in unexpected behavior and there may be breaking changes in future releases up to the 1.0 release. title: Narrative Data Collaboration Platform API version: 2.4.x servers: - url: https://api-dev.narrative.io - url: https://api.narrative.io security: [] tags: - name: Access Rules description: >- Access rules let data providers control who can purchase their data and the terms of purchase. The `access-rules` API allows you to manage access rules for your datasets. Related guides: - [What is an access rule?](https://kb.narrative.io/access-rules) - name: Agent Conversations description: >- Build LLM-driven workflows that can call MCP tools, ask the caller for input, and return structured answers. A conversation pins a model, a system prompt, and a tool catalog. Each run sends the model a user message (or a batch of tool outputs from a previously-paused run); the model decides whether to answer directly, call a server-side tool (resolved by the platform via Model Context Protocol), or call a client-side tool (which pauses the run with `requires_action` and waits for the caller to reply). Related guides: - [Agent Conversations Reference](https://docs.narrative.io/reference/architecture/agent-conversations) - [Error catalog](https://docs.narrative.io/reference/architecture/agent-conversations/errors/conversation-not-found) - name: MCP Connections description: >- Connect external (non-Narrative) MCP servers so agent runs can call their tools with the calling user's own OAuth authorization. A connection is created interactively: `POST /mcp-connections` runs OAuth discovery and Dynamic Client Registration against the server and returns a consent URL; the user authorizes in a browser; the authorization server redirects to `GET /mcp-connections/callback`, which finishes the token exchange and marks the connection `connected`. Tokens are stored encrypted and used server-side — they are never returned by the API. Once connected, reference the connection by id from an agent run's `mcp_servers[].connection_id`. - name: App Invites description: >- App invites allow applications to create one-time, shareable links on behalf of their users. These links enable users to invite third parties who do not have a Narrative account to perform actions within the application. For example, a Narrative user can send an invite link to a third party who then completes a Pinterest OAuth flow and creates a connector profile in the inviter's account, allowing the inviter to deliver audience data to the third party's Pinterest account without the third party needing a Narrative account. This API is intended to be used by applications (via app client credentials) to create and manage invites on behalf of their users, which can then be shared with invitees. - name: Apps description: >- Apps are applications bundled with a UI that can perform various actions on behalf of a user utilizing the Narrative API. Related guides: - [Building a Narrative Native App](https://www.narrative.io/knowledge-base/how-to-guides/building-a-narrative-native-app) - name: Access Tokens description: API Access Token management - name: Attributes description: >- An attribute models a standardized data point available for sale on the Narrative marketplace. Narrative automatically turns data points from provider datasets into attributes so that buyers can purchase well-formed, standardized data from any supplier on the marketplace. - name: Auth description: >- API token is a crucial step for developers to securely authenticate requests to the Narrative API Related guides: - [How to Create an API Token](https://www.narrative.io/knowledge-base/how-to-guides/understanding-narratives-apis/create-an-api-token) - name: Authentication description: User login and registration - name: Billings description: Used by Narrative internally to bill customers - name: Companies description: A collection of employees - name: Company Marketing Information description: Useful information related to companies - name: Compute Pools description: >- Compute pools represent compute resources (e.g. Snowflake warehouses) provisioned within a data plane. Companies can manage and share compute pools, assign them to jobs, and set a default compute pool on data planes. - name: Connections description: Associations between connectors and datasets - name: Data Shops description: |- Self-hosted website to sell your data Related guides: - [Setting up your datashop](https://www.narrative.io/knowledge-base/how-to-guides/shop-builder/settting-up-your-data-shop) - name: Data Streams description: >- The `data-stream` API endpoints allows one to create and update data-streams. Additionally the endpoints allow finding data-streams using free text search. A few of the endpoints are behind authorization. Update endpoint allows a client to post an edited data-stream document as is, without having to change its shape. The API ensures that only certain fields are allowed to be modified. Attempts to modify fields not up for client modifications are ignored. Related guides: - [What is a data stream?](https://kb.narrative.io/what-is-a-data-stream) - name: Contracts description: Contracts related APIs - name: Datasets description: >- Any kind of data, in any schema, can be pushed into the Narrative Data Collaboration Platform as a dataset exactly as it is stored in your own system. The `datasets` API allows you to manage your datasets. - name: Destinations description: >- Destinations associate a subscription to a profile. Optionally, ad-hoc quick settings can be configured to a destination. Those quick settings have to match the format defined on the app manifest. - name: Installations description: Installations of Applications for a profile - name: Jobs description: >- Jobs represent an operation done on a given data plane. All jobs today are tied to a query that represents a forecast or a materialized view. The jobs API provides an interface for interacting with the jobs table, which stores various operations involving reading or writing data. This API allows users to retrieve detailed information about specific jobs, including cost forecasts, data forecasts, NQL forecasts and materialized views. - name: Mappings description: >- A mapping is a transformation from a dataset to an attribute. Defining a mapping between a dataset and an attribute makes the dataset eligible to participate in subscriptions where a buyer is purchasing the target attribute. - name: Model Inference description: Model Inference - name: Models description: >- Machine learning models for training and inference. Models can be stored in HuggingFace or Narrative repositories and have configurable collaborator permissions for training and inference access. The `models` API allows you to list, retrieve, and update models accessible to your company. - name: NQL description: >- Narrative Query Language (NQL) is a specialized, SQL-inspired language designed to query and manipulate data within the Narrative platform. While it looks and feels much like standard SQL, it offers extended functionality and syntax that let you leverage platform-specific features—such as referencing datasets by their IDs, creating materialized views, or generating forecasts—without having to manage the complexities of different query engines behind the scenes. NQL queries can ultimately compile down to multiple underlying engines (e.g., Snowflake, Spark) to execute your requests efficiently in the Narrative ecosystem. - name: Payment Methods description: Payment methods used to purchase data - name: Products description: Internal routes used to offer datastream as products - name: Profiles description: >- App profiles are associated with an installation. They represent a reference to a configuration that the app can use to save confidential information outside of Narrative's control. Profiles are currently used to configure settings for connector apps. - name: Resources description: >- Narrative gives you access to managed resources, like your own AWS S3 bucket, so that you can effortlessly buy and sell data on the platform. The `resources` API allows you to manage your resources. - name: Schema Inference description: >- The `schema-inference` API analyzes submitted files to automatically infer and return their structure as a dataset schema. - name: Schema Presets description: >- The `schema-presets` API allows you to list the available schema presets, get detailed information about a specific one and manage its life cycle. You can create a schema preset from scratch or create one based on an existing one, administrators can create platform wide available (public) schema preset. - name: Subscriptions description: >- In the Narrative Data Collaboration Platform a subscription represents a set of rules dictating the commercial terms related to the licensing of data. The `subscriptions` API allows you to set and get information about `subscription` objects owned by the authenticated account. - name: Uploads description: >- The `uploads` API allows you to send files to Narrative and use them to perform tasks like creating a list or adding data to a dataset. - name: Usage description: >- The `usage` API enables the recording of usage events associated with a product. - name: Workflows description: >- The `workflows` API allows you to create, schedule, trigger, and archive workflows. Workflows are defined using a serverlessworkflow YAML specification. paths: /datasets/{dataset_id}/retention-policy: put: tags: - Dataset Configuration - Retention Policies summary: Upsert retention policies for a dataset description: > Accepts either a legacy payload (snapshot-only) or the unified v1 policy object. The response is always returned in the unified v1 shape with a `compat.mode` indicating whether the server interpreted the request as legacy or v1. **Row TTL clock default:** If `row_ttl.enabled=true` and `row_ttl.clock` is omitted, the server will default to the configured Rosetta Stone attribute (nio_last_modified) **if that mapping exists**; otherwise it returns HTTP 400. operationId: putRetentionPolicy parameters: - $ref: '#/components/parameters/parameters_DatasetId-2' requestBody: required: true content: application/json: schema: oneOf: - $ref: '#/components/schemas/SnapshotRetentionPolicy' - $ref: '#/components/schemas/UnifiedV1Request' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/DatasetResponse' security: - BearerAuth: [] components: parameters: parameters_DatasetId-2: in: path name: dataset_id required: true description: Unique identifier for a dataset. schema: type: integer schemas: SnapshotRetentionPolicy: type: object additionalProperties: false description: > Backwards-compatible legacy payloads. Interpreted as snapshot-level retention on Iceberg. Returned response will be unified v1 shape with compat.mode=legacy. oneOf: - $ref: '#/components/schemas/ExpireEverything' - $ref: '#/components/schemas/RetainEverything' - $ref: '#/components/schemas/ExpireWhen' - $ref: '#/components/schemas/RetainWhen' UnifiedV1Request: type: object additionalProperties: false properties: policies: type: array items: $ref: '#/components/schemas/RetentionPolicy' schedule_config: $ref: '#/components/schemas/RefreshSchedule' required: - policies example: policies: - type: kind: table_ttl clock: kind: created_at enabled: true interval: P1D schedule_config: schedule: '@hourly' schedule_zone_id: UTC status: active DatasetResponse: type: object required: - id - company_id - created_at - name - display_name - retention_policy - schema - status - tags - write_mode - updated_at properties: id: $ref: '#/components/schemas/DatasetId' company_id: description: The ID of the company owning the dataset. type: integer compute_pool_config: description: Compute pool configuration for the dataset. type: object properties: default_compute_pool_id: description: The default compute pool to use for dataset operations. type: string format: uuid created_at: description: ISO-8601 timestamp indicating when the dataset was created type: string name: $ref: '#/components/schemas/DatasetName' display_name: $ref: '#/components/schemas/DatasetDisplayName' description: $ref: '#/components/schemas/DatasetDescription' retention_policy: $ref: '#/components/schemas/SnapshotRetentionPolicy' schema: $ref: '#/components/schemas/DatasetSchema' status: $ref: '#/components/schemas/DatasetStatus' tags: $ref: '#/components/schemas/DatasetTags' write_mode: $ref: '#/components/schemas/DatasetWriteMode' subscription_id: description: Id of a subscription which writes output to the dataset type: string updated_at: description: ISO-8601 timestamp indicating when the dataset was last updated. type: string last_snapshot_created_at: description: ISO-8601 timestamp indicating when the last snapshot was created. type: string materialized_view_config: description: >- Materialized-view configuration. Present only for materialized-view datasets. type: object required: - nql properties: nql: description: The NQL query that defines the materialized view. type: string refresh_schedule_config: description: The active refresh schedule, if one is set. type: object required: - schedule - schedule_zone_id - status properties: schedule: type: string description: >- cron expression the refresh runs on, for example: @hourly, 0 12 * * 1-5 schedule_zone_id: type: string description: 'zone id the schedule runs in, for example: EST, GMT+2' stats_enabled: type: boolean description: whether advanced statistics calculation is enabled status: type: string description: >- active runs on the cron schedule; pending and archived do not run enum: - active - pending - archived created_at: type: string description: >- ISO-8601 timestamp indicating when the refresh schedule was created. updated_at: type: string description: >- ISO-8601 timestamp indicating when the refresh schedule was last updated. stats: $ref: '#/components/schemas/DatasetResponseStats' example: id: 18923 company_id: 382 created_at: '2021-06-24T00:54:40.029056Z' description: >- Mobile identifiers tied to birth information provided as part of app registration. name: declared_age display_name: Declared Age schema: file_config: type: flat delimiter: ',' escape: '"' header: true quote: '"' type: object properties: idValue: display_name: id value order: 0 approximate_cardinality: 100000 description: ID values type: string idType: display_name: id type order: 1 approximate_cardinality: 1 type: string enum: - md5_email - sha256_email eventTimestamp: display_name: event timestamp order: 2 description: When the observation was collected type: timestamptz validations: - eventTimestamp < current_timestamp() countryCode: display_name: country code order: 3 approximate_cardinality: 6 description: The country of residence of the individual type: string age: display_name: age order: 4 approximate_cardinality: 90 description: How old this individual is in years type: long birthInfo: display_name: birth info order: 5 description: The year this individual was born type: double required: - idValue - idType - eventTimestamp - countryCode - age primary: /age status: active tags: - age - demo - demographic - demographics updated_at: '2021-06-24T00:54:40.029056Z' write_mode: append subscription_id: 123e4567-e89b-12d3-a456-426614174000 last_snapshot_created_at: '2023-06-09T16:51:34.215Z' stats: est_total_dataset_stored_files: 10 est_total_dataset_stored_records: 500059 est_total_dataset_stored_bytes: 4206454 active_dataset_stored_files: 10 active_dataset_stored_records: 500059 active_dataset_stored_bytes: 4206454 last_snapshot_added_files: 2 last_snapshot_added_records: 499999 last_snapshot_deleted_records: 0 last_snapshot_added_bytes: 4193500 last_snapshot_deleted_files: 0 last_snapshot_added_delete_files: 0 last_snapshot_removed_delete_files: 0 last_snapshot_removed_bytes: 0 ExpireEverything: type: object properties: type: type: string enum: - expire_everything example: type: expire_everything RetainEverything: type: object properties: type: type: string enum: - retain_everything example: type: retain_everything ExpireWhen: type: object properties: type: type: string enum: - expire_when expression: oneOf: - $ref: '#/components/schemas/SnapshotAge' RetainWhen: type: object properties: type: type: string enum: - retain_when expression: oneOf: - $ref: '#/components/schemas/SnapshotAge' RetentionPolicy: type: object additionalProperties: false properties: type: oneOf: - $ref: '#/components/schemas/RowTTLPolicy' - $ref: '#/components/schemas/TableTTLPolicy' - $ref: '#/components/schemas/SnapshotTTLPolicy' enabled: type: boolean default: false interval: type: string description: ISO-8601 duration, e.g., P30D, PT12H. RefreshSchedule: type: object additionalProperties: false properties: schedule: type: string description: cron expression schedule_zone_id: type: string default: UTC description: Timezone for schedule execution status: enum: - active - pending - archived description: The status DatasetId: type: integer description: Unique identifier for the dataset. DatasetName: type: string description: >- The Dataset's name. Should be unique, contain only alpanum chars and underscores. Max length is 256. DatasetDisplayName: type: string description: The Dataset's name to display at UI. DatasetDescription: type: string description: A description of the data contained with the dataset. DatasetSchema: description: > Schema describing a dataset's input file type and the kind of data the file contains. Note: when the file type is "flat" then the order of the properties describing the must be given in the order they appear in the input file. allOf: - type: object required: - file_config properties: file_config: $ref: '#/components/schemas/FileConfig' primary: $ref: '#/components/schemas/PrimaryField' - $ref: '#/components/schemas/Dataset_ObjectType' DatasetStatus: description: >- The dataset status. - `active` means that data can be added to the dataset but the schema is locked in place - `pending` means the dataset has been created, but the schema can still be altered and no data can be added to the dataset type: string enum: - active - pending DatasetTags: description: Tags that describe the dataset. type: array items: type: string DatasetWriteMode: description: >- How Narrative will treat new data uploaded for the dataset. - `append` means data will be added to the dataset, this is what to choose if you incrementally update your dataset over time. - `overwrite` means that all existing data will be overwritten with new data. type: string enum: - append - overwrite DatasetResponseStats: type: object required: - est_total_dataset_stored_files - est_total_dataset_stored_records - est_total_dataset_stored_bytes - active_dataset_stored_files - active_dataset_stored_records - active_dataset_stored_bytes - snapshot_added_files - snapshot_added_records - snapshot_deleted_records - snapshot_added_bytes - snapshot_deleted_files - snapshot_added_delete_files - snapshot_removed_delete_files - snapshot_removed_bytes properties: est_total_dataset_stored_files: $ref: '#/components/schemas/EstTotalDatasetStoredFiles' est_total_dataset_stored_records: $ref: '#/components/schemas/EstTotalDatasetStoredRecords' est_total_dataset_stored_bytes: $ref: '#/components/schemas/EstTotalDatasetStoredBytes' active_dataset_stored_files: $ref: '#/components/schemas/ActiveDatasetStoredFiles' active_dataset_stored_records: $ref: '#/components/schemas/ActiveDatasetStoredRecords' active_dataset_stored_bytes: $ref: '#/components/schemas/ActiveDatasetStoredBytes' snapshot_added_files: $ref: '#/components/schemas/SnapshotAddedFiles' snapshot_added_records: $ref: '#/components/schemas/SnapshotAddedRecords' snapshot_deleted_records: $ref: '#/components/schemas/SnapshotDeletedRecords' snapshot_added_bytes: $ref: '#/components/schemas/SnapshotAddedBytes' snapshot_deleted_files: $ref: '#/components/schemas/SnapshotDeletedFiles' snapshot_added_delete_files: $ref: '#/components/schemas/SnapshotAddedDeleteFiles' snapshot_removed_delete_files: $ref: '#/components/schemas/SnapshotRemovedDeleteFiles' snapshot_removed_bytes: $ref: '#/components/schemas/SnapshotRemovedBytes' SnapshotAge: type: object properties: type: type: string enum: - snapshot_age operator: type: string enum: - '>' - < - '>=' - <= period: type: string description: An ISO 8601 period format example: type: snapshot_age operator: '>' period: P30D RowTTLPolicy: type: object additionalProperties: false description: > Row-level deletion policy. Action is always HARD_DELETE. Clock must be EVENT_TIME. If `clock` is omitted, server uses configured Rosetta Stone attribute (defaults to nio_last_modified); if mapping doesn't exist, returns HTTP 400. properties: type: const: row_ttl clock: $ref: '#/components/schemas/RowClock' enabled: type: boolean default: false required: - type - enabled TableTTLPolicy: type: object additionalProperties: false description: > Drop the dataset object when the table age exceeds the interval. Action is always DROP_TABLE. properties: type: const: table_ttl clock: $ref: '#/components/schemas/TableClock' enabled: type: boolean default: false required: - type - enabled SnapshotTTLPolicy: type: object additionalProperties: false description: | Iceberg only: delete snapshots older than the interval. properties: type: const: snapshot_ttl snapshot_retention_policy: oneOf: - $ref: '#/components/schemas/ExpireEverything' - $ref: '#/components/schemas/RetainEverything' - $ref: '#/components/schemas/ExpireWhen' - $ref: '#/components/schemas/RetainWhen' enabled: type: boolean default: false required: - type - enabled FileConfig: description: >- Defines the input file format. Files provided to Narrative for ingestion into a Dataset must match the file configuration. oneOf: - $ref: '#/components/schemas/FlatFileConfig' - $ref: '#/components/schemas/JsonFileConfig' - $ref: '#/components/schemas/ParquetFileConfig' PrimaryField: description: >- An optional RFC-6901 JSON pointer to a Dataset field to indicate that the field is "primary" -- i.e. it is of particular interest to buyers. Narrative uses this information to optimize buyer experience when purchasing your Dataset. The primary key, if specified, must refer to a _required_ field that has a type of `boolean`, `long`, `string`, or `timestamptz`. type: string Dataset_ObjectType: type: object required: - type - properties properties: type: description: A record consisting of named fields of any type. enum: - object properties: type: object description: >- A map from field names to field types. Field names must consist of only alphanumeric characters or underscores and be < 255 characters long. required: description: >- An optional array of properties whose values are never missing or null. type: array items: type: string unsellable: description: An optional array of properties that cannot be sold to buyers. type: array items: type: string sensitive: description: >- An optional array of properties that are sensitive. Data in sensitive fields are redacted when showed to Narrative users via the UI or API but can still be bought and sold. type: array items: type: string EstTotalDatasetStoredFiles: description: >- The total estimated number of files across this dataset snapshot and all unexpired snapshots that precede it. type: integer EstTotalDatasetStoredRecords: description: >- The total estimated number of records across this dataset snapshot and all unexpired snapshots that precede it. type: integer EstTotalDatasetStoredBytes: description: >- The total estimated storage size in bytes used by this dataset snapshot and all unexpired snapshots that precede it. type: integer ActiveDatasetStoredFiles: description: The number of files in this dataset snapshot. type: integer ActiveDatasetStoredRecords: description: The number of records in this dataset snapshot. type: integer ActiveDatasetStoredBytes: description: The total storage size in bytes used by this dataset snapshot. type: integer SnapshotAddedFiles: description: The number of files added when generating this snapshot. type: integer SnapshotAddedRecords: description: The number of records added when generating this snapshot. type: integer SnapshotDeletedRecords: description: The number of records deleted when generating this snapshot. type: integer SnapshotAddedBytes: description: The amount of storage in bytes added when generating this snapshot. type: integer SnapshotDeletedFiles: description: The number of files deleted when generating this snapshot. type: integer SnapshotAddedDeleteFiles: description: The number of delete tracking files added when generating this snapshot. type: integer SnapshotRemovedDeleteFiles: description: >- The number of delete tracking files removed when generating this snapshot. type: integer SnapshotRemovedBytes: description: The amount of storage in bytes removed when generating this snapshot. type: integer RowClock: type: object additionalProperties: false properties: kind: type: string enum: - event_time attribute: type: string description: Rosetta Stone attribute name for timestamp evaluation. required: - kind TableClock: type: object additionalProperties: false properties: kind: type: string enum: - created_at - max_event_time - static_time column: type: string description: Required when kind={max_event_time, static_time}. required: - kind FlatFileConfig: type: object required: - type - header - delimiter - quote - escape properties: type: type: string description: >- Describes input files where each row is a set of fields separated by some delimiter. Common examples include comma-separated (CSV) and tab-separated (TSV) files. Files can be compressed using gzip or bzip2 as long as they include the appropriate suffix to indicate what kind of compression has been applied. E.g. bzip2 files must end in ".bz2" and gzipped files must end in ".gz". enum: - flat header: type: boolean description: Whether or not files include a header. delimiter: type: string description: Character separating values within a row in a file. quote: type: string description: >- Character used for escaping quoted values where the separator can be part of the value. escape: type: string description: Character used for escaping quotes inside an already quoted value. JsonFileConfig: type: object required: - type properties: type: type: string description: >- Describes input files in the [JSON Lines](https://jsonlines.org/) file format. enum: - json ParquetFileConfig: type: object required: - type properties: type: type: string description: >- Describes input files in the [Parquet](https://parquet.apache.org/) file format. enum: - parquet securitySchemes: BearerAuth: type: http scheme: bearer ````