Managing Datasets

The Narrative SDK provides comprehensive methods for managing datasets programmatically. This guide covers common dataset operations and patterns.

Prerequisites

SDK installed and configured (see SDK Quickstart)
An API key with dataset permissions

Listing datasets

Retrieve all datasets accessible to your account:

import { NarrativeApi } from '@narrative.io/data-collaboration-sdk-ts';

const api = new NarrativeApi({
  apiKey: process.env.NARRATIVE_API_KEY,
});

const response = await api.getDatasets();

console.log(`Found ${response.records.length} datasets:`);
for (const dataset of response.records) {
  console.log(`- ${dataset.name} (ID: ${dataset.id}, Status: ${dataset.status})`);
}

Getting dataset details

Retrieve details for a specific dataset by ID:

const dataset = await api.getDataset(12345);

console.log('Name:', dataset.name);
console.log('Description:', dataset.description);
console.log('Status:', dataset.status);
console.log('Created:', dataset.created_at);
console.log('Updated:', dataset.updated_at);
console.log('Tags:', dataset.tags);

Dataset properties

Property	Type	Description
`id`	`number`	Unique dataset identifier
`name`	`string`	Dataset name
`display_name`	`string`	Human-readable display name
`description`	`string`	Dataset description
`status`	`string`	`active`, `archived`, or `pending`
`schema`	`object`	Dataset schema definition
`tags`	`string[]`	Tags for organization
`created_at`	`string`	Creation timestamp
`updated_at`	`string`	Last update timestamp
`write_mode`	`string`	`append` or `overwrite`

Creating a dataset

Create a new dataset with schema definition:

const newDataset = await api.createDataset({
  name: 'customer_events',
  display_name: 'Customer Events',
  description: 'Customer interaction events from our platform',
  schema: {
    properties: {
      customer_id: { type: 'string' },
      event_type: { type: 'string' },
      event_timestamp: { type: 'timestamptz' },
      event_value: { type: 'double' },
    },
  },
  tags: ['customers', 'events'],
});

console.log('Created dataset:', newDataset.id);

Schema property types

Type	Description
`string`	Text values
`long`	Integer numbers
`double`	Floating-point numbers
`boolean`	True/false values
`timestamptz`	Timestamp with timezone
`object`	Nested object
`array`	Array of values

Updating a dataset

Update dataset metadata:

const updated = await api.updateDataset({
  dataset_id: 12345,
  display_name: 'Customer Events (Updated)',
  description: 'Updated description for customer events',
});

console.log('Updated:', updated.name);

Managing tags

Add tags to a dataset:

const dataset = await api.addTags(12345, ['production', 'verified']);
console.log('Tags:', dataset.tags);

Remove tags from a dataset:

const dataset = await api.removeTags(12345, ['deprecated']);
console.log('Tags:', dataset.tags);

Getting dataset statistics

Retrieve summary statistics:

const stats = await api.getStatistics(12345);

console.log('Statistics:', stats.records);

Get column-level statistics:

const columnStats = await api.getColumnStats(12345);

console.log('Column statistics:', columnStats);

Dataset summary

Get a summary with sample data:

const summary = await api.getDatasetSummary(
  12345,    // dataset ID
  false,    // allRecords - set to true for all records
  100,      // perPage
  0         // offset
);

console.log('Summary:', summary);

Sampling data

Sampling provides a preview of dataset records without downloading the full dataset. When you run an interactive query, results are stored as a dataset in your data plane. To view these results in the UI or API, a sample job retrieves up to 1,000 rows and stores them in the control plane for quick access.

Samples are one of the few cases where actual data leaves your data plane. For governance implications, see Sample Data.

Samples are useful for:

Viewing query results — See what your query returned without downloading everything
Validating data — Check data quality before sharing with partners
Quick inspection — Preview dataset contents during development

Request a sample of records:

// Request a sample (async operation)
const sampleRequest = await api.requestDatasetSample(12345);
console.log('Sample job ID:', sampleRequest.job_id);

// After the job completes, retrieve the sample
const sample = await api.getDatasetSample(12345, 1000);
console.log(`Sample contains ${sample.records.length} records`);

Delete a sample:

await api.deleteDatasetSample(12345);
console.log('Sample deleted');

Working with files

List files in a dataset:

const files = await api.getDatasetFiles(
  12345,           // dataset ID
  100,             // size (number of snapshots)
  0,               // offset
  '2024-01-01',    // afterTimestamp (optional)
);

for (const snapshot of files.snapshots) {
  console.log(`Snapshot ${snapshot.id}:`, snapshot.files.length, 'files');
}

Get a download URL for a specific file:

const download = await api.getDatasetFileDownloadUrl(
  12345,           // dataset ID
  67890,           // snapshot ID
  'file-id-here'   // file ID
);

console.log('Download URL:', download.download_url);

Materialized views

Refresh a materialized view:

const job = await api.refreshMaterializedView(12345);
console.log('Refresh job ID:', job.id);

Create a refresh schedule:

await api.createDatasetRefreshSchedule(12345, {
  cron_expression: '0 0 * * *',  // Daily at midnight
  enabled: true,
});

Update a refresh schedule:

await api.updateDatasetRefreshSchedule(12345, {
  cron_expression: '0 */6 * * *',  // Every 6 hours
  enabled: true,
});

Delete a refresh schedule:

await api.deleteDatasetRefreshSchedule(12345);

Retention policy

Update the retention policy:

const updated = await api.updateRetentionPolicy(12345, {
  type: 'time_based',
  retention_days: 365,
});

console.log('Retention policy updated');

Activating a dataset

Activate a pending dataset:

await api.activateDataset(12345);
console.log('Dataset activated');

Deleting a dataset

Deleting a dataset is permanent and cannot be undone. Ensure you have backups if needed.

await api.deleteDataset(12345);
console.log('Dataset deleted');

Error handling

try {
  const dataset = await api.getDataset(12345);
  console.log('Dataset:', dataset.name);
} catch (error) {
  if (error.status === 404) {
    console.error('Dataset not found');
  } else if (error.status === 403) {
    console.error('Permission denied');
  } else {
    console.error('Error:', error);
  }
}

Uploading Data

Upload files to datasets

Executing Queries

Query dataset data

API Modules Reference

Complete API documentation

Overview

Platform

Data Ingestion

Data Planes

Querying with NQL

Data Collaboration

Identity Resolution

Data Activation

Audience Studio

Connectors

Compliance

Rosetta Stone

Account Settings

Workflows

Webhooks

Data Collaboration MCP Server

Tools

SDKs

Prerequisites

Listing datasets

Getting dataset details

Dataset properties

Creating a dataset

Schema property types

Updating a dataset

Managing tags

Getting dataset statistics

Dataset summary

Sampling data

Working with files

Materialized views

Retention policy

Activating a dataset

Deleting a dataset

Error handling

Uploading Data

Executing Queries

API Modules Reference

​Prerequisites

​Listing datasets

​Getting dataset details

​Dataset properties

​Creating a dataset

​Schema property types

​Updating a dataset

​Managing tags

​Getting dataset statistics

​Dataset summary

​Sampling data

​Working with files

​Materialized views

​Retention policy

​Activating a dataset

​Deleting a dataset

​Error handling

​Related content

Uploading Data

Executing Queries

API Modules Reference

Prerequisites

Listing datasets

Getting dataset details

Dataset properties

Creating a dataset

Schema property types

Updating a dataset

Managing tags

Getting dataset statistics

Dataset summary

Sampling data

Working with files

Materialized views

Retention policy

Activating a dataset

Deleting a dataset

Error handling

Related content