The Narrative SDK provides comprehensive methods for managing datasets programmatically. This guide covers common dataset operations and patterns.
Prerequisites
- SDK installed and configured (see SDK Quickstart)
- An API key with dataset permissions
Listing datasets
Retrieve all datasets accessible to your account:
import { NarrativeApi } from '@narrative.io/data-collaboration-sdk-ts';
const api = new NarrativeApi({
apiKey: process.env.NARRATIVE_API_KEY,
});
const response = await api.getDatasets();
console.log(`Found ${response.records.length} datasets:`);
for (const dataset of response.records) {
console.log(`- ${dataset.name} (ID: ${dataset.id}, Status: ${dataset.status})`);
}
Getting dataset details
Retrieve details for a specific dataset by ID:
const dataset = await api.getDataset(12345);
console.log('Name:', dataset.name);
console.log('Description:', dataset.description);
console.log('Status:', dataset.status);
console.log('Created:', dataset.created_at);
console.log('Updated:', dataset.updated_at);
console.log('Tags:', dataset.tags);
Dataset properties
| Property | Type | Description |
|---|
id | number | Unique dataset identifier |
name | string | Dataset name |
display_name | string | Human-readable display name |
description | string | Dataset description |
status | string | active, archived, or pending |
schema | object | Dataset schema definition |
tags | string[] | Tags for organization |
created_at | string | Creation timestamp |
updated_at | string | Last update timestamp |
write_mode | string | append or overwrite |
Creating a dataset
Create a new dataset with schema definition:
const newDataset = await api.createDataset({
name: 'customer_events',
display_name: 'Customer Events',
description: 'Customer interaction events from our platform',
schema: {
properties: {
customer_id: { type: 'string' },
event_type: { type: 'string' },
event_timestamp: { type: 'timestamptz' },
event_value: { type: 'double' },
},
},
tags: ['customers', 'events'],
});
console.log('Created dataset:', newDataset.id);
Schema property types
| Type | Description |
|---|
string | Text values |
long | Integer numbers |
double | Floating-point numbers |
boolean | True/false values |
timestamptz | Timestamp with timezone |
object | Nested object |
array | Array of values |
Updating a dataset
Update dataset metadata:
const updated = await api.updateDataset({
dataset_id: 12345,
display_name: 'Customer Events (Updated)',
description: 'Updated description for customer events',
});
console.log('Updated:', updated.name);
const dataset = await api.addTags(12345, ['production', 'verified']);
console.log('Tags:', dataset.tags);
const dataset = await api.removeTags(12345, ['deprecated']);
console.log('Tags:', dataset.tags);
Getting dataset statistics
Retrieve summary statistics:
const stats = await api.getStatistics(12345);
console.log('Statistics:', stats.records);
const columnStats = await api.getColumnStats(12345);
console.log('Column statistics:', columnStats);
Dataset summary
Get a summary with sample data:
const summary = await api.getDatasetSummary(
12345, // dataset ID
false, // allRecords - set to true for all records
100, // perPage
0 // offset
);
console.log('Summary:', summary);
Sampling data
Sampling provides a preview of dataset records without downloading the full dataset. When you run an interactive query, results are stored as a dataset in your data plane. To view these results in the UI or API, a sample job retrieves up to 1,000 rows and stores them in the control plane for quick access.
Samples are one of the few cases where actual data leaves your data plane. For governance implications, see Sample Data.
- Viewing query results — See what your query returned without downloading everything
- Validating data — Check data quality before sharing with partners
- Quick inspection — Preview dataset contents during development
Request a sample of records:
// Request a sample (async operation)
const sampleRequest = await api.requestDatasetSample(12345);
console.log('Sample job ID:', sampleRequest.job_id);
// After the job completes, retrieve the sample
const sample = await api.getDatasetSample(12345, 1000);
console.log(`Sample contains ${sample.records.length} records`);
await api.deleteDatasetSample(12345);
console.log('Sample deleted');
Working with files
List files in a dataset:
const files = await api.getDatasetFiles(
12345, // dataset ID
100, // size (number of snapshots)
0, // offset
'2024-01-01', // afterTimestamp (optional)
);
for (const snapshot of files.snapshots) {
console.log(`Snapshot ${snapshot.id}:`, snapshot.files.length, 'files');
}
const download = await api.getDatasetFileDownloadUrl(
12345, // dataset ID
67890, // snapshot ID
'file-id-here' // file ID
);
console.log('Download URL:', download.download_url);
Materialized views
Refresh a materialized view:
const job = await api.refreshMaterializedView(12345);
console.log('Refresh job ID:', job.id);
await api.createDatasetRefreshSchedule(12345, {
cron_expression: '0 0 * * *', // Daily at midnight
enabled: true,
});
await api.updateDatasetRefreshSchedule(12345, {
cron_expression: '0 */6 * * *', // Every 6 hours
enabled: true,
});
await api.deleteDatasetRefreshSchedule(12345);
Retention policy
Update the retention policy:
const updated = await api.updateRetentionPolicy(12345, {
type: 'time_based',
retention_days: 365,
});
console.log('Retention policy updated');
Activating a dataset
Activate a pending dataset:
await api.activateDataset(12345);
console.log('Dataset activated');
Deleting a dataset
Deleting a dataset is permanent and cannot be undone. Ensure you have backups if needed.
await api.deleteDataset(12345);
console.log('Dataset deleted');
Error handling
try {
const dataset = await api.getDataset(12345);
console.log('Dataset:', dataset.name);
} catch (error) {
if (error.status === 404) {
console.error('Dataset not found');
} else if (error.status === 403) {
console.error('Permission denied');
} else {
console.error('Error:', error);
}
}
Related content
