> ## Documentation Index
> Fetch the complete documentation index at: https://docs.narrative.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Amazon S3 Connector API

> Deliver Narrative datasets as files to your own Amazon S3 bucket over the API, and set up what the S3 API guide needs

The Amazon S3 Connector API lets an unattended integration deliver the rows of a Narrative dataset as files into an S3 bucket you own. Narrative delivers each new batch as it lands, and can also deliver the batches from up to 30 days before you connect the dataset. A person signs in once to create an API key and once to set up the bucket, and the integration works from the API key after that.

Complete the [prerequisites](#prerequisites) and [set up API access](#set-up-api-access) on this page before you follow the guide.

## Choose a guide

<CardGroup cols={2}>
  <Card title="File delivery" icon="file-export" href="/guides/connector-apis/amazon-s3/file-delivery-api">
    Create a dataset, upload rows, connect it to your bucket, and keep delivering as new rows arrive.
  </Card>
</CardGroup>

## Prerequisites

| Prerequisite | Needed for |
| - | - |
| [API key](#api-key) | Everything |
| [S3 Connector installed](#s3-connector-installed) | Everything |
| [A profile for your bucket](#a-profile-for-your-bucket) | File delivery |

### API key

Create the key under **Settings → API Keys** with read and write on `datasets` and `connections`, write on `uploads`, and read on `installations`, `apps`, and `app_profiles`. See [API Keys](/account-settings/api-keys) for the procedure and the [Permissions Reference](/reference/security/permissions) for what each resource covers.

Create it from a shared team account rather than a personal login, so the integration keeps working when someone changes roles.

### S3 Connector installed

The S3 Connector is app `7`. Find your installation of it:

```bash theme={null}
curl -s "https://api.narrative.io/installations?app_categories=destination_connector" \
  -H "Authorization: Bearer $NIO_API_TOKEN"
# find the record with "app_id": 7 and note its "id"
```

Record the installation ID. If no record has `"app_id": 7`, install the S3 Connector from the Narrative Marketplace.

### A profile for your bucket

A connector profile names one S3 bucket and the file format Narrative writes into it. You set up a profile once per bucket, in the Narrative UI, with access to the bucket's settings in AWS:

<Steps>
  <Step title="Open the connector">
    In Narrative, go to **Installed Apps**, select **S3 Connector**, and choose **Add profile**.
  </Step>

  <Step title="Name the profile and bucket">
    Enter a profile name, a description, and the name of the bucket Narrative delivers to.
  </Step>

  <Step title="Apply the bucket policy in AWS">
    Copy the bucket policy the wizard shows, and apply it in the AWS console under the bucket's **Permissions → Bucket policy**, or with `aws s3api put-bucket-policy`. The policy lets Narrative's delivery role write, list, and delete objects in the bucket. You can also fetch the policy over the API, as shown in [Get the bucket policy](#get-the-bucket-policy).
  </Step>

  <Step title="Tag the bucket in AWS">
    Add the tag the wizard shows to the bucket, under **Properties → Tags**. The tag is named `NARRATIVE_S3_CONNECTOR_ID`, and its value ties the bucket to your company.
  </Step>

  <Step title="Test access">
    Run the wizard's access test. Narrative writes, lists, and deletes a test object in the bucket and reads the tag.
  </Step>

  <Step title="Choose the file format and save">
    Choose CSV, TSV, JSON, or Parquet. For CSV or TSV, also set the delimiter, quote, and escape characters, and whether to include a header row. Choose **Save and finish**, and the profile is enabled and ready for connections.
  </Step>
</Steps>

Every request in the guide then uses the profile ID. [List your profiles](#list-your-profiles) shows how to look it up.

## Set up API access

The guide talks to two hosts, and each takes its own credential:

| Host | Base URL | Credential | Handles |
| - | - | - | - |
| Narrative API | `https://api.narrative.io` | Your API key | Datasets, uploads, connections, profiles |
| S3 connector | `https://aws-s3.narrativeconnectors.com` | An installation token | Bucket and file format of each profile, bucket policy |

### Get an installation token

The S3 connector authenticates requests with an installation token, which you get by exchanging your API key. Use the installation ID from [S3 Connector installed](#s3-connector-installed):

```bash theme={null}
curl -s -X POST https://api.narrative.io/installations/{installation_id}/token \
  -H "Authorization: Bearer $NIO_API_TOKEN"
# → {"access_token": "<INSTALLATION_TOKEN>", "access_token_expires_in": 10799,
#    "refresh_token": "<REFRESH_TOKEN>", "refresh_token_expires_in": 86399}
```

Store `access_token` as `$INSTALLATION_TOKEN`. It lasts about three hours, so request a new one at the start of each run of your integration.

Confirm that both hosts answer:

```bash theme={null}
curl -s https://api.narrative.io/company-info/whoami \
  -H "Authorization: Bearer $NIO_API_TOKEN"
# → {"id": 1234}

curl -s https://aws-s3.narrativeconnectors.com/profiles \
  -H "Authorization: Bearer $INSTALLATION_TOKEN"
# → [{"id": "<PROFILE_ID>", "status": "enabled", "bucket_name": "<BUCKET_NAME>", ...}]
```

Send the API key to `api.narrative.io` and the installation token to `aws-s3.narrativeconnectors.com`.

<Tip>
  API keys contain characters that shells mangle. Read the key from a file or an environment variable rather than pasting it inline.
</Tip>

### List your profiles

The Narrative API lists the profiles under your installation, with the status of each:

```bash theme={null}
curl -s https://api.narrative.io/installations/{installation_id}/profiles \
  -H "Authorization: Bearer $NIO_API_TOKEN"
```

```json theme={null}
{
  "records": [
    {
      "id": "<PROFILE_ID>",
      "name": "Example delivery bucket",
      "description": "Example delivery bucket",
      "status": "enabled",
      "app_id": null,
      "app_name": null,
      "tags": []
    }
  ]
}
```

The S3 connector returns the same profiles along with the bucket and file format of each:

```bash theme={null}
curl -s https://aws-s3.narrativeconnectors.com/profiles \
  -H "Authorization: Bearer $INSTALLATION_TOKEN"
```

```json theme={null}
[
  {
    "id": "<PROFILE_ID>",
    "name": "Example delivery bucket",
    "description": "Example delivery bucket",
    "status": "enabled",
    "bucket_name": "<BUCKET_NAME>",
    "config": {
      "file_type": "csv",
      "header": true,
      "delimiter": ",",
      "quote": "\"",
      "escape": "\"",
      "sort_columns": true,
      "compression": null
    }
  }
]
```

To read one profile, call `GET /profiles/{profile_id}` on the connector, which returns the same shape. Use a profile whose `status` is `enabled`, and record its `id`.

### Get the bucket policy

The connector generates the bucket policy for a bucket name:

```bash theme={null}
curl -s https://aws-s3.narrativeconnectors.com/bucket/{bucket_name}/bucket-policy \
  -H "Authorization: Bearer $INSTALLATION_TOKEN"
```

```json theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "AWS": "<NARRATIVE_DELIVERY_ROLE_ARN>" },
      "Action": ["s3:DeleteObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
    },
    {
      "Effect": "Allow",
      "Principal": { "AWS": "<NARRATIVE_DELIVERY_ROLE_ARN>" },
      "Action": ["s3:List*", "s3:GetBucketTagging"],
      "Resource": "arn:aws:s3:::<BUCKET_NAME>"
    }
  ]
}
```

Apply the policy exactly as the connector returns it, with Narrative's delivery role in `Principal`.

### Read the delivery settings schema

Each connection carries delivery settings for the connector's `delivery` interface. The schema lists every setting with its type and description:

```bash theme={null}
curl -s https://api.narrative.io/apps/7/interfaces/delivery \
  -H "Authorization: Bearer $NIO_API_TOKEN"
# → {"id": "delivery", "app_id": 7, "metadata": {"tags": ["s3"], ...},
#    "schema": {"properties": {"bucket_prefix": {...}, "historical_data_enabled": {...}, ...}, ...}}
```

The connector serves the same schema at `https://aws-s3.narrativeconnectors.com/interfaces/delivery`, with no credential. [What the connection does](#what-the-connection-does) explains each field.

## What the connection does

A connection's `quick_settings` set where and how the connection writes files. The S3 Connector has one interface, `delivery`. The file format, compression, and CSV options come from the profile.

| Field | What it does | Allowed values | Default |
| - | - | - | - |
| `bucket_prefix` | The folder under the bucket that files go into | A path such as `exports/narrative` | The bucket root |
| `historical_data_enabled` | Whether the connection also delivers data already in the dataset | `true`, `false` | `false` |
| `historical_period_seconds` | How far back `historical_data_enabled` reaches, in seconds | Any positive integer. Values over 2,592,000 (30 days) count as 30 days. | 2,592,000 |
| `remove_metadata` | Drops Narrative's `_nio` metadata columns from delivered files | `true`, `false` | `false` |
| `use_delivery_date` | Names each folder after the delivery date (`yyyy-MM-dd`, UTC) instead of the snapshot ID | `true`, `false` | `false` |
| `include_success_file` | Writes an empty `_SUCCESS` file once every file in a delivery has landed | `true`, `false` | `false` |
| `write_narrative_commit_file` | Writes an empty `_NIO_COMMIT` file once every file in a delivery has landed | `true`, `false` | `false` |

### Existing and new rows

A connection delivers every snapshot written to the dataset after the connection is created. The `historical_data_enabled` quick setting decides whether the connection also delivers data that was already in the dataset when you created it:

* With `historical_data_enabled` set to `true`, the connection first delivers data already in the dataset, then keeps delivering new snapshots as they land.
* With `historical_data_enabled` set to `false`, the connection delivers only snapshots written after it was created.

How far back the first delivery reaches depends on the connector, and each connector's guide states it.

For the S3 Connector, `historical_data_enabled` delivers the dataset's snapshots from the last `historical_period_seconds` before the connection was created. After that, each new snapshot is delivered as it lands.

Each source file in a snapshot becomes one object in the bucket, under `<bucket_prefix>/snapshotId=<snapshot ID>/`. A delivery only adds objects. When you overwrite the dataset, the new snapshot lands in a new folder and the earlier folders stay in the bucket.

***

## Related content

<CardGroup cols={2}>
  <Card title="Amazon S3 Connector" icon="aws" href="/reference/connectors/amazon-s3">
    Supported file formats and delivery options
  </Card>

  <Card title="Connector Interfaces" icon="plug" href="/concepts/data-activation/connector-interfaces">
    Why a dataset connects to an interface rather than a connector
  </Card>

  <Card title="API Keys" icon="key" href="/account-settings/api-keys">
    Create and rotate keys for programmatic access
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.