> ## Documentation Index
> Fetch the complete documentation index at: https://docs.narrative.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Run a NQL Query

> The run endpoint in the NQL API allows users to execute their NQL queries.
This endpoint processes the query and performs the specified operations
within the specified data plane. Depending on the NQL provided, the run
endpoint can create materialized views or generate forecasts.



## OpenAPI

````yaml https://docs-cdn.narrative.io/api-reference/main/openapi.json post /nql/run
openapi: 3.1.1
info:
  contact:
    email: support@narrative.io
    name: Narrative Support
    url: https://www.narrative.io
  termsOfService: https://www.narrative.io/legal/terms-of-service
  x-logo:
    url: >-
      https://cdn.narrative.io/images/company-logos/prod/narrative-logo-text-white.svg
    backgroundColor: rgb(9, 34, 166)
    altText: Narrative Logo
  description: >-
    The [Narrative Data Collaboration Platform](https://app.narrative.io) API is
    organized around REST. Our API has predictable resource-oriented URLs,
    accepts form-encoded request bodies, returns JSON-encoded responses, and
    uses standard HTTP response codes, authentication, and verbs.



    The current version is a pre-release beta.  It may result in unexpected
    behavior and there may be breaking changes in future releases up to the 1.0
    release.
  title: Narrative Data Collaboration Platform API
  version: 2.4.x
servers:
  - url: https://api-dev.narrative.io
  - url: https://api.narrative.io
security: []
tags:
  - name: Access Rules
    description: >-
      Access rules let data providers control who can purchase their data and
      the terms of purchase.


      The `access-rules` API allows you to manage access rules for your
      datasets.


      Related guides:
        - [What is an access rule?](https://kb.narrative.io/access-rules)
  - name: Agent Conversations
    description: >-
      Build LLM-driven workflows that can call MCP tools, ask the caller for
      input, and return structured answers.


      A conversation pins a model, a system prompt, and a tool catalog. Each run
      sends the model a user message (or

      a batch of tool outputs from a previously-paused run); the model decides
      whether to answer directly, call a

      server-side tool (resolved by the platform via Model Context Protocol), or
      call a client-side tool (which pauses

      the run with `requires_action` and waits for the caller to reply).


      Related guides:
        - [Agent Conversations Reference](https://docs.narrative.io/reference/architecture/agent-conversations)
        - [Error catalog](https://docs.narrative.io/reference/architecture/agent-conversations/errors/conversation-not-found)
  - name: MCP Connections
    description: >-
      Connect external (non-Narrative) MCP servers so agent runs can call their
      tools with the

      calling user's own OAuth authorization.


      A connection is created interactively: `POST /mcp-connections` runs OAuth
      discovery and

      Dynamic Client Registration against the server and returns a consent URL;
      the user authorizes

      in a browser; the authorization server redirects to `GET
      /mcp-connections/callback`, which

      finishes the token exchange and marks the connection `connected`. Tokens
      are stored encrypted

      and used server-side — they are never returned by the API. Once connected,
      reference the

      connection by id from an agent run's `mcp_servers[].connection_id`.
  - name: App Invites
    description: >-
      App invites allow applications to create one-time, shareable links on
      behalf of their users. These links

      enable users to invite third parties who do not have a Narrative account
      to perform actions within the

      application.


      For example, a Narrative user can send an invite link to a third party who
      then completes a Pinterest

      OAuth flow and creates a connector profile in the inviter's account,
      allowing the inviter to deliver

      audience data to the third party's Pinterest account without the third
      party needing a Narrative account.


      This API is intended to be used by applications (via app client
      credentials) to create and manage invites

      on behalf of their users, which can then be shared with invitees.
  - name: Apps
    description: >-
      Apps are applications bundled with a UI that can perform various actions
      on behalf of a user utilizing the Narrative API.

      Related guides:
        - [Building a Narrative Native App](https://www.narrative.io/knowledge-base/how-to-guides/building-a-narrative-native-app)
  - name: Access Tokens
    description: API Access Token management
  - name: Attributes
    description: >-
      An attribute models a standardized data point available for sale on the
      Narrative marketplace.


      Narrative automatically turns data points from provider datasets into
      attributes so that buyers can purchase well-formed, standardized data from
      any supplier on the marketplace.
  - name: Auth
    description: >-
      API token is a crucial step for developers to securely authenticate
      requests to the Narrative API

      Related guides:
        - [How to Create an API Token](https://www.narrative.io/knowledge-base/how-to-guides/understanding-narratives-apis/create-an-api-token)
  - name: Authentication
    description: User login and registration
  - name: Billings
    description: Used by Narrative internally to bill customers
  - name: Companies
    description: A collection of employees
  - name: Company Marketing Information
    description: Useful information related to companies
  - name: Compute Pools
    description: >-
      Compute pools represent compute resources (e.g. Snowflake warehouses)
      provisioned within a data plane.

      Companies can manage and share compute pools, assign them to jobs, and set
      a default compute pool on data planes.
  - name: Connections
    description: Associations between connectors and datasets
  - name: Data Shops
    description: |-
      Self-hosted website to sell your data
      Related guides:
        - [Setting up your datashop](https://www.narrative.io/knowledge-base/how-to-guides/shop-builder/settting-up-your-data-shop)
  - name: Data Streams
    description: >-
      The `data-stream` API endpoints allows one to create and update
      data-streams. Additionally the endpoints allow

      finding data-streams using free text search. A few of the endpoints are
      behind authorization.


      Update endpoint allows a client to post an edited data-stream document as
      is, without having to change its shape.

      The API ensures that only certain fields are allowed to be modified.
      Attempts to modify fields not up for client

      modifications are ignored.


      Related guides:
        - [What is a data stream?](https://kb.narrative.io/what-is-a-data-stream)
  - name: Contracts
    description: Contracts related APIs
  - name: Datasets
    description: >-
      Any kind of data, in any schema, can be pushed into the Narrative Data
      Collaboration Platform as a dataset exactly as it is stored in your own
      system.


      The `datasets` API allows you to manage your datasets.
  - name: Destinations
    description: >-
      Destinations associate a subscription to a profile. Optionally, ad-hoc
      quick settings can be configured to a destination.

      Those quick settings have to match the format defined on the app manifest.
  - name: Installations
    description: Installations of Applications for a profile
  - name: Jobs
    description: >-
      Jobs represent an operation done on a given data plane. All jobs today are
      tied to a query that represents a forecast or a materialized view.


      The jobs API provides an interface for interacting with the jobs table,
      which stores various operations involving reading or writing data. This
      API allows users to retrieve detailed information about specific jobs,
      including cost forecasts, data forecasts, NQL forecasts and materialized
      views.
  - name: Mappings
    description: >-
      A mapping is a transformation from a dataset to an attribute. Defining a
      mapping between a dataset and an attribute makes the dataset eligible to
      participate in subscriptions where a buyer is purchasing the target
      attribute.
  - name: Model Inference
    description: Model Inference
  - name: Models
    description: >-
      Machine learning models for training and inference.


      Models can be stored in HuggingFace or Narrative repositories and have
      configurable

      collaborator permissions for training and inference access.


      The `models` API allows you to list, retrieve, and update models
      accessible to your company.
  - name: NQL
    description: >-
      Narrative Query Language (NQL) is a specialized, SQL-inspired language
      designed to query and manipulate data within the Narrative platform. While
      it looks and feels much like standard SQL, it offers extended
      functionality and syntax that let you leverage platform-specific
      features—such as referencing datasets by their IDs, creating materialized
      views, or generating forecasts—without having to manage the complexities
      of different query engines behind the scenes. NQL queries can ultimately
      compile down to multiple underlying engines (e.g., Snowflake, Spark) to
      execute your requests efficiently in the Narrative ecosystem.
  - name: Payment Methods
    description: Payment methods used to purchase data
  - name: Products
    description: Internal routes used to offer datastream as products
  - name: Profiles
    description: >-
      App profiles are associated with an installation. They represent a
      reference to a configuration that the app can use to save confidential
      information outside of Narrative's control.

      Profiles are currently used to configure settings for connector apps.
  - name: Resources
    description: >-
      Narrative gives you access to managed resources, like your own AWS S3
      bucket, so that you can effortlessly buy and sell data on the platform.


      The `resources` API allows you to manage your resources.
  - name: Schema Inference
    description: >-
      The `schema-inference` API analyzes submitted files to automatically infer
      and return their structure as a dataset schema.
  - name: Schema Presets
    description: >-
      The `schema-presets` API allows you to list the available schema presets,
      get detailed information about a specific one and manage its life cycle.


      You can create a schema preset from scratch or create one based on an
      existing one, administrators can create platform wide available (public)
      schema preset.
  - name: Subscriptions
    description: >-
      In the Narrative Data Collaboration Platform a subscription represents a
      set of rules dictating the commercial terms related to the licensing of
      data.


      The `subscriptions` API allows you to set and get information about
      `subscription` objects owned by the authenticated account.
  - name: Uploads
    description: >-
      The `uploads` API allows you to send files to Narrative and use them to
      perform tasks like creating a list or adding data to a dataset.
  - name: Usage
    description: >-
      The `usage` API enables the recording of usage events associated with a
      product.
  - name: Workflows
    description: >-
      The `workflows` API allows you to create, schedule, trigger, and archive
      workflows.

      Workflows are defined using a serverlessworkflow YAML specification.
paths:
  /nql/run:
    post:
      tags:
        - NQL
      summary: Run a NQL Query
      description: >-
        The run endpoint in the NQL API allows users to execute their NQL
        queries.

        This endpoint processes the query and performs the specified operations

        within the specified data plane. Depending on the NQL provided, the run

        endpoint can create materialized views or generate forecasts.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - nql
              properties:
                nql:
                  $ref: '#/components/schemas/NQL'
                data_plane_id:
                  $ref: '#/components/schemas/DataPlaneId'
                compute_pool_id:
                  type: string
                  format: uuid
                  description: >-
                    The ID of the compute pool to use for running this query.
                    The compute pool

                    must be active, belong to your company, and be associated
                    with the target

                    data plane. If not specified, the default compute pool for
                    the data plane

                    will be used (if one is configured).
            example:
              nql: >-
                CREATE MATERIALIZED VIEW "dataset_name" AS SELECT
                "company_data"."1".x AS xx, "company_data"."1".y AS yy,
                "company_data"."1".z AS zz, "company_data"."1".x +
                "company_data"."1".y + "company_data"."1".z AS xyz_sum FROM
                "company_data"."1"
      responses:
        '201':
          description: Created
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/NQL_RunResponse'
              examples:
                create_materialized_view:
                  summary: create materialized view
                  description: |-
                    NQL Query:

                    ```sql
                    CREATE MATERIALIZED VIEW "test_gui"
                      EXPIRE = 'retain_everything'
                      WRITE_MODE = 'append'
                      EXTENDED_STATS = 'all' AS
                    SELECT
                      1 as a,
                      2 as b
                    ```
                  value:
                    id: 7352a290-dea9-4932-b7d3-66723e145259
                    company_id: 1
                    completed_at: null
                    created_at: '2025-04-14T17:33:24.941123Z'
                    failures: []
                    idempotency_key: 3c7afdbb-1b2a-458b-a8bf-ac7b7942cb6c
                    input:
                      compiled_select: |-
                        SELECT
                          *
                        FROM
                          (VALUES
                              (1, 2)) `t` (`a`, `b`)
                      create_as_view: false
                      dataset:
                        id: 23499
                        company_id: 1
                        created_at: '2025-04-14T17:33:23.500965Z'
                        data_plane:
                          type: owned
                          id: f79cbdae-4848-47ca-95e8-69588364d185
                          company_id: 1
                          created_at: '2016-01-01T00:00:00Z'
                          description: ...
                          display_name: Narrative Data Collaboration Platform (us-east-1)
                          external_id: '704349335716'
                          platform:
                            type: platform_aws
                            account_id: '704349335716'
                            region:
                              type: region_aws
                              id: us-east-1
                          status: active
                          updated_at: '2024-01-12T00:00:00Z'
                          collaborators:
                            participants:
                              type: all
                        display_name: test_gui
                        is_narrative_managed: true
                        materialized_view_config:
                          nql: SELECT 1 AS a, 2 AS b
                        name: test_gui
                        retention_policy:
                          type: retain_everything
                        schema:
                          file_config:
                            type: json
                          type: object
                          properties:
                            a:
                              display_name: a
                              type: long
                            b:
                              display_name: b
                              type: long
                        status: pending
                        tags:
                          - _nio_materialized_view
                        updated_at: '2025-04-14T17:33:23.500965Z'
                        write_mode: append
                      nql: SELECT 1 AS a, 2 AS b
                    request_source:
                      type: api_user
                      company_id: 1
                      user_id: 1414
                    result: null
                    state: pending
                    updated_at: '2025-04-14T17:33:24.941123Z'
                explain:
                  summary: explain
                  description: >-
                    To run a forecast query, add the `EXPLAIN` keyword at the
                    beginning of the query

                    ```sql

                    EXPLAIN

                    SELECT hashed_email

                    FROM company_data."23021"

                    ```
                  value:
                    id: ab97b835-216f-4048-aabf-e2ac22e563fc
                    company_id: 1
                    completed_at: null
                    created_at: '2025-04-14T17:44:46.843651Z'
                    failures: []
                    idempotency_key: 4d864006-9b4d-4376-a4a9-c43393b24fbe
                    input:
                      nql: EXPLAIN select hashed_email from company_data."23021"
                      compiled_sql: |-
                        SELECT
                          SUM(NAMED_STRUCT('price_micro_cents_usd', 0, 'count_multiplier', 1)['price_micro_cents_usd'] * NAMED_STRUCT('price_micro_cents_usd', 0, 'count_multiplier', 1)['count_multiplier']) `totalCost`,
                          SUM(NAMED_STRUCT('price_micro_cents_usd', 0, 'count_multiplier', 1)['count_multiplier']) `totalRows`
                        FROM
                          narrative.datasets.ds_23021
                    request_source:
                      type: api_user
                      company_id: 1
                      user_id: 1414
                    result: null
                    state: pending
                    updated_at: '2025-04-14T17:44:46.843651Z'
        '400':
          description: >-
            Bad Request

            visit
            https://www.narrative.io/knowledge-base/support-troubleshooting/nql
            for more details
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Response'
        '422':
          description: >-
            Unprocessable Entity — the query parsed but is semantically invalid,
            for example a materialized

            view output column whose type is not supported (such as DECIMAL), or
            a query that produces no

            output columns.

            visit
            https://www.narrative.io/knowledge-base/support-troubleshooting/nql
            for more details
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Response'
              examples:
                unsupported_output_type:
                  summary: unsupported output column type
                  value:
                    title: 'Unsupported Type Error: DECIMAL'
                    status: 422
                    detail: >-
                      Column 'decimal_value' has type DECIMAL, which is not
                      supported as an output column of a materialized view. CAST
                      the column to a supported type such as DOUBLE, BIGINT,
                      VARCHAR, BOOLEAN, or TIMESTAMP.
                    instance: /nql/run
                    log_id: c904b703-d41e-4372-9276-9f4bdc8ad031
                    debug:
                      column: decimal_value
                      type_name: DECIMAL
      security:
        - BearerAuth: []
components:
  schemas:
    NQL:
      type: string
      description: A NQL query.
    DataPlaneId:
      type: string
      format: uuid
      description: >-
        A dataplane represent where you would run the query. Your query get
        compiled

        to the SQL dialect that your data engine understands. If you leave it
        blank,

        your query will target Narrative's dataplane on AWS using Apache Spark.
        We

        currently support Snowflake and we plan to support other dataplanes in
        the

        future. See https://next.narrative.io/products/narrative-anywhere for
        more

        details.
    NQL_RunResponse:
      oneOf:
        - $ref: '#/components/schemas/ForecastRunResponse'
        - $ref: '#/components/schemas/CreateMaterialiedViewRunResponse'
    Response:
      type: object
      required:
        - title
        - status
        - detail
        - instance
        - log_id
      properties:
        type:
          type: string
        title:
          type: string
          description: a summary of the error
        status:
          type: number
          description: http status code
        detail:
          type: string
          description: a longer description of the error
        instance:
          type: string
          description: path of the query
        log_id:
          type: string
          format: uuid
          description: an internal logging id, Narrative can use to help debug a problem.
      example:
        type: null
        title: Failed to Parse Query
        status: 400
        detail: Invalid query format.
        instance: /nql/validate
        log_id: c06fea02-0950-45c6-aaff-07ab07c0d04c
        debug: null
    ForecastRunResponse:
      allOf:
        - $ref: '#/components/schemas/RunResponseBase'
        - type: object
          required:
            - input
          properties:
            input:
              $ref: '#/components/schemas/ForecastInput'
              description: The query and it's compiled sql sent to the executor.
    CreateMaterialiedViewRunResponse:
      allOf:
        - $ref: '#/components/schemas/RunResponseBase'
        - type: object
          required:
            - input
          properties:
            input:
              $ref: '#/components/schemas/DatasetResponse'
              description: The dataset we are creating.
    RunResponseBase:
      type: object
      required:
        - id
        - company_id
        - completed_at
        - created_at
        - failures
        - idempotency_key
        - request_source
        - state
        - updated_at
      properties:
        id:
          $ref: '#/components/schemas/JobId'
        company_id:
          description: Company that ran the job.
          type: string
          format: uuid
        completed_at:
          description: The timestamp representing when the job finished.
          type: string
          format: date-time
        created_at:
          description: The timestamp representing when the job was created.
          type: string
          format: date-time
        failures:
          description: causes of job failure
          type: array
          items:
            $ref: '#/components/schemas/Failure'
        idempotency_key:
          description: The unique ID associated with the job.
          type: string
        request_source:
          $ref: '#/components/schemas/RequestSource'
        result:
          $ref: '#/components/schemas/ForecastResult'
        state:
          $ref: '#/components/schemas/State'
        updated_at:
          description: The timestamp representing when the job was updated.
          type: string
          format: date-time
    ForecastInput:
      type: object
      required:
        - nql
        - compiled_sql
      properties:
        nql:
          $ref: '#/components/schemas/NQL'
        compiled_sql:
          type: string
    DatasetResponse:
      type: object
      required:
        - id
        - company_id
        - created_at
        - name
        - display_name
        - retention_policy
        - schema
        - status
        - tags
        - write_mode
        - updated_at
      properties:
        id:
          $ref: '#/components/schemas/DatasetId'
        company_id:
          description: The ID of the company owning the dataset.
          type: integer
        compute_pool_config:
          description: Compute pool configuration for the dataset.
          type: object
          properties:
            default_compute_pool_id:
              description: The default compute pool to use for dataset operations.
              type: string
              format: uuid
        created_at:
          description: ISO-8601 timestamp indicating when the dataset was created
          type: string
        name:
          $ref: '#/components/schemas/DatasetName'
        display_name:
          $ref: '#/components/schemas/DatasetDisplayName'
        description:
          $ref: '#/components/schemas/DatasetDescription'
        retention_policy:
          $ref: '#/components/schemas/SnapshotRetentionPolicy'
        schema:
          $ref: '#/components/schemas/DatasetSchema'
        status:
          $ref: '#/components/schemas/DatasetStatus'
        tags:
          $ref: '#/components/schemas/DatasetTags'
        write_mode:
          $ref: '#/components/schemas/DatasetWriteMode'
        subscription_id:
          description: Id of a subscription which writes output to the dataset
          type: string
        updated_at:
          description: ISO-8601 timestamp indicating when the dataset was last updated.
          type: string
        last_snapshot_created_at:
          description: ISO-8601 timestamp indicating when the last snapshot was created.
          type: string
        materialized_view_config:
          description: >-
            Materialized-view configuration. Present only for materialized-view
            datasets.
          type: object
          required:
            - nql
          properties:
            nql:
              description: The NQL query that defines the materialized view.
              type: string
            refresh_schedule_config:
              description: The active refresh schedule, if one is set.
              type: object
              required:
                - schedule
                - schedule_zone_id
                - status
              properties:
                schedule:
                  type: string
                  description: >-
                    cron expression the refresh runs on, for example: @hourly, 0
                    12 * * 1-5
                schedule_zone_id:
                  type: string
                  description: 'zone id the schedule runs in, for example: EST, GMT+2'
                stats_enabled:
                  type: boolean
                  description: whether advanced statistics calculation is enabled
                status:
                  type: string
                  description: >-
                    active runs on the cron schedule; pending and archived do
                    not run
                  enum:
                    - active
                    - pending
                    - archived
                created_at:
                  type: string
                  description: >-
                    ISO-8601 timestamp indicating when the refresh schedule was
                    created.
                updated_at:
                  type: string
                  description: >-
                    ISO-8601 timestamp indicating when the refresh schedule was
                    last updated.
        stats:
          $ref: '#/components/schemas/DatasetResponseStats'
      example:
        id: 18923
        company_id: 382
        created_at: '2021-06-24T00:54:40.029056Z'
        description: >-
          Mobile identifiers tied to birth information provided as part of app
          registration.
        name: declared_age
        display_name: Declared Age
        schema:
          file_config:
            type: flat
            delimiter: ','
            escape: '"'
            header: true
            quote: '"'
          type: object
          properties:
            idValue:
              display_name: id value
              order: 0
              approximate_cardinality: 100000
              description: ID values
              type: string
            idType:
              display_name: id type
              order: 1
              approximate_cardinality: 1
              type: string
              enum:
                - md5_email
                - sha256_email
            eventTimestamp:
              display_name: event timestamp
              order: 2
              description: When the observation was collected
              type: timestamptz
              validations:
                - eventTimestamp < current_timestamp()
            countryCode:
              display_name: country code
              order: 3
              approximate_cardinality: 6
              description: The country of residence of the individual
              type: string
            age:
              display_name: age
              order: 4
              approximate_cardinality: 90
              description: How old this individual is in years
              type: long
            birthInfo:
              display_name: birth info
              order: 5
              description: The year this individual was born
              type: double
          required:
            - idValue
            - idType
            - eventTimestamp
            - countryCode
            - age
          primary: /age
        status: active
        tags:
          - age
          - demo
          - demographic
          - demographics
        updated_at: '2021-06-24T00:54:40.029056Z'
        write_mode: append
        subscription_id: 123e4567-e89b-12d3-a456-426614174000
        last_snapshot_created_at: '2023-06-09T16:51:34.215Z'
        stats:
          est_total_dataset_stored_files: 10
          est_total_dataset_stored_records: 500059
          est_total_dataset_stored_bytes: 4206454
          active_dataset_stored_files: 10
          active_dataset_stored_records: 500059
          active_dataset_stored_bytes: 4206454
          last_snapshot_added_files: 2
          last_snapshot_added_records: 499999
          last_snapshot_deleted_records: 0
          last_snapshot_added_bytes: 4193500
          last_snapshot_deleted_files: 0
          last_snapshot_added_delete_files: 0
          last_snapshot_removed_delete_files: 0
          last_snapshot_removed_bytes: 0
    JobId:
      type: string
      format: uuid
      description: Unique identifier for the job.
    Failure:
      type: object
      properties:
        message:
          type: string
        timestamp:
          type: string
          format: date-time
        value:
          description: >-
            useful information to diagnostic a failure such as a Apache Spark
            stacktrace
          type: object
    RequestSource:
      type: object
      properties:
        type:
          description: The job request type.
          type: string
        company_id:
          description: The company associated with the job.
          type: integer
        user_id:
          description: The user associated with the job.
          type: integer
      example:
        type: api_user
        company_id: 1
        user_id: 1248
    ForecastResult:
      type: object
      properties:
        nql_type:
          type: string
          const: forecast
        type:
          type: string
          const: success
        rows:
          type: integer
        cost:
          description: cost in micro cents USD
          type: integer
    State:
      type: string
      default: pending
      enum:
        - failed
        - cancelled
        - completed
        - pending
        - scheduled
        - pending_cancellation
        - running
    DatasetId:
      type: integer
      description: Unique identifier for the dataset.
    DatasetName:
      type: string
      description: >-
        The Dataset's name. Should be unique, contain only alpanum chars and
        underscores. Max length is 256.
    DatasetDisplayName:
      type: string
      description: The Dataset's name to display at UI.
    DatasetDescription:
      type: string
      description: A description of the data contained with the dataset.
    SnapshotRetentionPolicy:
      type: object
      additionalProperties: false
      description: >
        Backwards-compatible legacy payloads. Interpreted as snapshot-level
        retention on Iceberg. Returned response will be unified v1 shape with
        compat.mode=legacy.
      oneOf:
        - $ref: '#/components/schemas/ExpireEverything'
        - $ref: '#/components/schemas/RetainEverything'
        - $ref: '#/components/schemas/ExpireWhen'
        - $ref: '#/components/schemas/RetainWhen'
    DatasetSchema:
      description: >
        Schema describing a dataset's input file type and the kind of data the
        file contains.

        Note: when the file type is "flat" then the order of the properties
        describing the must be given in the order they appear in the input file.
      allOf:
        - type: object
          required:
            - file_config
          properties:
            file_config:
              $ref: '#/components/schemas/FileConfig'
            primary:
              $ref: '#/components/schemas/PrimaryField'
        - $ref: '#/components/schemas/Dataset_ObjectType'
    DatasetStatus:
      description: >-
        The dataset status.


        - `active` means that data can be added to the dataset but the schema is
        locked in place

        - `pending` means the dataset has been created, but the schema can still
        be altered and no data can be added to the dataset
      type: string
      enum:
        - active
        - pending
    DatasetTags:
      description: Tags that describe the dataset.
      type: array
      items:
        type: string
    DatasetWriteMode:
      description: >-
        How Narrative will treat new data uploaded for the dataset.

        - `append` means data will be added to the dataset, this is what to
        choose if you incrementally update your dataset over time.

        - `overwrite` means that all existing data will be overwritten with new
        data.
      type: string
      enum:
        - append
        - overwrite
    DatasetResponseStats:
      type: object
      required:
        - est_total_dataset_stored_files
        - est_total_dataset_stored_records
        - est_total_dataset_stored_bytes
        - active_dataset_stored_files
        - active_dataset_stored_records
        - active_dataset_stored_bytes
        - snapshot_added_files
        - snapshot_added_records
        - snapshot_deleted_records
        - snapshot_added_bytes
        - snapshot_deleted_files
        - snapshot_added_delete_files
        - snapshot_removed_delete_files
        - snapshot_removed_bytes
      properties:
        est_total_dataset_stored_files:
          $ref: '#/components/schemas/EstTotalDatasetStoredFiles'
        est_total_dataset_stored_records:
          $ref: '#/components/schemas/EstTotalDatasetStoredRecords'
        est_total_dataset_stored_bytes:
          $ref: '#/components/schemas/EstTotalDatasetStoredBytes'
        active_dataset_stored_files:
          $ref: '#/components/schemas/ActiveDatasetStoredFiles'
        active_dataset_stored_records:
          $ref: '#/components/schemas/ActiveDatasetStoredRecords'
        active_dataset_stored_bytes:
          $ref: '#/components/schemas/ActiveDatasetStoredBytes'
        snapshot_added_files:
          $ref: '#/components/schemas/SnapshotAddedFiles'
        snapshot_added_records:
          $ref: '#/components/schemas/SnapshotAddedRecords'
        snapshot_deleted_records:
          $ref: '#/components/schemas/SnapshotDeletedRecords'
        snapshot_added_bytes:
          $ref: '#/components/schemas/SnapshotAddedBytes'
        snapshot_deleted_files:
          $ref: '#/components/schemas/SnapshotDeletedFiles'
        snapshot_added_delete_files:
          $ref: '#/components/schemas/SnapshotAddedDeleteFiles'
        snapshot_removed_delete_files:
          $ref: '#/components/schemas/SnapshotRemovedDeleteFiles'
        snapshot_removed_bytes:
          $ref: '#/components/schemas/SnapshotRemovedBytes'
    ExpireEverything:
      type: object
      properties:
        type:
          type: string
          enum:
            - expire_everything
      example:
        type: expire_everything
    RetainEverything:
      type: object
      properties:
        type:
          type: string
          enum:
            - retain_everything
      example:
        type: retain_everything
    ExpireWhen:
      type: object
      properties:
        type:
          type: string
          enum:
            - expire_when
        expression:
          oneOf:
            - $ref: '#/components/schemas/SnapshotAge'
    RetainWhen:
      type: object
      properties:
        type:
          type: string
          enum:
            - retain_when
        expression:
          oneOf:
            - $ref: '#/components/schemas/SnapshotAge'
    FileConfig:
      description: >-
        Defines the input file format. Files provided to Narrative for ingestion
        into a Dataset must match the file configuration.
      oneOf:
        - $ref: '#/components/schemas/FlatFileConfig'
        - $ref: '#/components/schemas/JsonFileConfig'
        - $ref: '#/components/schemas/ParquetFileConfig'
    PrimaryField:
      description: >-
        An optional RFC-6901 JSON pointer to a Dataset field to indicate that
        the field is "primary" -- i.e. it is of particular interest to buyers.
        Narrative uses this information to optimize buyer experience when
        purchasing your Dataset.

        The primary key, if specified, must refer to a _required_ field that has
        a type of `boolean`, `long`, `string`, or `timestamptz`.
      type: string
    Dataset_ObjectType:
      type: object
      required:
        - type
        - properties
      properties:
        type:
          description: A record consisting of named fields of any type.
          enum:
            - object
        properties:
          type: object
          description: >-
            A map from field names to field types.


            Field names must consist of only alphanumeric characters or
            underscores and be < 255 characters long.
        required:
          description: >-
            An optional array of properties whose values are never missing or
            null.
          type: array
          items:
            type: string
        unsellable:
          description: An optional array of properties that cannot be sold to buyers.
          type: array
          items:
            type: string
        sensitive:
          description: >-
            An optional array of properties that are sensitive. Data in
            sensitive fields are redacted when showed to Narrative users via the
            UI or API but can still be bought and sold.
          type: array
          items:
            type: string
    EstTotalDatasetStoredFiles:
      description: >-
        The total estimated number of files across this dataset snapshot and all
        unexpired snapshots that precede it.
      type: integer
    EstTotalDatasetStoredRecords:
      description: >-
        The total estimated number of records across this dataset snapshot and
        all unexpired snapshots that precede it.
      type: integer
    EstTotalDatasetStoredBytes:
      description: >-
        The total estimated storage size in bytes used by this dataset snapshot
        and all unexpired snapshots that precede it.
      type: integer
    ActiveDatasetStoredFiles:
      description: The number of files in this dataset snapshot.
      type: integer
    ActiveDatasetStoredRecords:
      description: The number of records in this dataset snapshot.
      type: integer
    ActiveDatasetStoredBytes:
      description: The total storage size in bytes used by this dataset snapshot.
      type: integer
    SnapshotAddedFiles:
      description: The number of files added when generating this snapshot.
      type: integer
    SnapshotAddedRecords:
      description: The number of records added when generating this snapshot.
      type: integer
    SnapshotDeletedRecords:
      description: The number of records deleted when generating this snapshot.
      type: integer
    SnapshotAddedBytes:
      description: The amount of storage in bytes added when generating this snapshot.
      type: integer
    SnapshotDeletedFiles:
      description: The number of files deleted when generating this snapshot.
      type: integer
    SnapshotAddedDeleteFiles:
      description: The number of delete tracking files added when generating this snapshot.
      type: integer
    SnapshotRemovedDeleteFiles:
      description: >-
        The number of delete tracking files removed when generating this
        snapshot.
      type: integer
    SnapshotRemovedBytes:
      description: The amount of storage in bytes removed when generating this snapshot.
      type: integer
    SnapshotAge:
      type: object
      properties:
        type:
          type: string
          enum:
            - snapshot_age
        operator:
          type: string
          enum:
            - '>'
            - <
            - '>='
            - <=
        period:
          type: string
          description: An ISO 8601 period format
      example:
        type: snapshot_age
        operator: '>'
        period: P30D
    FlatFileConfig:
      type: object
      required:
        - type
        - header
        - delimiter
        - quote
        - escape
      properties:
        type:
          type: string
          description: >-
            Describes input files where each row is a set of fields separated by
            some delimiter. Common examples include comma-separated (CSV) and
            tab-separated (TSV) files. Files can be compressed using gzip or
            bzip2 as long as they include the appropriate suffix to indicate
            what kind of compression has been applied. E.g. bzip2 files must end
            in ".bz2" and gzipped files must end in ".gz".
          enum:
            - flat
        header:
          type: boolean
          description: Whether or not files include a header.
        delimiter:
          type: string
          description: Character separating values within a row in a file.
        quote:
          type: string
          description: >-
            Character used for escaping quoted values where the separator can be
            part of the value.
        escape:
          type: string
          description: Character used for escaping quotes inside an already quoted value.
    JsonFileConfig:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: >-
            Describes input files in the [JSON Lines](https://jsonlines.org/)
            file format.
          enum:
            - json
    ParquetFileConfig:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: >-
            Describes input files in the [Parquet](https://parquet.apache.org/)
            file format.
          enum:
            - parquet
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer

````