> ## Documentation Index
> Fetch the complete documentation index at: https://docs.narrative.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Refresh Materialized View

> Starts a run of the view's refresh workflow. The response has the dataset and the workflow's id.

A materialized view has one refresh workflow, so every call returns the same workflow id. Scheduled refreshes are
runs of the same workflow. `GET /workflows/{workflow_id}/runs` lists them all, and needs `read` access to
`workflows`.

The request is refused with `DatasetNotFoundError` when no dataset with the id belongs to the caller's company, or
when the dataset isn't `active`.

The request is refused with `MaterializedViewRefreshInProgressError` (409) when there is already a pending or
running refresh job for the view. It's refused with a 403 when the company is over its credit limit or has no
credit account with Narrative, and with `RefreshWorkflowRefusedError` (503) when the refresh workflow couldn't be
started for another reason. A 503 can be retried.

Requires `write` access to `datasets`, and a token issued to a user: a user's token, or an app installation's
token that acts for a user.




## OpenAPI

````yaml https://docs-cdn.narrative.io/api-reference/main/openapi.json post /v2/datasets/{dataset_id}/refresh-materialized-view
openapi: 3.1.1
info:
  contact:
    email: support@narrative.io
    name: Narrative Support
    url: https://www.narrative.io
  termsOfService: https://www.narrative.io/legal/terms-of-service
  x-logo:
    url: >-
      https://cdn.narrative.io/images/company-logos/prod/narrative-logo-text-white.svg
    backgroundColor: rgb(9, 34, 166)
    altText: Narrative Logo
  description: >-
    The [Narrative Data Collaboration Platform](https://app.narrative.io) API is
    organized around REST. Our API has predictable resource-oriented URLs,
    accepts form-encoded request bodies, returns JSON-encoded responses, and
    uses standard HTTP response codes, authentication, and verbs.



    The current version is a pre-release beta.  It may result in unexpected
    behavior and there may be breaking changes in future releases up to the 1.0
    release.
  title: Narrative Data Collaboration Platform API
  version: 2.4.x
servers:
  - url: https://api-dev.narrative.io
  - url: https://api.narrative.io
security: []
tags:
  - name: Access Rules
    description: >-
      Access rules let data providers control who can purchase their data and
      the terms of purchase.


      The `access-rules` API allows you to manage access rules for your
      datasets.


      Related guides:
        - [What is an access rule?](https://kb.narrative.io/access-rules)
  - name: Agent Conversations
    description: >-
      Build LLM-driven workflows that can call MCP tools, ask the caller for
      input, and return structured answers.


      A conversation pins a model, a system prompt, and a tool catalog. Each run
      sends the model a user message (or

      a batch of tool outputs from a previously-paused run); the model decides
      whether to answer directly, call a

      server-side tool (resolved by the platform via Model Context Protocol), or
      call a client-side tool (which pauses

      the run with `requires_action` and waits for the caller to reply).


      Related guides:
        - [Agent Conversations Reference](https://docs.narrative.io/reference/architecture/agent-conversations)
        - [Error catalog](https://docs.narrative.io/reference/architecture/agent-conversations/errors/conversation-not-found)
  - name: MCP Connections
    description: >-
      Connect external (non-Narrative) MCP servers so agent runs can call their
      tools with the

      calling user's own OAuth authorization.


      A connection is created interactively: `POST /mcp-connections` runs OAuth
      discovery and

      Dynamic Client Registration against the server and returns a consent URL;
      the user authorizes

      in a browser; the authorization server redirects to `GET
      /mcp-connections/callback`, which

      stores the authorization code and sends the browser on to the Narrative
      app; the app calls

      `POST /mcp-connections/complete` with the user's own bearer token, which
      exchanges the code and

      marks the connection `connected`.


      That last step is authenticated on purpose. The callback is public — it's
      a cross-site browser

      redirect, so no bearer reaches it — so it is not allowed to complete
      anything. Only the user

      who started the flow can turn the stored code into a token.


      Tokens are stored encrypted and used server-side — they are never returned
      by the API. Once

      connected, reference the connection by id from an agent run's
      `mcp_servers[].connection_id`.
  - name: App Invites
    description: >-
      App invites allow applications to create one-time, shareable links on
      behalf of their users. These links

      enable users to invite third parties who do not have a Narrative account
      to perform actions within the

      application.


      For example, a Narrative user can send an invite link to a third party who
      then completes a Pinterest

      OAuth flow and creates a connector profile in the inviter's account,
      allowing the inviter to deliver

      audience data to the third party's Pinterest account without the third
      party needing a Narrative account.


      This API is intended to be used by applications (via app client
      credentials) to create and manage invites

      on behalf of their users, which can then be shared with invitees.
  - name: Apps
    description: >-
      Apps are applications bundled with a UI that can perform various actions
      on behalf of a user utilizing the Narrative API.

      Related guides:
        - [Building a Narrative Native App](https://www.narrative.io/knowledge-base/how-to-guides/building-a-narrative-native-app)
  - name: Access Tokens
    description: >-
      System Access Tokens are long-lived bearer credentials for calling the API
      without a user present.


      A token belongs to a company, carries an explicit list of permissions, and
      expires on a date you choose — at

      least a day out and at most a year. `admin` access cannot be granted to
      one.


      The credential is returned once, when a token is created or its credential
      is rotated. Everything else works

      in terms of the token's id: the list, get and update responses describe a
      token without its `access_token`.

      If you lose the credential, rotate it with `POST
      /access-tokens/tokens/{access_token_id}/rotate`, which keeps

      the token's id and revokes the credential it had.


      Related guides:
        - [How to Create an API Token](https://www.narrative.io/knowledge-base/how-to-guides/understanding-narratives-apis/create-an-api-token)
  - name: Attributes
    description: >-
      An attribute models a standardized data point available for sale on the
      Narrative marketplace.


      Narrative automatically turns data points from provider datasets into
      attributes so that buyers can purchase well-formed, standardized data from
      any supplier on the marketplace.
  - name: Auth
    description: >-
      API token is a crucial step for developers to securely authenticate
      requests to the Narrative API

      Related guides:
        - [How to Create an API Token](https://www.narrative.io/knowledge-base/how-to-guides/understanding-narratives-apis/create-an-api-token)
  - name: Authentication
    description: User login and registration
  - name: Billings
    description: Used by Narrative internally to bill customers
  - name: Companies
    description: A collection of employees
  - name: Company Marketing Information
    description: Useful information related to companies
  - name: Compute Pools
    description: >-
      Compute pools represent compute resources (e.g. Snowflake warehouses)
      provisioned within a data plane.

      Companies can manage and share compute pools, assign them to jobs, and set
      a default compute pool on data planes.
  - name: Connections
    description: Associations between connectors and datasets
  - name: Datasets
    description: >-
      Any kind of data, in any schema, can be pushed into the Narrative Data
      Collaboration Platform as a dataset exactly as it is stored in your own
      system.


      The `datasets` API allows you to manage your datasets.
  - name: Derivations
    description: >-
      Derivation rules describe how the value of one Rosetta Stone attribute can
      be computed from another — for

      example, hashing a raw email into a SHA-256 hashed email. Each rule
      carries a transformation expression that

      reads the source attribute's value through `$source`.


      The `derivations` API allows you to browse the rules your company can use
      — the ones it owns plus the ones

      shared with it through collaborators. Creating and changing them requires
      an `admin` grant on

      `attribute_derivations`, and a rule can only be changed by the company
      that owns it.
  - name: Installations
    description: Installations of Applications for a profile
  - name: Jobs
    description: >-
      Jobs represent an operation done on a given data plane. All jobs today are
      tied to a query that represents a forecast or a materialized view.


      The jobs API provides an interface for interacting with the jobs table,
      which stores various operations involving reading or writing data. This
      API allows users to retrieve detailed information about specific jobs,
      including NQL forecasts and materialized views.
  - name: Mappings
    description: >-
      A mapping is a transformation from a dataset to an attribute. Defining a
      mapping between a dataset and an attribute makes the dataset eligible to
      participate in subscriptions where a buyer is purchasing the target
      attribute.
  - name: Model Inference
    description: Model Inference
  - name: Model Training
    description: >-
      Train machine learning models on datasets, e.g. text classifiers used to
      power attribute mappings.
  - name: Models
    description: >-
      Machine learning models for training and inference.


      Models can be stored in HuggingFace or Narrative repositories and have
      configurable

      collaborator permissions for training and inference access.


      The `models` API allows you to list, retrieve, and update models
      accessible to your company.
  - name: NQL
    description: >-
      Narrative Query Language (NQL) is a specialized, SQL-inspired language
      designed to query and manipulate data within the Narrative platform. While
      it looks and feels much like standard SQL, it offers extended
      functionality and syntax that let you leverage platform-specific
      features—such as referencing datasets by their IDs, creating materialized
      views, or generating forecasts—without having to manage the complexities
      of different query engines behind the scenes. NQL queries can ultimately
      compile down to multiple underlying engines (e.g., Snowflake, Spark) to
      execute your requests efficiently in the Narrative ecosystem.


      Datasets, views, and access rules share one NQL naming namespace per
      company. Use

      `GET /nql/name-availability?name=monthly_purchases` to check whether a
      name is free before creating an object.

      Archiving a dataset, view, or access rule releases its name. Availability
      is a check, not a reservation.
  - name: Profiles
    description: >-
      App profiles are associated with an installation. They represent a
      reference to a configuration that the app can use to save confidential
      information outside of Narrative's control.

      Profiles are currently used to configure settings for connector apps.
  - name: Resources
    description: >-
      Narrative gives you access to managed resources, like your own AWS S3
      bucket, so that you can effortlessly buy and sell data on the platform.


      The `resources` API allows you to manage your resources.
  - name: Schema Inference
    description: >-
      The `schema-inference` API analyzes submitted files to automatically infer
      and return their structure as a dataset schema.
  - name: Uploads
    description: >-
      The `uploads` API allows you to send files to Narrative and use them to
      perform tasks like creating a list or adding data to a dataset.
  - name: Usage
    description: >-
      The `usage` API enables the recording of usage events associated with a
      product.
  - name: Webhooks
    description: >-
      Webhooks push events to a URL you control instead of making you poll for
      them.


      You create a subscription with the endpoint you want events sent to and a
      filter describing which events you

      want. Two kinds of subscription exist: job subscriptions follow the
      lifecycle of Narrative jobs, and app

      subscriptions follow the events a Narrative app reports through `POST
      /apps/events`.


      Creating a subscription returns a `secret`. Narrative sends it back in the
      `X-Narrative-Secret` header on

      every delivery, and comparing the two is how you tell a real callback from
      a forged one — so store it and do

      not share it.


      Deliveries are described under the "Event delivery" webhook below. Return
      any 2xx; anything else is retried

      with exponential backoff, and because a retry reuses the envelope's `id`,
      handlers should be idempotent

      on it.
  - name: Workflows
    description: >-
      The `workflows` API allows you to create, schedule, trigger, and archive
      workflows.

      Workflows are defined using a serverlessworkflow YAML specification.
paths:
  /v2/datasets/{dataset_id}/refresh-materialized-view:
    post:
      tags:
        - Datasets
      summary: Refresh Materialized View
      description: >
        Starts a run of the view's refresh workflow. The response has the
        dataset and the workflow's id.


        A materialized view has one refresh workflow, so every call returns the
        same workflow id. Scheduled refreshes are

        runs of the same workflow. `GET /workflows/{workflow_id}/runs` lists
        them all, and needs `read` access to

        `workflows`.


        The request is refused with `DatasetNotFoundError` when no dataset with
        the id belongs to the caller's company, or

        when the dataset isn't `active`.


        The request is refused with `MaterializedViewRefreshInProgressError`
        (409) when there is already a pending or

        running refresh job for the view. It's refused with a 403 when the
        company is over its credit limit or has no

        credit account with Narrative, and with `RefreshWorkflowRefusedError`
        (503) when the refresh workflow couldn't be

        started for another reason. A 503 can be retried.


        Requires `write` access to `datasets`, and a token issued to a user: a
        user's token, or an app installation's

        token that acts for a user.
      operationId: RefreshMaterializedView
      parameters:
        - name: dataset_id
          in: path
          schema:
            type: integer
            format: int64
          required: true
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/RefreshMaterializedViewRequest'
        required: true
      responses:
        '202':
          description: RefreshMaterializedView 202 response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/RefreshMaterializedViewOutput'
        '400':
          description: >-
            One of the errors below. The `X-Error-Type` response header names
            which one.


            - `DatasetNotRefreshableError`: The dataset can't be refreshed as
            requested, and no refresh workflow was started. The dataset isn't a
            materialized view, it doesn't have a refresh workflow, its first
            refresh hasn't completed yet, or the refresh type doesn't fit it: an
            incremental refresh was requested for a view that overwrites its
            data, or a full refresh for a view that appends to its data with a
            query that can't be recompiled.

            - `InvalidComputePoolError`: The compute pool doesn't exist, isn't
            active, isn't on the dataset's data plane, or the company has no
            access to it. For refresh requests: even when no pool is specified,
            the one that will be used is validated according to the same rules.

            - `BadRequestError`: The request can't be read, or one of its values
            breaks a rule the API documents.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorBody'
        '401':
          description: UnauthorizedError 401 response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/UnauthorizedError'
        '403':
          description: >-
            One of the errors below. The `X-Error-Type` response header names
            which one.


            - `ForbiddenError`: The request carries no valid bearer token, or
            the caller isn't allowed to do this.

            - `UserTokenRequiredError`: The operation needs a token issued to a
            user, and the caller's token has none.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorBody'
        '404':
          description: DatasetNotFoundError 404 response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/DatasetNotFoundError'
        '409':
          description: MaterializedViewRefreshInProgressError 409 response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/MaterializedViewRefreshInProgressError'
        '500':
          description: InternalError 500 response
        '503':
          description: RefreshWorkflowRefusedError 503 response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/RefreshWorkflowRefusedError'
      security:
        - BearerAuth: []
components:
  schemas:
    RefreshMaterializedViewRequest:
      type: object
      properties:
        type:
          $ref: '#/components/schemas/MaterializedViewRefreshType'
        compute_pool_id:
          type: string
          description: >-
            The compute pool that runs the refresh. Defaults to the pool of the
            view's refresh schedule, and then to the

            dataset's default pool.
      required:
        - type
      example:
        type: full
    RefreshMaterializedViewOutput:
      type: object
      properties:
        dataset:
          $ref: '#/components/schemas/DatasetResponse'
        workflow_id:
          type: string
          description: The id of the workflow that refreshes the view.
      required:
        - dataset
        - workflow_id
    ErrorBody:
      type: object
      description: >-
        The body of an error. `error` is a short code for the kind of failure,
        and `error_description` says what went wrong.
      properties:
        error:
          type: string
        error_description:
          type: string
      required:
        - error
        - error_description
    UnauthorizedError:
      type: object
      description: The token has expired, or lacks a permission the operation requires.
      properties:
        error:
          type: string
        error_description:
          type: string
      required:
        - error
        - error_description
    DatasetNotFoundError:
      type: object
      description: >-
        No dataset with this id belongs to the caller's company. Refreshing also
        refuses a dataset that isn't active

        with this error.
      properties:
        error:
          type: string
        error_description:
          type: string
      required:
        - error
        - error_description
    MaterializedViewRefreshInProgressError:
      type: object
      description: >-
        There is already a pending or running refresh job for the view, so no
        refresh workflow was started. The message has

        the job's id.
      properties:
        error:
          type: string
        error_description:
          type: string
      required:
        - error
        - error_description
    RefreshWorkflowRefusedError:
      type: object
      description: >-
        The view's refresh workflow couldn't be started. The request can be
        retried.
      properties:
        error:
          type: string
        error_description:
          type: string
      required:
        - error
        - error_description
    MaterializedViewRefreshType:
      type: string
      description: >-
        `full` runs the view from scratch, without its progress tracking.
        `incremental` continues from the progress

        tracking of earlier refreshes, and needs a view that appends to its
        table.
      enum:
        - full
        - incremental
    DatasetResponse:
      type: object
      description: >-
        A dataset.


        The encoder drops nulls and empty collections, so an optional field is
        absent rather than null,

        and `tags` and `mappings` are absent rather than `[]` when there are
        none. `DatasetAdminResponse`

        composes this schema with `allOf` but encodes without that step, so on
        the admin endpoints the

        same optional fields arrive as null. Either way they are optional, and
        both shapes validate.
      required:
        - id
        - company_id
        - created_at
        - data_plane
        - is_narrative_managed
        - name
        - display_name
        - retention_policy
        - schema
        - status
        - write_mode
        - updated_at
      properties:
        id:
          $ref: '#/components/schemas/DatasetId'
        company_id:
          description: The ID of the company owning the dataset.
          type: integer
        data_plane:
          $ref: '#/components/schemas/DataPlaneResponse'
          description: The data plane the dataset lives on.
        external_id:
          description: >-
            The dataset's identifier in the system it came from. Absent when the
            dataset did not come from

            one.
          type:
            - string
            - 'null'
        is_narrative_managed:
          description: >-
            Whether Narrative manages the dataset rather than the owning
            company.
          type: boolean
        mappings:
          description: >-
            The mappings from this dataset's fields to attributes. Absent when
            the dataset has none.
          type: array
          items:
            $ref: '#/components/schemas/MappingResponse'
        compute_pool_config:
          description: Compute pool configuration for the dataset.
          type: object
          properties:
            default_compute_pool_id:
              description: The default compute pool to use for dataset operations.
              type: string
              format: uuid
        created_at:
          description: ISO-8601 timestamp indicating when the dataset was created
          type: string
        name:
          $ref: '#/components/schemas/DatasetName'
        display_name:
          $ref: '#/components/schemas/DatasetDisplayName'
        description:
          $ref: '#/components/schemas/DatasetDescription'
        retention_policy:
          $ref: '#/components/schemas/UnifiedV1Response'
        schema:
          $ref: '#/components/schemas/DatasetSchema'
        status:
          $ref: '#/components/schemas/DatasetStatus'
        tags:
          $ref: '#/components/schemas/DatasetTags'
        write_mode:
          $ref: '#/components/schemas/DatasetWriteMode'
        subscription_id:
          description: Id of a subscription which writes output to the dataset
          type:
            - string
            - 'null'
        creation_workflow_id:
          description: >-
            The workflow `POST /v2/datasets` started to create the dataset.
            Absent for datasets created

            any other way.
          type:
            - string
            - 'null'
          format: uuid
        updated_at:
          description: ISO-8601 timestamp indicating when the dataset was last updated.
          type: string
        last_snapshot_created_at:
          description: ISO-8601 timestamp indicating when the last snapshot was created.
          type:
            - string
            - 'null'
        materialized_view_config:
          description: >-
            Materialized-view configuration. Present only for materialized-view
            datasets.
          type:
            - object
            - 'null'
          required:
            - nql
          properties:
            nql:
              description: The NQL query that defines the materialized view.
              type: string
            refresh_schedule_config:
              description: The active refresh schedule, if one is set.
              type: object
              required:
                - schedule
                - schedule_zone_id
                - status
              properties:
                schedule:
                  type: string
                  description: >-
                    cron expression the refresh runs on, for example: @hourly, 0
                    12 * * 1-5
                schedule_zone_id:
                  type: string
                  description: 'zone id the schedule runs in, for example: EST, GMT+2'
                stats_enabled:
                  type: boolean
                  description: whether advanced statistics calculation is enabled
                status:
                  type: string
                  description: >-
                    active runs on the cron schedule; pending and archived do
                    not run
                  enum:
                    - active
                    - pending
                    - archived
                created_at:
                  type: string
                  description: >-
                    ISO-8601 timestamp indicating when the refresh schedule was
                    created.
                updated_at:
                  type: string
                  description: >-
                    ISO-8601 timestamp indicating when the refresh schedule was
                    last updated.
        stats:
          oneOf:
            - $ref: '#/components/schemas/DatasetResponseStats'
            - type: 'null'
      example:
        id: 18923
        company_id: 382
        created_at: '2021-06-24T00:54:40.029056Z'
        data_plane:
          type: owned
          id: c0bed5c2-8af8-4f7a-9787-1ffc3def339b
          company_id: 382
          created_at: '2021-06-01T00:00:00Z'
          description: The company's Snowflake account
          display_name: Snowflake (us-west-2)
          external_id: snowflake-prod
          is_narrative: false
          platform:
            type: platform_snowflake
            account_locator: ab12345
            account_name: acme
            organization_name: acmeorg
            region:
              type: region_aws
              id: us-west-2
          status: active
          tags: []
          updated_at: '2021-06-01T00:00:00Z'
          collaborators:
            participants:
              type: all
            manage_compute_pools:
              type: all
          compute_pools: []
        is_narrative_managed: true
        retention_policy:
          policies:
            - type:
                kind: snapshot_ttl
                snapshot_retention_policy:
                  type: retain_everything
              enabled: true
        description: >-
          Mobile identifiers tied to birth information provided as part of app
          registration.
        name: declared_age
        display_name: Declared Age
        schema:
          file_config:
            type: flat
            delimiter: ','
            escape: '"'
            header: true
            quote: '"'
          type: object
          properties:
            idValue:
              display_name: id value
              order: 0
              approximate_cardinality: 100000
              description: ID values
              type: string
            idType:
              display_name: id type
              order: 1
              approximate_cardinality: 1
              type: string
              enum:
                - md5_email
                - sha256_email
            eventTimestamp:
              display_name: event timestamp
              order: 2
              description: When the observation was collected
              type: timestamptz
              validations:
                - eventTimestamp < current_timestamp()
            countryCode:
              display_name: country code
              order: 3
              approximate_cardinality: 6
              description: The country of residence of the individual
              type: string
            age:
              display_name: age
              order: 4
              approximate_cardinality: 90
              description: How old this individual is in years
              type: long
            birthInfo:
              display_name: birth info
              order: 5
              description: The year this individual was born
              type: double
          required:
            - idValue
            - idType
            - eventTimestamp
            - countryCode
            - age
          primary: /age
        status: active
        tags:
          - age
          - demo
          - demographic
          - demographics
        updated_at: '2021-06-24T00:54:40.029056Z'
        write_mode: append
        subscription_id: 123e4567-e89b-12d3-a456-426614174000
        last_snapshot_created_at: '2023-06-09T16:51:34.215Z'
        stats:
          est_total_dataset_stored_files: 10
          est_total_dataset_stored_records: 500059
          est_total_dataset_stored_bytes: 4206454
          active_dataset_stored_files: 10
          active_dataset_stored_records: 500059
          active_dataset_stored_bytes: 4206454
          last_snapshot_added_files: 2
          last_snapshot_added_records: 499999
          last_snapshot_deleted_records: 0
          last_snapshot_added_bytes: 4193500
          last_snapshot_deleted_files: 0
          last_snapshot_added_delete_files: 0
          last_snapshot_removed_delete_files: 0
          last_snapshot_removed_bytes: 0
    DatasetId:
      type: integer
      description: Unique identifier for the dataset.
    DataPlaneResponse:
      oneOf:
        - $ref: '#/components/schemas/DataPlaneOwnedResponse'
        - $ref: '#/components/schemas/DataPlaneSharedResponse'
      discriminator:
        propertyName: type
        mapping:
          owned: '#/components/schemas/DataPlaneOwnedResponse'
          shared: '#/components/schemas/DataPlaneSharedResponse'
    MappingResponse:
      type: object
      properties:
        id:
          allOf:
            - $ref: '#/components/schemas/MappingId'
            - description: >-
                Absent on a derived mapping (`source: derived`), which is
                computed per request and cannot be fetched or

                changed by id.
        attribute_id:
          type: integer
          description: The target attribute's ID.
        created_at:
          type: string
          description: ISO-8601 timestamp indicating when the mapping was created.
          example: '2021-08-26T21:06:07.710357Z'
        dataset_id:
          type: integer
          description: The source dataset's ID.
        mapping:
          $ref: '#/components/schemas/MappingDefinition'
        status:
          $ref: '#/components/schemas/MappingStatus'
        updated_at:
          type: string
          description: ISO-8601 timestamp indicating when the mapping was last updated.
          example: '2021-08-26T21:06:07.710357Z'
        created_by:
          allOf:
            - $ref: '#/components/schemas/UserId'
            - description: Unique id from a user who created it.
        updated_by:
          allOf:
            - $ref: '#/components/schemas/UserId'
            - description: Unique id from a user who updated it.
        company_id:
          description: The company that owns the mapping. Null for platform-wide mappings.
          oneOf:
            - $ref: '#/components/schemas/CompanyId'
            - type: 'null'
        scope:
          $ref: '#/components/schemas/MappingScope'
        source:
          $ref: '#/components/schemas/MappingSource'
        provenance:
          allOf:
            - $ref: '#/components/schemas/MappingProvenance'
            - description: >-
                How a derived mapping was produced. Present only when `source`
                is `derived`.
        tags:
          $ref: '#/components/schemas/MappingTags'
        derived_from:
          description: >-
            The stored mapping this one was copied from when it was authored
            (lineage). Unrelated to derivation rules;

            see `provenance` for those. Null when the mapping was authored
            directly.
          oneOf:
            - $ref: '#/components/schemas/MappingId'
            - type: 'null'
      required:
        - attribute_id
        - created_at
        - dataset_id
        - mapping
        - status
        - updated_at
        - created_by
        - updated_by
        - scope
        - source
      example:
        id: ea9dddd2-e3ee-40b5-b03e-c3cd37c8a7f0
        attribute_id: 1
        created_at: '2021-12-10T00:00:00Z'
        dataset_id: 3
        mapping:
          type: object_mapping
          property_mappings:
            - path: foo
              expression: upper(`US Kennel ID`)
              dependencies:
                properties:
                  - US Kennel ID
            - path: bar.baz.qux
              expression: weight * 10000
              dependencies:
                properties:
                  - weight
        status: active
        updated_at: '2021-12-10T00:00:01Z'
        created_by: 1
        updated_by: 1
        company_id: 100
        scope: global
        source: admin
        derived_from: ea9dddd2-e3ee-40b5-b03e-c3cd37c8a6f0
    DatasetName:
      type: string
      description: >-
        The Dataset's name. Should be unique, contain only alpanum chars and
        underscores. Max length is 256.
    DatasetDisplayName:
      type: string
      description: The Dataset's name to display at UI.
    DatasetDescription:
      type:
        - string
        - 'null'
      description: A description of the data contained with the dataset.
    UnifiedV1Response:
      type: object
      additionalProperties: false
      description: >
        A dataset's retention policies, plus the schedule they are enforced on.
        This is the shape every dataset response carries; legacy snapshot-only
        payloads are accepted on write and returned in this form.
      properties:
        policies:
          type: array
          description: Left out when the dataset has no retention policies.
          items:
            $ref: '#/components/schemas/RetentionPolicy'
        schedule_config:
          $ref: '#/components/schemas/RefreshSchedule'
      example:
        policies:
          - type:
              kind: table_ttl
              clock:
                kind: created_at
            enabled: true
            interval: P1D
        schedule_config:
          schedule: '@hourly'
          schedule_zone_id: UTC
          status: active
    DatasetSchema:
      description: >
        Schema describing a dataset's input file type and the kind of data the
        file contains.

        Note: when the file type is "flat" then the order of the properties
        describing the must be given in the order they appear in the input file.
      allOf:
        - type: object
          required:
            - file_config
          properties:
            file_config:
              $ref: '#/components/schemas/FileConfig'
            primary:
              $ref: '#/components/schemas/PrimaryField'
            partition_key:
              $ref: '#/components/schemas/PartitionKey'
        - $ref: '#/components/schemas/DatasetObjectType'
    DatasetStatus:
      description: >-
        The dataset status.


        - `active` means that data can be added to the dataset but the schema is
        locked in place

        - `pending` means the dataset has been created, but the schema can still
        be altered and no data can be added to the dataset

        - `archived` means the dataset has been retired and can no longer be
        read from or written to
      type: string
      enum:
        - active
        - pending
        - archived
    DatasetTags:
      description: Tags that describe the dataset.
      type: array
      items:
        type: string
    DatasetWriteMode:
      description: >-
        How Narrative will treat new data uploaded for the dataset.

        - `append` means data will be added to the dataset, this is what to
        choose if you incrementally update your dataset over time.

        - `overwrite` means that all existing data will be overwritten with new
        data.
      type: string
      enum:
        - append
        - overwrite
    DatasetResponseStats:
      type: object
      required:
        - est_total_dataset_stored_files
        - est_total_dataset_stored_records
        - est_total_dataset_stored_bytes
        - active_dataset_stored_files
        - active_dataset_stored_records
        - active_dataset_stored_bytes
        - last_snapshot_added_files
        - last_snapshot_added_records
        - last_snapshot_deleted_records
        - last_snapshot_added_bytes
        - last_snapshot_deleted_files
        - last_snapshot_added_delete_files
        - last_snapshot_removed_delete_files
        - last_snapshot_removed_bytes
      properties:
        est_total_dataset_stored_files:
          $ref: '#/components/schemas/EstTotalDatasetStoredFiles'
        est_total_dataset_stored_records:
          $ref: '#/components/schemas/EstTotalDatasetStoredRecords'
        est_total_dataset_stored_bytes:
          $ref: '#/components/schemas/EstTotalDatasetStoredBytes'
        active_dataset_stored_files:
          $ref: '#/components/schemas/ActiveDatasetStoredFiles'
        active_dataset_stored_records:
          $ref: '#/components/schemas/ActiveDatasetStoredRecords'
        active_dataset_stored_bytes:
          $ref: '#/components/schemas/ActiveDatasetStoredBytes'
        last_snapshot_added_files:
          $ref: '#/components/schemas/SnapshotAddedFiles'
        last_snapshot_added_records:
          $ref: '#/components/schemas/SnapshotAddedRecords'
        last_snapshot_deleted_records:
          $ref: '#/components/schemas/SnapshotDeletedRecords'
        last_snapshot_added_bytes:
          $ref: '#/components/schemas/SnapshotAddedBytes'
        last_snapshot_deleted_files:
          $ref: '#/components/schemas/SnapshotDeletedFiles'
        last_snapshot_added_delete_files:
          $ref: '#/components/schemas/SnapshotAddedDeleteFiles'
        last_snapshot_removed_delete_files:
          $ref: '#/components/schemas/SnapshotRemovedDeleteFiles'
        last_snapshot_removed_bytes:
          $ref: '#/components/schemas/SnapshotRemovedBytes'
    DataPlaneOwnedResponse:
      type: object
      required:
        - type
        - id
        - company_id
        - created_at
        - display_name
        - external_id
        - platform
        - status
        - updated_at
        - collaborators
        - compute_pools
        - is_narrative
      properties:
        type:
          type: string
          enum:
            - owned
        id:
          type: string
          format: uuid
        company_id:
          $ref: '#/components/schemas/CompanyId'
        created_at:
          type: string
          format: date-time
        description:
          type:
            - string
            - 'null'
        display_name:
          type: string
        external_id:
          type: string
        platform:
          $ref: '#/components/schemas/Platform'
        status:
          type: string
          enum:
            - active
            - archived
        last_heartbeat_at:
          type:
            - string
            - 'null'
          format: date-time
        tags:
          type:
            - array
            - 'null'
          items:
            type: string
        updated_at:
          type: string
          format: date-time
        collaborators:
          $ref: '#/components/schemas/DataPlaneCollaborators'
        default_compute_pool_id:
          type:
            - string
            - 'null'
          format: uuid
        compute_pools:
          type: array
          items:
            $ref: '#/components/schemas/ComputePoolResponse'
        is_narrative:
          description: >-
            True for the shared data plane Narrative operates. Use this rather
            than matching on `display_name`, which the owner can change.
          type: boolean
    DataPlaneSharedResponse:
      type: object
      required:
        - type
        - id
        - company_id
        - created_at
        - display_name
        - platform
        - status
        - updated_at
        - compute_pools
        - is_narrative
      properties:
        type:
          type: string
          enum:
            - shared
        id:
          type: string
          format: uuid
        company_id:
          $ref: '#/components/schemas/CompanyId'
        created_at:
          type: string
          format: date-time
        description:
          type:
            - string
            - 'null'
        display_name:
          type: string
        platform:
          $ref: '#/components/schemas/PlatformSharedResponse'
        status:
          type: string
          enum:
            - active
            - archived
        last_heartbeat_at:
          type:
            - string
            - 'null'
          format: date-time
        tags:
          type:
            - array
            - 'null'
          items:
            type: string
        updated_at:
          type: string
          format: date-time
        default_compute_pool_id:
          type:
            - string
            - 'null'
          format: uuid
        compute_pools:
          type: array
          items:
            $ref: '#/components/schemas/ComputePoolResponse'
        is_narrative:
          description: >-
            True for the shared data plane Narrative operates. Use this rather
            than matching on `display_name`, which the owner can change.
          type: boolean
    MappingId:
      description: Unique identifier for a mapping.
      type: string
      format: uuid
      example: ea9dddd2-e3ee-40b5-b03e-c3cd37c8a6f0
    MappingDefinition:
      oneOf:
        - $ref: '#/components/schemas/ObjectMapping'
        - $ref: '#/components/schemas/ValueMapping'
        - $ref: '#/components/schemas/CachedMapping'
    MappingStatus:
      type: string
      description: |
        The status of the mapping.
          * `active` - Active mappings can be used
          * `archived` - Archived/Deleted mappings - mapping no longer in use
          * `pending` - Proposed mappings that are not active
          * `rejected` - Proposed mappings the owner turned down
      enum:
        - active
        - archived
        - pending
        - rejected
    UserId:
      type: integer
      format: int64
      description: The id of an existing user
      example: 20
    CompanyId:
      type: integer
      format: int64
      description: The id of an existing company
      example: 345
    MappingScope:
      type: string
      description: |
        The scope of the mapping.
          * `global` - Mappings available to all
          * `private` - Company private mappings
      enum:
        - global
        - private
    MappingSource:
      type: string
      description: |
        The source of the mapping.
          * `company` - Company private mapping
          * `admin` - Admin created/promoted mapping
          * `system` - System (Automated/ML) mappings
          * `lineage` - Lineage-derived mappings (editable without admin)
          * `rosetta_stone` - Rosetta Stone mappings (editable without admin)
          * `derived` - Computed on the request from a stored mapping and a derivation rule. Not stored, so it has
            no `id`; listed only when asked for by source
      enum:
        - company
        - admin
        - system
        - lineage
        - rosetta_stone
        - derived
    MappingProvenance:
      type: object
      description: How a derived mapping was produced.
      required:
        - source_mapping_id
        - source_attribute_id
        - derivation_path
        - lossy
        - imprecise
      properties:
        source_mapping_id:
          allOf:
            - $ref: '#/components/schemas/MappingId'
            - description: The stored mapping the derivation was applied to.
        source_attribute_id:
          type: integer
          description: The attribute of the source mapping.
        derivation_path:
          type: array
          minItems: 1
          description: >-
            The derivation rules applied, in order, from the source attribute to
            this mapping's attribute. One

            element today; longer once derivations chain.
          items:
            type: integer
            description: A derivation rule id, as returned by `GET /derivations/{id}`.
        lossy:
          type: boolean
          description: >-
            True when any rule on the path is lossy (the source value cannot be
            recovered).
        imprecise:
          type: boolean
          description: >-
            True when any rule on the path is imprecise (the value may differ
            from ground truth).
      example:
        source_mapping_id: ea9dddd2-e3ee-40b5-b03e-c3cd37c8a6f0
        source_attribute_id: 291
        derivation_path:
          - 42
        lossy: true
        imprecise: false
    MappingTags:
      description: >-
        Free-form labels on the mapping. Null and an empty array both mean no
        tags; `nio:`-prefixed tags

        are reserved for Narrative.
      type:
        - array
        - 'null'
      items:
        type: string
      example:
        - nio:rosetta
    RetentionPolicy:
      type: object
      additionalProperties: false
      description: >
        One retention rule. `type` says what is deleted and by which clock;
        `enabled` and `interval` are siblings of it, not part of it.
      properties:
        type:
          oneOf:
            - $ref: '#/components/schemas/RowTTLType'
            - $ref: '#/components/schemas/TableTTLType'
            - $ref: '#/components/schemas/SnapshotTTLType'
          discriminator:
            propertyName: kind
            mapping:
              row_ttl: '#/components/schemas/RowTTLType'
              table_ttl: '#/components/schemas/TableTTLType'
              snapshot_ttl: '#/components/schemas/SnapshotTTLType'
        enabled:
          type: boolean
          default: false
        interval:
          type: string
          description: ISO-8601 duration, e.g., P30D, PT12H.
      required:
        - type
        - enabled
        - interval
    RefreshSchedule:
      type: object
      additionalProperties: false
      properties:
        schedule:
          type: string
          description: cron expression
        schedule_zone_id:
          type: string
          default: UTC
          description: Timezone for schedule execution
        status:
          enum:
            - active
            - pending
            - archived
          description: The status
    FileConfig:
      description: >-
        Defines the input file format. Files provided to Narrative for ingestion
        into a Dataset must match the file configuration.
      oneOf:
        - $ref: '#/components/schemas/FlatFileConfig'
        - $ref: '#/components/schemas/JsonFileConfig'
        - $ref: '#/components/schemas/ParquetFileConfig'
    PrimaryField:
      description: >-
        An optional RFC-6901 JSON pointer to a Dataset field to indicate that
        the field is "primary" -- i.e. it is of particular interest to buyers.
        Narrative uses this information to optimize buyer experience when
        purchasing your Dataset.

        The primary key, if specified, must refer to a _required_ field that has
        a type of `boolean`, `long`, `string`, or `timestamptz`.
      type: string
    PartitionKey:
      description: >-
        An optional RFC-6901 JSON pointer to the Dataset field the stored data
        is partitioned by.

        The partition key, if specified, must refer to a field that has a type
        of `boolean`, `long`, `string`, or `timestamptz`. The pointer may
        descend through nested object fields, e.g. `/event/timestamp`.
      type: string
    DatasetObjectType:
      type: object
      required:
        - type
        - properties
      properties:
        type:
          description: A record consisting of named fields of any type.
          enum:
            - object
        properties:
          type: object
          description: >-
            A map from field names to field types.


            Field names must consist of only alphanumeric characters or
            underscores and be at most 256 characters long.
          additionalProperties:
            $ref: '#/components/schemas/DatasetObjectProperty'
        required:
          description: >-
            An optional array of properties whose values are never missing or
            null.
          type: array
          items:
            type: string
        unsellable:
          description: An optional array of properties that cannot be sold to buyers.
          type: array
          items:
            type: string
        sensitive:
          description: >-
            An optional array of properties that are sensitive. Data in
            sensitive fields are redacted when showed to Narrative users via the
            UI or API but can still be bought and sold.
          type: array
          items:
            type: string
    EstTotalDatasetStoredFiles:
      description: >-
        The total estimated number of files across this dataset snapshot and all
        unexpired snapshots that precede it.
      type: integer
    EstTotalDatasetStoredRecords:
      description: >-
        The total estimated number of records across this dataset snapshot and
        all unexpired snapshots that precede it.
      type: integer
    EstTotalDatasetStoredBytes:
      description: >-
        The total estimated storage size in bytes used by this dataset snapshot
        and all unexpired snapshots that precede it.
      type: integer
    ActiveDatasetStoredFiles:
      description: The number of files in this dataset snapshot.
      type: integer
    ActiveDatasetStoredRecords:
      description: The number of records in this dataset snapshot.
      type: integer
    ActiveDatasetStoredBytes:
      description: The total storage size in bytes used by this dataset snapshot.
      type: integer
    SnapshotAddedFiles:
      description: The number of files added when generating this snapshot.
      type: integer
    SnapshotAddedRecords:
      description: The number of records added when generating this snapshot.
      type: integer
    SnapshotDeletedRecords:
      description: The number of records deleted when generating this snapshot.
      type: integer
    SnapshotAddedBytes:
      description: The amount of storage in bytes added when generating this snapshot.
      type: integer
    SnapshotDeletedFiles:
      description: The number of files deleted when generating this snapshot.
      type: integer
    SnapshotAddedDeleteFiles:
      description: The number of delete tracking files added when generating this snapshot.
      type: integer
    SnapshotRemovedDeleteFiles:
      description: >-
        The number of delete tracking files removed when generating this
        snapshot.
      type: integer
    SnapshotRemovedBytes:
      description: The amount of storage in bytes removed when generating this snapshot.
      type: integer
    Platform:
      oneOf:
        - $ref: '#/components/schemas/AwsPlatform'
        - $ref: '#/components/schemas/SnowflakePlatform'
      discriminator:
        propertyName: type
        mapping:
          platform_aws: '#/components/schemas/AwsPlatform'
          platform_snowflake: '#/components/schemas/SnowflakePlatform'
    DataPlaneCollaborators:
      type: object
      required:
        - participants
        - manage_compute_pools
      properties:
        participants:
          $ref: '#/components/schemas/DataPlaneParticipants'
        manage_compute_pools:
          $ref: '#/components/schemas/DataPlaneParticipants'
    ComputePoolResponse:
      oneOf:
        - $ref: '#/components/schemas/ComputePoolOwnedResponse'
        - $ref: '#/components/schemas/ComputePoolSharedResponse'
      discriminator:
        propertyName: type
        mapping:
          owned: '#/components/schemas/ComputePoolOwnedResponse'
          shared: '#/components/schemas/ComputePoolSharedResponse'
    PlatformSharedResponse:
      oneOf:
        - $ref: '#/components/schemas/AwsSharedResponse'
        - $ref: '#/components/schemas/SnowflakeSharedResponse'
      discriminator:
        propertyName: type
        mapping:
          platform_shared_aws: '#/components/schemas/AwsSharedResponse'
          platform_shared_snowflake: '#/components/schemas/SnowflakeSharedResponse'
    ObjectMapping:
      type: object
      required:
        - type
        - property_mappings
      properties:
        type:
          type: string
          description: A mapping from a dataset to an attribute with a type of `object`.
          enum:
            - object_mapping
        property_mappings:
          type: array
          description: >-
            `property_mappings` define how a value for each property of the
            target attribute is constructed from a row of the source dataset.


            A mapping must be provided for each of the target attribute's
            _required_ properties.
          items:
            type: object
            required:
              - path
              - expression
            properties:
              expression:
                $ref: '#/components/schemas/MappingExpression'
              path:
                $ref: '#/components/schemas/MappingPath'
              dependencies:
                $ref: '#/components/schemas/MappingExpressionDependencies'
    ValueMapping:
      type: object
      required:
        - type
        - expression
      properties:
        type:
          type: string
          description: >-
            A mapping from a dataset to an attribute with a primitive type, i.e.
            one of `array`, `boolean`, `double`, `long`, `string`, or
            `timestamptz`.
          enum:
            - value_mapping
        expression:
          $ref: '#/components/schemas/MappingExpression'
        dependencies:
          $ref: '#/components/schemas/MappingExpressionDependencies'
    CachedMapping:
      type: object
      description: >-
        Read shape for cached mappings. See `RawCachedMapping` for create/update
        requests and the surrounding constraints.
      required:
        - type
        - cache_dataset_id
        - input_expressions
      properties:
        type:
          type: string
          enum:
            - cached_mapping
        cache_dataset_id:
          type: integer
        input_expressions:
          type: array
          minItems: 1
          items:
            $ref: '#/components/schemas/MappingExpression'
        dependencies:
          $ref: '#/components/schemas/MappingExpressionDependencies'
    RowTTLType:
      type: object
      additionalProperties: false
      description: >
        Row-level deletion policy. Action is always HARD_DELETE. Clock must be
        EVENT_TIME. If `clock` is omitted, server uses configured Rosetta Stone
        attribute (defaults to nio_last_modified); if mapping doesn't exist,
        returns HTTP 400.
      properties:
        kind:
          const: row_ttl
        clock:
          $ref: '#/components/schemas/RowClock'
      required:
        - kind
    TableTTLType:
      type: object
      additionalProperties: false
      description: >
        Drop the dataset object when the table age exceeds the interval. Action
        is always DROP_TABLE.
      properties:
        kind:
          const: table_ttl
        clock:
          $ref: '#/components/schemas/TableClock'
      required:
        - kind
    SnapshotTTLType:
      type: object
      additionalProperties: false
      description: |
        Iceberg only: delete snapshots older than the interval.
      properties:
        kind:
          const: snapshot_ttl
        snapshot_retention_policy:
          oneOf:
            - $ref: '#/components/schemas/ExpireEverything'
            - $ref: '#/components/schemas/RetainEverything'
            - $ref: '#/components/schemas/ExpireWhen'
            - $ref: '#/components/schemas/RetainWhen'
            - $ref: '#/components/schemas/RetainSnapshots'
          discriminator:
            propertyName: type
            mapping:
              expire_everything: '#/components/schemas/ExpireEverything'
              retain_everything: '#/components/schemas/RetainEverything'
              expire_when: '#/components/schemas/ExpireWhen'
              retain_when: '#/components/schemas/RetainWhen'
              retain_snapshots: '#/components/schemas/RetainSnapshots'
      required:
        - kind
    FlatFileConfig:
      type: object
      required:
        - type
        - header
        - delimiter
        - quote
        - escape
      properties:
        type:
          type: string
          description: >-
            Describes input files where each row is a set of fields separated by
            some delimiter. Common examples include comma-separated (CSV) and
            tab-separated (TSV) files. Files can be compressed using gzip or
            bzip2 as long as they include the appropriate suffix to indicate
            what kind of compression has been applied. E.g. bzip2 files must end
            in ".bz2" and gzipped files must end in ".gz".
          enum:
            - flat
        header:
          type: boolean
          description: Whether or not files include a header.
        delimiter:
          type: string
          description: Character separating values within a row in a file.
        quote:
          type: string
          description: >-
            Character used for escaping quoted values where the separator can be
            part of the value.
        escape:
          type: string
          description: Character used for escaping quotes inside an already quoted value.
    JsonFileConfig:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: >-
            Describes input files in the [JSON Lines](https://jsonlines.org/)
            file format.
          enum:
            - json
    ParquetFileConfig:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: >-
            Describes input files in the [Parquet](https://parquet.apache.org/)
            file format.
          enum:
            - parquet
    DatasetObjectProperty:
      allOf:
        - type: object
          properties:
            order:
              type: integer
              description: >-
                The zero-based position of the field among its siblings.
                Responses always carry it. For a `flat` file

                type it is the column's position in the input file, so a request
                should set it rather than rely on the

                order of the keys in `properties`.
            display_name:
              type: string
              description: A name that will be represent at User Interface.
            approximate_cardinality:
              type: integer
              description: >-
                An optional estimate of how many distinct values of the field
                will be provided to Narrative. Narrative uses this information
                to optimizer buyer experience.
            description:
              type: string
              description: A description of the named field.
        - $ref: '#/components/schemas/DatasetType'
    AwsPlatform:
      type: object
      description: >-
        A data plane hosted in an AWS account. AWS data planes outside
        `us-east-1` currently only run (and cancel) `health_check` jobs, and
        don't support dataset creation yet: other such requests are rejected
        with a `400`.
      required:
        - type
        - account_id
        - region
      properties:
        type:
          type: string
          enum:
            - platform_aws
        account_id:
          type: string
          description: The 12-digit AWS account ID of the account hosting the data plane.
          pattern: ^[0-9]{12}$
        region:
          $ref: '#/components/schemas/AwsRegion'
    SnowflakePlatform:
      type: object
      required:
        - type
        - account_locator
        - account_name
        - organization_name
      properties:
        type:
          type: string
          enum:
            - platform_snowflake
        account_locator:
          type: string
        account_name:
          type: string
        organization_name:
          type: string
        region:
          oneOf:
            - $ref: '#/components/schemas/Region'
            - type: 'null'
    DataPlaneParticipants:
      oneOf:
        - $ref: '#/components/schemas/ParticipantsAll'
        - $ref: '#/components/schemas/ParticipantsNone'
        - $ref: '#/components/schemas/DataPlaneParticipantsInclusion'
        - $ref: '#/components/schemas/DataPlaneParticipantsExclusion'
      discriminator:
        propertyName: type
        mapping:
          all: '#/components/schemas/ParticipantsAll'
          none: '#/components/schemas/ParticipantsNone'
          inclusion: '#/components/schemas/DataPlaneParticipantsInclusion'
          exclusion: '#/components/schemas/DataPlaneParticipantsExclusion'
    ComputePoolOwnedResponse:
      type: object
      required:
        - type
        - id
        - company_id
        - data_plane_id
        - external_id
        - name
        - display_name
        - status
        - created_at
        - provider
        - collaborators
        - tags
        - default_source
      properties:
        type:
          type: string
          enum:
            - owned
        id:
          type: string
          format: uuid
        company_id:
          $ref: '#/components/schemas/CompanyId'
        data_plane_id:
          type: string
          format: uuid
        external_id:
          $ref: '#/components/schemas/ExternalId'
        name:
          type: string
        display_name:
          type: string
        description:
          type:
            - string
            - 'null'
        status:
          type: string
          enum:
            - active
            - archived
        created_at:
          type: string
          format: date-time
        updated_at:
          description: Null until the pool has been updated at least once.
          type:
            - string
            - 'null'
          format: date-time
        tags:
          type: array
          items:
            type: string
        provider:
          $ref: '#/components/schemas/Provider'
        collaborators:
          $ref: '#/components/schemas/ComputePoolCollaborators'
        default_source:
          description: >-
            Set when this pool is the compute pool the authenticated company's
            jobs run on when they don't name one themselves, and says which
            level made it the default. `null` on every other pool. Exactly one
            active pool per data plane can carry a non-null value.
          oneOf:
            - $ref: '#/components/schemas/ComputePoolDefaultSource'
            - type: 'null'
    ComputePoolSharedResponse:
      type: object
      required:
        - type
        - id
        - company_id
        - data_plane_id
        - external_id
        - name
        - display_name
        - status
        - created_at
        - provider
        - tags
        - default_source
      properties:
        type:
          type: string
          enum:
            - shared
        id:
          type: string
          format: uuid
        company_id:
          $ref: '#/components/schemas/CompanyId'
        data_plane_id:
          type: string
          format: uuid
        external_id:
          $ref: '#/components/schemas/ExternalId'
        name:
          type: string
        display_name:
          type: string
        description:
          type:
            - string
            - 'null'
        status:
          type: string
          enum:
            - active
            - archived
        created_at:
          type: string
          format: date-time
        updated_at:
          description: Null until the pool has been updated at least once.
          type:
            - string
            - 'null'
          format: date-time
        tags:
          type: array
          items:
            type: string
        provider:
          $ref: '#/components/schemas/Provider'
        default_source:
          description: >-
            Set when this pool is the compute pool the authenticated company's
            jobs run on when they don't name one themselves, and says which
            level made it the default. `null` on every other pool. Exactly one
            active pool per data plane can carry a non-null value.
          oneOf:
            - $ref: '#/components/schemas/ComputePoolDefaultSource'
            - type: 'null'
    AwsSharedResponse:
      type: object
      required:
        - type
        - region
      properties:
        type:
          type: string
          enum:
            - platform_shared_aws
        region:
          $ref: '#/components/schemas/AwsRegion'
    SnowflakeSharedResponse:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          enum:
            - platform_shared_snowflake
        region:
          oneOf:
            - $ref: '#/components/schemas/Region'
            - type: 'null'
    MappingExpression:
      type: string
      description: >-
        A raw SQL expression which produces a value from the source dataset for
        the target primitive attribute type.


        E.g. for a source dataset with the following type definition:


        ```

        {
          "type": "object",
          "properties": {
            "firstName": {
              "type": "string"
            },
            "lastName": {
              "type": "string"
            }
          },
          "required": [
            "lastName"
          ]
        }

        ```



        Valid expressions include:


        - Lower casing the first name:


        ```sql

        lower(firstName)

        ```


        - Constructing a "full name" from the first and last name:


        ```sql

        concat_ws(' ', coalesce(firstName, ''), lastName)

        ```


        - Taking the first initial of the first name if is not null:


        ```sql

        case when firstName is not null and char_length(firstName) > 0 then
        substr(firstName, 1) else null end

        ```
    MappingPath:
      type: string
      description: >-
        A string indicating which property of the target attribute the
        expression is producing a value for.


        E.g. for an attribute with the following type definition;


        ```

        {
          "type": "object",
          "properties": {
            "latitude": {
              "type": "double"
            },
            "longitude": {
              "type": "double"
            },
            "location_accuracy": {
              "type": "object",
              "properties": {
                "horizontal": {
                  "type": "double"
                },
                "vertical": {
                  "type": "double"
                }
              }
            }
          },
          "required": [
            "latitude",
            "longitude"
          ]
        }

        ```


        The possible paths are:


        - `latitude`

        - `longitude`

        - `location_accuracy.horizontal`

        - `location_accuracy.vertical`
    MappingExpressionDependencies:
      type: object
      description: Describes the dataset fields that a mapping expression depends on.
      properties:
        properties:
          type: array
          description: List of dataset field references that the mapping expression uses.
          items:
            type: string
          example:
            - firstName
            - lastName
      example:
        properties: []
    RowClock:
      type: object
      additionalProperties: false
      properties:
        kind:
          type: string
          enum:
            - event_time
        attribute:
          type: string
          description: Rosetta Stone attribute name for timestamp evaluation.
      required:
        - kind
    TableClock:
      type: object
      additionalProperties: false
      properties:
        kind:
          type: string
          enum:
            - created_at
            - max_event_time
        attribute:
          type: string
          description: Rosetta Stone attribute name evaluated when kind is max_event_time.
      required:
        - kind
    ExpireEverything:
      type: object
      properties:
        type:
          type: string
          enum:
            - expire_everything
      example:
        type: expire_everything
    RetainEverything:
      type: object
      properties:
        type:
          type: string
          enum:
            - retain_everything
      example:
        type: retain_everything
    ExpireWhen:
      type: object
      properties:
        type:
          type: string
          enum:
            - expire_when
        expression:
          oneOf:
            - $ref: '#/components/schemas/SnapshotAgeExpression'
          discriminator:
            propertyName: type
            mapping:
              snapshot_age: '#/components/schemas/SnapshotAgeExpression'
    RetainWhen:
      type: object
      properties:
        type:
          type: string
          enum:
            - retain_when
        expression:
          oneOf:
            - $ref: '#/components/schemas/SnapshotAgeExpression'
          discriminator:
            propertyName: type
            mapping:
              snapshot_age: '#/components/schemas/SnapshotAgeExpression'
    RetainSnapshots:
      type: object
      additionalProperties: false
      description: Keep a fixed number of the most recent snapshots.
      required:
        - type
        - snapshots_to_retain
      properties:
        type:
          type: string
          enum:
            - retain_snapshots
        snapshots_to_retain:
          type: integer
      example:
        type: retain_snapshots
        snapshots_to_retain: 5
    DatasetType:
      oneOf:
        - $ref: '#/components/schemas/ArrayType-2'
        - $ref: '#/components/schemas/DatasetBooleanType'
        - $ref: '#/components/schemas/DatasetDoubleType'
        - $ref: '#/components/schemas/DatasetLongType'
        - $ref: '#/components/schemas/DatasetObjectType'
        - $ref: '#/components/schemas/DatasetStringType'
        - $ref: '#/components/schemas/TimestampTzType-2'
    AwsRegion:
      type: object
      required:
        - type
        - id
      properties:
        type:
          type: string
          enum:
            - region_aws
        id:
          type: string
          description: AWS region identifier, e.g. `us-east-1`.
    Region:
      oneOf:
        - $ref: '#/components/schemas/AwsRegion'
        - $ref: '#/components/schemas/AzureRegion'
        - $ref: '#/components/schemas/GcpRegion'
      discriminator:
        propertyName: type
        mapping:
          region_aws: '#/components/schemas/AwsRegion'
          region_azure: '#/components/schemas/AzureRegion'
          region_gcp: '#/components/schemas/GcpRegion'
    ParticipantsAll:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          enum:
            - all
    ParticipantsNone:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          enum:
            - none
    DataPlaneParticipantsInclusion:
      type: object
      required:
        - type
        - company_ids
      properties:
        type:
          type: string
          enum:
            - inclusion
        company_ids:
          type: array
          minItems: 1
          items:
            $ref: '#/components/schemas/CompanyId'
    DataPlaneParticipantsExclusion:
      type: object
      required:
        - type
        - company_ids
      properties:
        type:
          type: string
          enum:
            - exclusion
        company_ids:
          type: array
          minItems: 1
          items:
            $ref: '#/components/schemas/CompanyId'
    ExternalId:
      oneOf:
        - $ref: '#/components/schemas/SnowflakeWarehouse'
        - $ref: '#/components/schemas/AwsEmrExternalId'
      discriminator:
        propertyName: type
        mapping:
          snowflake_warehouse: '#/components/schemas/SnowflakeWarehouse'
          aws_emr: '#/components/schemas/AwsEmrExternalId'
    Provider:
      oneOf:
        - $ref: '#/components/schemas/SnowflakeWarehouseProvider'
        - $ref: '#/components/schemas/AwsEmrProvider'
      discriminator:
        propertyName: type
        mapping:
          snowflake_warehouse: '#/components/schemas/SnowflakeWarehouseProvider'
          aws_emr: '#/components/schemas/AwsEmrProvider'
    ComputePoolCollaborators:
      type: object
      required:
        - use
      properties:
        use:
          $ref: '#/components/schemas/ComputePoolParticipants'
          description: >-
            Companies that can use this compute pool for jobs. Use participants
            automatically have view access.
    ComputePoolDefaultSource:
      description: >-
        Where the authenticated company's default compute pool for a data plane
        comes from. `company` is the company's own default for the data plane
        (`PUT
        /company/{company_id}/data-planes/{data_plane_id}/default-compute-pool/{compute_pool_id}`)
        and takes precedence; `data_plane` is the default set by the data
        plane's owner (`PUT
        /data-planes/{data_plane_id}/default-compute-pool/{compute_pool_id}`).
      type: string
      enum:
        - company
        - data_plane
    SnapshotAgeExpression:
      allOf:
        - $ref: '#/components/schemas/SnapshotAge'
        - type: object
          properties:
            type:
              type: string
              enum:
                - snapshot_age
          required:
            - type
      example:
        type: snapshot_age
        operator: '>'
        period: P30D
    ArrayType-2:
      type: object
      required:
        - type
        - items
      properties:
        type:
          type: string
          description: An array with elements of any data type.
          enum:
            - array
        items:
          $ref: '#/components/schemas/DatasetType'
        validations:
          $ref: '#/components/schemas/Validations-2'
    DatasetBooleanType:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: True/false values. Input values must be one of `true` or `false`.
          enum:
            - boolean
        validations:
          $ref: '#/components/schemas/Validations-2'
    DatasetDoubleType:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: 64-bit IEEE 754 floating point values.
          enum:
            - double
        enum:
          type: array
          description: An optional set of possible values for the field.
          items:
            type: number
            format: double
        validations:
          $ref: '#/components/schemas/Validations-2'
    DatasetLongType:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: 64-bit signed integer values.
          enum:
            - long
        enum:
          type: array
          description: An optional set of possible values for the field.
          items:
            type: integer
            format: int64
        validations:
          $ref: '#/components/schemas/Validations-2'
    DatasetStringType:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: Abitrary-length character sequences encoded with UTF-8.
          enum:
            - string
        enum:
          type: array
          description: An optional set of possible values for the field.
          items:
            type: string
        validations:
          $ref: '#/components/schemas/Validations-2'
    TimestampTzType-2:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          description: >-
            Timestamp with timezone, stored as microseconds. Input values are
            expected to be valid
            [ISO-8601](https://en.wikipedia.org/wiki/ISO_8601) strings.
          enum:
            - timestamptz
        validations:
          $ref: '#/components/schemas/Validations-2'
    AzureRegion:
      type: object
      required:
        - type
        - id
      properties:
        type:
          type: string
          enum:
            - region_azure
        id:
          type: string
          description: Azure region identifier, e.g. `uaenorth`.
    GcpRegion:
      type: object
      required:
        - type
        - id
      properties:
        type:
          type: string
          enum:
            - region_gcp
        id:
          type: string
          description: GCP region identifier, e.g. `us-east4`.
    SnowflakeWarehouse:
      type: object
      required:
        - type
        - name
        - alias
      properties:
        type:
          type: string
          enum:
            - snowflake_warehouse
        name:
          type: string
          description: >-
            Snowflake warehouse name. Must start with a letter or underscore and
            contain only alphanumeric characters, underscores, and dollar signs.
          pattern: ^[a-zA-Z_][a-zA-Z0-9_$]*$
          maxLength: 255
        alias:
          type: string
          format: uuid
    AwsEmrExternalId:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          enum:
            - aws_emr
    SnowflakeWarehouseProvider:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          enum:
            - snowflake_warehouse
        warehouse_name:
          type:
            - string
            - 'null'
          description: The name of the Snowflake warehouse to use.
        size:
          type:
            - string
            - 'null'
          enum:
            - x_small
            - small
            - medium
            - large
            - x_large
            - 2x_large
            - 3x_large
            - 4x_large
            - 5x_large
            - 6x_large
            - null
        warehouse_type:
          type:
            - string
            - 'null'
          enum:
            - standard
            - snowpark_optimized
            - null
        auto_suspend:
          type:
            - integer
            - 'null'
          description: Auto-suspend time in seconds.
        auto_resume:
          type:
            - boolean
            - 'null'
        min_cluster_count:
          type:
            - integer
            - 'null'
        max_cluster_count:
          type:
            - integer
            - 'null'
        scaling_policy:
          type:
            - string
            - 'null'
          enum:
            - standard
            - economy
            - null
        max_concurrency_level:
          type:
            - integer
            - 'null'
        statement_queued_timeout_in_seconds:
          type:
            - integer
            - 'null'
        statement_timeout_in_seconds:
          type:
            - integer
            - 'null'
        enable_query_acceleration:
          type:
            - boolean
            - 'null'
        query_acceleration_max_scale_factor:
          type:
            - integer
            - 'null'
        resource_constraint:
          type:
            - string
            - 'null'
        comment:
          type:
            - string
            - 'null'
    AwsEmrProvider:
      type: object
      required:
        - type
        - size
      properties:
        type:
          type: string
          enum:
            - aws_emr
        idle_timeout_seconds:
          type:
            - integer
            - 'null'
          minimum: -1
          maximum: 604800
          description: >
            EMR cluster auto-termination idle timeout, in seconds. Tri-state
            semantics (applied on both create and update — omitting this field
            on an update resets it to the default, it does not preserve the
            prior value):


            - omitted / `null`: the pool is set to the default of 15 minutes
            (900 seconds).

            - `-1`: idle-termination disabled — EMR's `AutoTerminationPolicy` is
            not set on the cluster
              and the operator-side idle reaper skips the idle check too. The cluster runs until it is
              explicitly terminated or recycled.

            - `60` ≤ n ≤ `604800` (7 days): set EMR's `AutoTerminationPolicy` to
            that value, and use the
              same value in the operator-side idle reaper.
        job_execution_timeout_seconds:
          type:
            - integer
            - 'null'
          minimum: 60
          maximum: 604800
          description: >
            Maximum time, in seconds, that a single job's EMR step may stay in
            `RUNNING` before the operator cancels the step and fails the job.
            Omitted / `null` means the pool is set to the default of 4 hours
            (14400 seconds), on both create and update (omitting it on an update
            resets it to the default rather than preserving the prior value).
            When set explicitly, must be between 60 seconds and 7 days.


            The shared Narrative compute pool is configured with a 1-hour
            timeout and is intended for small jobs. Pools the user provisions
            for themselves can set their own (longer) limit.
        size:
          type: string
          enum:
            - x_small
            - small
            - medium
            - large
            - x_large
            - 2x_large
            - 3x_large
            - 4x_large
            - 5x_large
            - 6x_large
            - x_small_storage
            - small_storage
            - medium_storage
            - large_storage
            - x_large_storage
            - 2x_large_storage
            - 3x_large_storage
            - 4x_large_storage
            - 5x_large_storage
            - 6x_large_storage
          description: >
            Compute pool size. Names mirror Snowflake's warehouse sizes and
            target the equivalent Snowflake warehouse's *memory* budget — a
            workload that fits in a Snowflake warehouse of size N gets
            equivalent RAM on the EMR cluster of size N. vCPU counts will not
            match Snowflake's because EMR runs on memory-optimized instances
            (r7g/r7gd/r6g, 8 GiB/vCPU), so the same memory budget delivers fewer
            vCPUs than Snowflake.


            Target worker (core+task) memory at max scale: x_small (~32 GiB, ~4
            vCPU; min size — overshoots Snowflake's 16 GiB), small (~32 GiB, ~4
            vCPU), medium (~64 GiB, ~8 vCPU), large (~128 GiB, ~16 vCPU),
            x_large (~256 GiB, ~32 vCPU), 2x_large (~512 GiB, ~64 vCPU),
            3x_large (~1 TiB, ~128 vCPU), 4x_large (~2 TiB, ~256 vCPU), 5x_large
            (~4 TiB, ~512 vCPU), 6x_large (~8 TiB, ~1024 vCPU).


            Each size has a `*_storage` variant (e.g. `medium_storage`) with the
            same memory budget, but running on `rXgd` instances with local NVMe
            SSD storage. Use these for shuffle-/scratch-heavy jobs that would
            otherwise spill to EBS and fail with "No space left on device".
    ComputePoolParticipants:
      oneOf:
        - $ref: '#/components/schemas/ParticipantsAll'
        - $ref: '#/components/schemas/ParticipantsNone'
        - $ref: '#/components/schemas/ParticipantsInclusion'
        - $ref: '#/components/schemas/ParticipantsExclusion'
      discriminator:
        propertyName: type
        mapping:
          all: '#/components/schemas/ParticipantsAll'
          none: '#/components/schemas/ParticipantsNone'
          inclusion: '#/components/schemas/ParticipantsInclusion'
          exclusion: '#/components/schemas/ParticipantsExclusion'
    SnapshotAge:
      type: object
      properties:
        operator:
          type: string
          enum:
            - '>'
            - <
            - '>='
            - <=
        period:
          type: string
          description: An ISO 8601 period format
      required:
        - operator
        - period
      example:
        operator: '>'
        period: P30D
    Validations-2:
      type: array
      description: >-
        A raw Spark SQL expression determining whether a value for the given
        column is valid or not.


        If more than one validation is specified then _all_ the validation must
        evaluate to `true` for the column value to be considered valid.


        If `validations` is not specified or empty then all rows will match the
        access rule.


        E.g. for a dataset with the following type definition:


        ```

        {
          "type": "object",
          "properties": {
            "idValue": {
              "type": "string"
            },
            "idType": {
              "type": "string",
              "enum": [
                "idfa",
                "adid"
              ]
            },
            "timestamp": {
              "type": "timestamptz"
            }
          },
          "required": [
            "idValue"
          ]
        }

        ```


        Valid constraints include:


        - Is the ID type IDFA?


        ```sql

        idType = 'idfa'

        ```

        - Is the ID type non-null?


        ```sql

        idType is not null

        ```

        - Was the row collected within the last 90 days?


        ```sql

        timestamp > date_sub(current_date(), 90)

        ```
      items:
        type: string
    ParticipantsInclusion:
      type: object
      required:
        - type
        - company_ids
      properties:
        type:
          type: string
          enum:
            - inclusion
        company_ids:
          type: array
          minItems: 1
          items:
            $ref: '#/components/schemas/CompanyId'
    ParticipantsExclusion:
      type: object
      required:
        - type
        - company_ids
      properties:
        type:
          type: string
          enum:
            - exclusion
        company_ids:
          type: array
          minItems: 1
          items:
            $ref: '#/components/schemas/CompanyId'
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.