Workflow Specification Syntax

Workflows are defined using the Serverless Workflow DSL in YAML format. This reference covers the specification structure. For the full list of available tasks, see the Task Reference.

Specification structure

document:
  dsl: '1.0.0'
  namespace: <namespace>
  name: <workflow-name>
  version: '<version>'
schedule:                          # optional
  cron: '<cron-expression>'
do:
  - <taskName>:
      call: <TaskName>
      with:
        <param>: <value>
      export:                      # optional
        as: '<jq-expression>'

Document fields

The document block defines workflow metadata.

Field	Type	Required	Description
`dsl`	string	Yes	DSL version. Use `'1.0.0'`
`namespace`	string	Yes	Logical grouping for the workflow (e.g., `analytics`, `etl`)
`name`	string	Yes	Unique identifier for the workflow within its namespace
`version`	string	Yes	Version string for tracking changes (e.g., `'1.0.0'`)

Example:

document:
  dsl: '1.0.0'
  namespace: etl
  name: daily-refresh-pipeline
  version: '2.1.0'

Schedule

The optional schedule block configures automatic execution.

Field	Type	Required	Description
`cron`	string	Yes (within `schedule`)	CRON expression defining when the workflow runs

CRON format:

┌───────────── minute (0-59)
│ ┌───────────── hour (0-23)
│ │ ┌───────────── day of month (1-31)
│ │ │ ┌───────────── month (1-12)
│ │ │ │ ┌───────────── day of week (0-6, Sunday=0)
│ │ │ │ │
* * * * *

Common expressions:

Expression	Schedule
`0 * * * *`	Every hour
`0 0 * * *`	Daily at midnight UTC
`0 0 * * 0`	Weekly on Sunday at midnight UTC
`0 0 1 * *`	Monthly on the 1st at midnight UTC
`15 15 * * 1`	Every Monday at 3:15 PM UTC
`0 2 1 * *`	1st of each month at 2:00 AM UTC

Example:

schedule:
  cron: '0 0 * * *'

Schedules are activated separately from workflow creation. Use the Schedule a workflow endpoint or set schedule_immediately: true when creating the workflow.

Task structure

The do block contains an ordered list of tasks. Each task is a YAML mapping with a unique name as the key.

Field	Type	Required	Description
`call`	string	Yes	The task to execute. Must be one of the supported tasks
`with`	object	Yes	Parameters for the task. Fields vary by task
`export`	object	No	Controls how task output is merged into the workflow context. See Export

Tasks execute sequentially in the order they appear. Each task waits for the previous one to complete before starting. Example:

do:
  - firstTask:
      call: CreateMaterializedViewIfNotExists
      with:
        nql: 'CREATE MATERIALIZED VIEW user_list AS SELECT id, name FROM users'
  - secondTask:
      call: RefreshMaterializedView
      with:
        datasetName: user_list

Task output

Every task produces a JSON object after execution. The output fields use camelCase names and vary by task — see the Task Reference for per-task output schemas. Within an export.as expression, . (dot) refers to the current task’s output. In ${…} variable expressions in the next task’s parameters, . also refers to the immediately preceding task’s output.

Export

The optional export block controls how a task’s output is merged into the workflow context — an object ($context) that accumulates data as the workflow runs. The context starts as an empty object {} and is threaded through every task.

Field	Type	Description
`export.as`	string (jq expression)	A jq expression that produces the new value of `$context`

Available variables in export.as:

Variable	Description
`.`	The current task’s output
`$context`	The accumulated workflow context before this task

When export is omitted, $context is unchanged. Examples: Store the entire task output as context:

export:
  as: '.'

Merge a single field into the existing context:

export:
  as: '$context + { datasetId: .datasetId }'

Build a running list of dataset IDs:

export:
  as: '$context + { datasetIds: (($context.datasetIds // []) + [.datasetId]) }'

Variable expressions

Task parameters support ${…} variable expressions that inject values from previous task output or the workflow context. Variable expressions are evaluated before the task executes. Syntax rules:

Pattern	Behavior	Example
`${<expr>}` as the entire value	Preserves the JSON type of the expression result	`${.datasetId}` → integer `42`
`${" ... "}` or mixed with text	Result is always a string	`${"Dataset \(.datasetId) created"}` → `"Dataset 42 created"`

Available variables in ${…}:

Variable	Description
`.`	The immediately preceding task’s output
`$context`	The accumulated workflow context

Use jq string interpolation \(expr) to embed values inside a string expression:

nql: |
  ${"INSERT INTO company_data.log (id) VALUES (\(.datasetId))"}

Supported tasks

The following tasks are available in workflows. See the Task Reference for full parameter details, constraints, and examples.

Task	Description
`CreateMaterializedViewIfNotExists`	Task that creates a materialized view if it does not already exist.
`RefreshMaterializedView`	Task that triggers a refresh of an existing materialized view.
`ExecuteDml`	Task that executes a DML statement on a dataset.
`RunModelInference`	Task that runs a model inference job.
`LabelConnectedComponents`	Task that runs bipartite label propagation for cross-system customer identity resolution. Finds connected components in a customer identity graph by linking customer IDs across platforms via shared identifiers.
`CreateRosettaStoneMappingsIfNotExist`	Task that creates Rosetta Stone attribute mappings for a dataset.
`CreateDatasetSample`	Task that generates a sample for a dataset.
`RecalculateStatistics`	Task that triggers a recalculation of a dataset’s column statistics and waits for it to complete.

Complete example

A multi-step workflow that creates two materialized views and refreshes one, running daily:

document:
  dsl: '1.0.0'
  namespace: analytics
  name: daily-user-pipeline
  version: '1.0.0'
schedule:
  cron: '0 0 * * *'
do:
  - createActiveUsers:
      call: CreateMaterializedViewIfNotExists
      with:
        nql: |
          CREATE MATERIALIZED VIEW active_users AS
          SELECT
            id,
            name,
            email
          FROM users
          WHERE active = true
  - createUserMetrics:
      call: CreateMaterializedViewIfNotExists
      with:
        nql: |
          CREATE MATERIALIZED VIEW active_users_with_names AS
          SELECT
            id,
            name
          FROM company_data.active_users
          WHERE name IS NOT NULL
  - refreshMetrics:
      call: RefreshMaterializedView
      with:
        datasetName: daily_kpis

Example with data passing

A workflow that creates a view, captures its dataset ID, and logs the result:

document:
  dsl: '1.0.0'
  namespace: etl
  name: create-and-log
  version: '1.0.0'
do:
  - createView:
      call: CreateMaterializedViewIfNotExists
      with:
        nql: "CREATE MATERIALIZED VIEW active_users AS SELECT user_id, email FROM company_data.users WHERE is_active = true"
      export:
        as: '$context + { datasetId: .datasetId }'
  - logCreatedDataset:
      call: ExecuteDml
      with:
        nql: |
          ${"INSERT INTO company_data.pipeline_log (dataset_id, status) VALUES (\(.datasetId), 'created')"}
  - logFromContext:
      call: ExecuteDml
      with:
        nql: |
          ${"INSERT INTO company_data.pipeline_log (dataset_id, status) VALUES (\($context.datasetId), 'created')"}

In this example:

createView creates a materialized view and exports datasetId into $context.datasetId
logCreatedDataset uses .datasetId (the previous task’s direct output) in a variable expression
logFromContext uses $context.datasetId to access the same value via the workflow context

Task Reference

Full reference for all supported tasks

Automating Multi-Step Pipelines

Step-by-step guide to creating and running workflows

Workflow Orchestration

How workflows work and why they’re designed this way

Workflows API

REST API endpoints for managing workflows

Overview

Connectors

Integrations

Workflow Reference

Webhook Reference

Security Reference

Architecture Reference

Error Reference

Model Inference Reference

Rosetta Stone Reference

UI Reference

SDKs

Glossary

Billing

Specification structure

Document fields

Schedule

Task structure

Task output

Export

Variable expressions

Supported tasks

Complete example

Example with data passing

Task Reference

Automating Multi-Step Pipelines

Workflow Orchestration

Workflows API

​Specification structure

​Document fields

​Schedule

​Task structure

​Task output

​Export

​Variable expressions

​Supported tasks

​Complete example

​Example with data passing

​Related content

Task Reference

Automating Multi-Step Pipelines

Workflow Orchestration

Workflows API

Specification structure

Document fields

Schedule

Task structure

Task output

Export

Variable expressions

Supported tasks

Complete example

Example with data passing

Related content