Compute pool types
Dedicated
Dedicated compute pools provide isolated resources reserved for your workloads. Your queries don’t compete with other users for processing power, which results in more predictable performance. Use dedicated compute pools when:- Running production workloads where performance consistency matters
- Processing large or complex queries that need guaranteed resources
- Operating time-sensitive pipelines where latency must stay predictable
Shared
Shared compute pools use pooled resources across multiple users. This is more cost-effective but means your query performance may vary depending on current platform load. Use shared compute pools when:- Running exploratory queries or ad-hoc analysis
- Developing and testing queries before promoting to production
- Working with smaller datasets where performance variability is acceptable
Snowflake warehouse
On Snowflake-based data planes, each compute pool maps to a Snowflake virtual warehouse. When you register warehouses through the Snowflake Native App, each warehouse becomes a compute pool on your data plane. You can register multiple warehouses to separate workloads—for example, a smaller warehouse for exploratory queries and a larger one for production pipelines. Each Snowflake compute pool has a collaboration policy that controls which companies can use it, and one pool can be designated as the default for the data plane.AWS EMR
On AWS-based data planes, compute pools map to Amazon EMR Spark clusters that the data plane operator provisions, reuses across jobs targeting the same pool, and terminates when idle. Each pool has a configured size that determines the cluster’s worker memory budget and vCPU count. The operator schedules NQL jobs—includingmaterialize-view, nql-forecast, datasets_sample, and datasets_execute_dml—onto the appropriate cluster based on the compute pool selected for the workload.
Sizes mirror Snowflake’s warehouse vocabulary (x_small through 6x_large) and target the same worker memory budget as the equivalent Snowflake warehouse, so a workload that fits in a Snowflake warehouse of size N gets equivalent RAM on the EMR cluster of size N. vCPU counts won’t match Snowflake because EMR uses memory-optimized instances (8 GiB per vCPU), so the same memory budget delivers fewer vCPUs.
Sizes
x_large and above use EMR Managed Scaling: the cluster boots small and expands toward the maximum based on YARN load, then contracts back down when idle. Sizes large and below run at a fixed instance count.
Instance-storage (_storage) variants
Every size has a sibling _storage variant — x_small_storage, small_storage, medium_storage, large_storage, x_large_storage, 2x_large_storage, 3x_large_storage, 4x_large_storage, 5x_large_storage, 6x_large_storage — with the same worker memory budget and node counts as its base size, but with core and task nodes running on Graviton instances with local NVMe SSD (r7gd/r8gd) instead of EBS-backed families. The master node stays on m6g (it does no shuffle).
Use a _storage variant when a job spills a large volume of shuffle or scratch data to disk and fails on an EBS-backed pool with No space left on device. Typical culprits are wide joins, large GROUP BY / DISTINCT aggregations, and materialized-view refreshes over very wide datasets. For jobs that are CPU- or memory-bound rather than shuffle-bound, stick with the base size — the local-NVMe families cost more per node.
You switch between the base and _storage tiers the same way you change any other size: update the pool’s provider.size via PATCH /data-planes/{id}/compute-pools/{poolId}, or create a new pool sized for the workload and pin it on the affected job or dataset.
Idle and job execution timeouts
EMR-backed pools expose two optional tunables that control cluster lifetime and per-job runtime. Both are set on theaws_emr provider when you create or update a pool, and both are validated server-side — invalid values return HTTP 400.
When either field is omitted (or set to
null) on create or update, the API fills in the default above before validating and persisting the pool. Explicit values — including the -1 disabled-idle sentinel — are always preserved. This matters when you update the pool’s size (for example, switching to a _storage variant): the provider block is replaced wholesale rather than merged, so a size-only update that omits the timeout fields still gets the standard 4h/15m defaults instead of silently dropping them. Set idle_timeout_seconds to -1 when you want a long-lived cluster — for example, a pool dedicated to back-to-back batch jobs where boot latency dominates run time.
The shared Narrative compute pool is configured with a 1-hour job execution timeout and is intended for small jobs. Pools you provision for yourself get the 4-hour default on create, and can set their own (longer or shorter) limit.
Which compute pools are available
The compute pool options available to you depend on your data plane’s underlying provider:
You select your compute pool through the context selector in the platform’s top navigation.
Default compute pool resolution
When a job is created without an explicit compute pool, the platform resolves one through a four-level fallback chain. Resolution happens at job-creation time, so a job’scompute_pool_id is fixed before it lands in Pending. The first level present wins.
A company admin can set the level-3 default for a given data plane with:
204 No Content on success.
When to use each type
Archiving compute pools
Deleting a compute pool is a soft delete: the pool’s status changes toarchived, the record stays in place, and it is filtered out of list responses but still resolvable by id. Archival behavior diverges by provider because each integration has different operational realities:
Snowflake users frequently mutate their warehouse list outside the platform — through the Snowflake UI, permissions, or infrastructure as code. When a backing warehouse disappears, the platform has to track reality, so the archive must succeed and dependent references are updated automatically. AWS compute pools are wholly managed inside the platform, so the API rejects an archive that would leave a dangling default and forces an explicit decision.
Runtime impact on in-flight jobs
Neither archive flow blocks in-flight jobs at archive time. Instead, the data plane operator fails any job whose compute pool resolves to a non-active status on its next polling iteration. The job reports as failed with an actionable message similar to:
Pending, Running, and PendingCancellation jobs. Jobs in PendingCancellation are failed rather than cancelled because the missing pool is the actionable signal — a successful cancellation report would mask the root cause.
See the Archive a compute pool guide for the API workflow and the order in which to clear references.
On AWS data planes, the first compute pool you create is not automatically promoted to the data plane default. This is the permanent behavior: jobs targeting an AWS data plane must either pin an explicit
computePoolId, resolve to a dataset-level default, or have a data plane default set explicitly via PUT /data-planes/{id}/default-compute-pool/{poolId}. The intent is that AWS workloads make a deliberate choice about where they run rather than implicitly routing through a “first pool wins” default.On Snowflake data planes, the first compute pool you create is currently auto-promoted to the data plane default for backward compatibility while the Snowflake-side migration off implicit defaults is in flight. This behavior is temporary and will be removed once every Snowflake workload pins its compute pool explicitly.Every newly registered company is provisioned a private, x-small AWS EMR compute pool on the Narrative data plane and that pool is set as the company-level default for the same data plane. Jobs that don’t pin a pool explicitly resolve to this default through the pool resolution chain (job → dataset → company → data plane). You can rename, archive, or replace the default at any time.
external_id is optional for aws_emr providers
When creating a compute pool with the aws_emr provider via POST /data-planes/{id}/compute-pools, you can omit external_id from the request body — the server fills in the trivial {"type": "aws_emr"} payload automatically. Supplying it explicitly still works, and the type is validated against the provider.
For the snowflake_warehouse provider, external_id is still required because it carries the warehouse name and alias that the platform uses to dispatch queries to the correct Snowflake warehouse.
How compute pools relate to the SDK
When executing queries through the TypeScript SDK, theexecution_cluster parameter maps to the compute pool concept:
execution_cluster.type accepts 'dedicated' or 'shared', corresponding directly to the Dedicated and Shared compute pool types. If omitted, the data plane’s default compute pool is used.
For Snowflake-based data planes, omitting execution_cluster uses the data plane’s default compute pool (the warehouse you’ve designated as default).
Related content
Execution Context
How data plane, compute pool, database, and schema work together
Data Planes
Where your data lives and is processed
Executing NQL Queries
Run queries programmatically with the TypeScript SDK
Migrate to Compute Pools
Transition from a single Snowflake warehouse to compute pools
Archive a Compute Pool
Safely retire a compute pool and clear its references