Structured Output - Narrative I/O Knowledge Base

Structured output is a core capability of Model Inference that guarantees the model returns data in a predictable, machine-readable format. By providing a JSON Schema, you constrain the model’s response to match your exact specification.

The problem with unstructured output

Traditional LLM interactions return free-form text:

User: Classify this review as positive, negative, or neutral.
Model: Based on my analysis, I believe this review expresses a positive sentiment. The customer seems satisfied with their purchase.

This creates challenges:

Parsing complexity: Must extract the actual classification from prose
Inconsistent formats: Response structure varies between calls
Error-prone: Regex or string matching can fail on edge cases
No type safety: Can’t validate response structure at compile time

How structured output works

With structured output, you define the exact response format:

const schema = {
  type: 'object',
  properties: {
    sentiment: { type: 'string', enum: ['positive', 'negative', 'neutral'] },
    confidence: { type: 'number', minimum: 0, maximum: 1 }
  },
  required: ['sentiment', 'confidence']
};

The model returns exactly what you specify:

{
  "sentiment": "positive",
  "confidence": 0.92
}

Why structured output matters

1. Reliable automation

Structured output enables reliable automation pipelines:

// Process results immediately without parsing
const result = job.result.structured_output;
await database.insert({
  category: result.category,
  confidence: result.confidence,
  tags: result.tags
});

2. Type safety

With TypeScript, you get compile-time type checking:

interface ClassificationResult {
  category: 'retail' | 'finance' | 'healthcare';
  confidence: number;
}

const result = job.result as ModelInferenceRunResult<ClassificationResult>;
// TypeScript knows result.structured_output.category is a string
// and result.structured_output.confidence is a number

3. Schema validation

The model is constrained to produce valid JSON matching your schema:

Required fields are always present
Types match your specification
Enum values are restricted to defined options
Numeric constraints are enforced

4. Consistent integration

Every response has the same structure, making integration predictable:

Without Schema	With Schema
Parse response text	Direct property access
Handle format variations	Consistent structure
Runtime validation needed	Schema-enforced validity
Type casting required	Native types

How structured output is enforced

Not every model enforces output_format_schema the same way. The platform picks a strategy based on the model’s capabilities, and the response is validated against the schema either way.

Path	Used by	How the schema is enforced
Native structured output	Claude Haiku 4.5, Sonnet 4.5 / 4.6, Opus 4.5 / 4.6, all OpenAI models	The model provider enforces the schema at generation time (grammar-constrained decoding).
Forced-tool-use	Claude Sonnet 5.0, Claude Opus 4.7 / 4.8	The schema is passed as a reserved final-answer tool the model must call. The platform then validates the tool arguments against your schema on the way out.

Both paths return the same structured_output shape, so client code does not need to branch on the model.

Client-side schema validation

After the model responds, the platform validates the payload against your output_format_schema (JSON Schema draft 2019-09). The inference job completes either way — validation just decides which field on the result is populated:

Validation passes. structured_output holds the parsed object matching your schema; failed_structured_output is null.
Validation fails. structured_output is null and failed_structured_output carries the raw JSON the model emitted along with the validator’s message explaining why it was rejected. The job does not fail, so downstream steps still run — check failed_structured_output before consuming structured_output.

Both fields are mutually exclusive: at most one is populated on any completed job.

Requirements for forced-tool-use models

When you target a forced-tool-use model (Sonnet 5.0 or Opus 4.7 / 4.8), a few extra constraints apply:

Top-level additionalProperties: false is required. The forced-tool path rejects a top-level object schema that omits it (or sets it to true) because the reserved final-answer tool needs a closed shape. See the JSON Schema Reference.
temperature and top_p are silently dropped. These models reject sampling knobs; the platform strips them from your request rather than failing the job.
Strict tool-use caps do not apply. Forced-tool-use models bypass the strict tool use limit, but the reserved tool name nio_emit_structured_output is off-limits — a caller-supplied tool cannot use that name.

JSON Schema capabilities

Model Inference supports standard JSON Schema features:

Basic types

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "count": { "type": "integer" },
    "score": { "type": "number" },
    "active": { "type": "boolean" }
  }
}

Constraints

{
  "type": "object",
  "properties": {
    "rating": {
      "type": "integer",
      "minimum": 1,
      "maximum": 5
    },
    "category": {
      "type": "string",
      "enum": ["A", "B", "C"]
    }
  }
}

Nested structures

{
  "type": "object",
  "properties": {
    "analysis": {
      "type": "object",
      "properties": {
        "summary": { "type": "string" },
        "details": { "type": "string" }
      }
    },
    "tags": {
      "type": "array",
      "items": { "type": "string" }
    }
  }
}

Design principles

1. Define what you need

Only include fields your application will use:

// Good: Focused schema
const schema = {
  type: 'object',
  properties: {
    category: { type: 'string', enum: ['spam', 'not_spam'] },
    confidence: { type: 'number' }
  },
  required: ['category', 'confidence']
};

// Avoid: Overly broad schema
const schema = {
  type: 'object',
  properties: {
    category: { type: 'string' },
    confidence: { type: 'number' },
    reasoning: { type: 'string' },
    alternatives: { type: 'array' },
    metadata: { type: 'object' }
    // ... many unused fields
  }
};

2. Use enums for known values

Constrain categorical outputs to valid options:

const schema = {
  type: 'object',
  properties: {
    priority: {
      type: 'string',
      enum: ['low', 'medium', 'high', 'critical']
    }
  }
};

3. Set appropriate bounds

Define numeric ranges when applicable:

const schema = {
  type: 'object',
  properties: {
    confidence: {
      type: 'number',
      minimum: 0,
      maximum: 1,
      description: 'Confidence score from 0 (uncertain) to 1 (certain)'
    }
  }
};

4. Add descriptions

Help the model understand field semantics:

const schema = {
  type: 'object',
  properties: {
    sentiment: {
      type: 'string',
      enum: ['positive', 'negative', 'neutral'],
      description: 'Overall emotional tone: positive for satisfaction, negative for complaints, neutral for factual'
    }
  }
};

Common patterns

Classification

{
  "type": "object",
  "properties": {
    "label": { "type": "string", "enum": ["class_a", "class_b", "class_c"] },
    "confidence": { "type": "number", "minimum": 0, "maximum": 1 },
    "reasoning": { "type": "string" }
  },
  "required": ["label", "confidence"]
}

Extraction

{
  "type": "object",
  "properties": {
    "entities": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "value": { "type": "string" },
          "type": { "type": "string" }
        },
        "required": ["value", "type"]
      }
    }
  },
  "required": ["entities"]
}

Transformation

{
  "type": "object",
  "properties": {
    "input_format": { "type": "string" },
    "output_format": { "type": "string" },
    "transformed_value": { "type": "string" }
  },
  "required": ["transformed_value"]
}

Using Structured Output

Practical guide to schema definition

JSON Schema Reference

Supported schema features

Model Inference Overview

How inference works

Running Inference

Submit inference requests

​The problem with unstructured output

​How structured output works

​Why structured output matters

​1. Reliable automation

​2. Type safety

​3. Schema validation

​4. Consistent integration

​How structured output is enforced

​Client-side schema validation

​Requirements for forced-tool-use models

​JSON Schema capabilities

​Basic types

​Constraints

​Nested structures

​Design principles

​1. Define what you need

​2. Use enums for known values

​3. Set appropriate bounds

​4. Add descriptions

​Common patterns

​Classification

​Extraction

​Transformation

​Related content