The privacy challenge with external AI
When using external AI APIs (OpenAI API, Anthropic API, etc.), your data flows to third-party servers: This creates several concerns:- Data residency: Data leaves your infrastructure and jurisdiction
- Third-party access: Provider can potentially access your data
- Compliance complexity: Must evaluate provider’s data handling practices
- Retention policies: Provider may retain data for improvement or debugging
How Model Inference protects your data
Model Inference eliminates these concerns by hosting models within your data plane:Key privacy guarantees
| Aspect | Guarantee |
|---|---|
| Data location | Data never leaves your data plane |
| No external calls | No network requests to Anthropic, OpenAI, or other providers |
| Your infrastructure | Models run on compute within your environment |
| No provider logging | AI providers have no access to your data |
| Full control | You control data access, retention, and deletion |
Compliance implications
Model Inference simplifies compliance with data protection regulations:GDPR considerations
- Data minimization: Only necessary data is processed
- Storage limitation: You control retention within your data plane
- Data transfers: No cross-border transfers to AI providers
- Processor agreements: No need for DPAs with AI providers for inference
CCPA considerations
- Service provider status: AI providers are not service providers for your data
- Sale of data: No data is shared with third parties
- Right to delete: Full control over data deletion
Industry regulations
For industries with strict data handling requirements (healthcare, finance, government), Model Inference enables AI capabilities without the compliance burden of external AI services:- HIPAA: PHI never leaves your controlled environment
- PCI DSS: Payment data stays within your secure perimeter
- FedRAMP: Data remains in authorized boundaries
Audit trail
All inference jobs are tracked through Narrative’s standard job system:- Job creation timestamp
- Data plane where inference ran
- Model used
- Token usage metrics
- Job completion status
What the control plane sees
The control plane only handles:- Job routing and coordination
- Metadata about requests (model choice, configuration)
- Job status updates
- Token usage statistics
- Your prompt content
- Your data
- The model’s responses
Comparison with external AI
| Aspect | External AI APIs | Model Inference |
|---|---|---|
| Data location | Provider’s servers | Your data plane |
| Network traffic | Data sent externally | Local only |
| Provider data access | Yes (per their policies) | No |
| Compliance burden | High (must evaluate provider) | Low (your infrastructure) |
| Audit complexity | Must rely on provider logs | Full control |
| Data retention | Provider-controlled | You control |
Best practices
- Use appropriate models: Don’t send more context than necessary
- Review prompts: Ensure prompts don’t unnecessarily include sensitive data
- Monitor usage: Track inference jobs through the job system
- Set retention policies: Configure data plane retention appropriately