Skip to main content
Developer docs

ModelSpend SDK

Use the ModelSpend SDK to route AI requests through the OpenAI-compatible gateway, capture routing telemetry and attach FinOps attribution. Available for TypeScript/JavaScript and Python.

TypeScript / JavaScript

Install

npm install @modelspend/sdk

The npm package is in the monorepo as @modelspend/sdk and must not be published until npm package ownership and release credentials are approved. For local workspace testing:

npm install ../packages/sdk-node

Quickstart

import { ModelSpend } from '@modelspend/sdk';

const client = new ModelSpend({
  apiKey: process.env.MODELSPEND_API_KEY,
});

const response = await client.chat.completions.create({
  model: 'auto',
  messages: [{ role: 'user', content: 'Summarise this support ticket.' }],
});

console.log(response.choices[0]?.message.content);
console.log(response.modelspend?.routedTo);
console.log(response.modelspend?.costUsd);

Routing controls

Use model: 'auto' for default routing, or pass routing preferences for cost, confidence and quality trade-offs.

await client.chat.completions.create({
  model: 'auto',
  routing: {
    prefer: 'lowest_cost_that_meets_quality',
    maxCostUsd: 0.02,
    minConfidence: 0.8,
  },
  messages: [{ role: 'user', content: 'Analyse this refund request.' }],
});

FinOps attribution

Attach customer, feature, environment and cost-centre metadata so dashboard reporting, chargeback and unit economics can group spend correctly.

await client.chat.completions.create({
  model: 'auto',
  messages: [{ role: 'user', content: 'Classify this ticket.' }],
  metadata: {
    customerId: 'acme',
    featureId: 'support-triage',
    environment: 'production',
    costCenter: 'customer-success',
  },
});

Error handling

API errors are surfaced as ModelSpendError and preserve the API error code, details and request ID where available.

import { ModelSpendError } from '@modelspend/sdk';

try {
  await client.chat.completions.create({ model: 'auto', messages });
} catch (error) {
  if (error instanceof ModelSpendError) {
    console.error(error.code, error.details, error.requestId);
  }
  throw error;
}

Python

The Python SDK is a drop-in replacement for openai.OpenAI that routes requests through the ModelSpend proxy. It requires the openai package.

Install

python -m pip install openai
# modelspend package: install from the monorepo until PyPI publication is approved
python -m pip install /path/to/packages/sdk-python

Quickstart

from modelspend import ModelSpend

client = ModelSpend(api_key="your-modelspend-api-key")

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarise this support ticket."}],
)

print(response.choices[0].message.content)
print(response.model) # actual model used after routing

Async usage

import asyncio
from modelspend import AsyncModelSpend

async def main():
    client = AsyncModelSpend()
    response = await client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Classify this ticket."}],
    )
    print(response.choices[0].message.content)

asyncio.run(main())

Environment variable

Set MODELSPEND_API_KEY in your environment to avoid passing the key explicitly. The client reads it automatically.

export MODELSPEND_API_KEY=your-modelspend-api-key
PyPI publication pending — The Python package must not be published to PyPI until package ownership and release credentials are explicitly approved. Use a direct install from the monorepo path for now.

Security

Never expose ModelSpend API keys in public browser JavaScript. Put the SDK in trusted server-side code and call your own backend from the browser.