Open source AI models, hosted for your applications

We host open source models on our own infrastructure and give you a standard API to call them from. These are models you build into an application, not a coding assistant you chat with, and there are no GPUs to manage and no framework to run.

Want help building AI into your product or getting your team fluent with it? See our AI application development services.

Create Free Account See Pricing

carpathian.ai / dashboard / ai-instances

Token streamstreamingsse / chat.completions

user Summarize the open jobs for the Fremont site.

Tok/s (3s avg)46.0

Output0 tok

First token240 ms

GPU / modelrunning

nodegpu-04 / us-central

modelosprey

precisionbf16

context128k

VRAM38.2 GB / 80 GB

80 GB capacity47.8%

GPU utilization58.0%

polled every 3 seconds

Throughput, last 60stok/s

active streams3

queue depth0

Request logcompletion latency, end to end

timemodeltokenslatency

09:41:12wren96769 ms

09:41:16cardinal1231557 ms

09:41:20osprey1503553 ms

09:41:25wren1771231 ms

09:41:29cardinal2042437 ms

09:41:33osprey2315353 ms

Free, unlimited AI chat

Before you integrate anything, just talk to a model. Our public AI chat is free and runs on the same US-based infrastructure we host customer models on, and when you are ready to build, those same models are available through an OpenAI-compatible API.

AI chat is free, and it always will be.

Try AI Chat Free Create Free Account

carpathian.ai / chat

Which model should I use to summarize support tickets?

For tickets under a few thousand words, Wren is the fastest option and costs the least per token. If the tickets include long email threads, step up to Osprey for better summaries of multi-party context.

Can I call it with the OpenAI SDK?

Yes. Point base_url at api.carpathian.ai/v1 and keep the rest of your code.

The public chat runs the same models the API serves.

What you get with Carpathian AI chat

The public chat is the fastest way to try Carpathian AI, with no account required to start and a clear path to production when you outgrow it.

No caps, no token limits

Chat as much as you want. No message quotas, no token metering, and no credit card. Open it and start talking.

US-based infrastructure

The chat runs on our own compute in the United States, the same infrastructure behind our managed model hosting, so your prompts are never handed to a third-party API.

A path to production

When you outgrow the chat, the same platform exposes an OpenAI-compatible API. Swap your SDK base URL and keep the code you already wrote.

Built-in guardrails

Every API key carries its own rate limit, token budget, and geo-aware firewall, so you stay in control of usage and spend.

Try AI Chat Free

Call it from the code you already have

The API follows the OpenAI chat completions format, so moving over is a base URL and a key. Streaming, system messages, and temperature behave the way your client already expects.

carpathian.ai / docs / chat-completions

python / openai sdk

# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.carpathian.ai/ai",
    api_key=os.environ["CARPATHIAN_API_KEY"],
)

stream = client.chat.completions.create(
    model="osprey",
    messages=[{"role": "user", "content": "Summarize the open jobs."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

shell / curl

# the same request, without an SDK
curl https://api.carpathian.ai/ai/chat/completions \
  -H "x-api-key: $CARPATHIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "osprey", "stream": true,
       "messages": [{"role": "user", "content": "Summarize the open jobs."}]}'

# list the models this key can call
curl https://api.carpathian.ai/ai/models \
  -H "x-api-key: $CARPATHIAN_API_KEY"

One base URL change. The rest of your OpenAI client code stays as written.

AI model hosting and API access

We host AI models on our own compute infrastructure and expose them through an OpenAI-compatible API. You pick a model, we run it, and you integrate it into your application the same way you would any other REST endpoint. If your code already works with the OpenAI SDK, it works with Carpathian AI.

Billing follows usage. The public chat is free, and on the API you pay for the tokens you send and receive, with every request logged with its token counts and response times so the bill always traces back to something you can see.

Create Free Account See Pricing

carpathian.ai / dashboard / instance-security

EndpointOpenAI-compatible

Compute locationUnited States

IsolationPer organization

FirewallMonitor or enforce

Auto-lock3 attempts / 5 min

Token budgetTotal or monthly

Each instance is scoped to your organization. You set the rate limit, the IP allowlist, and a token budget, and the geo-aware firewall decides what happens when an address you have not seen before shows up.

OpenAI-compatible endpoints

The standard chat completions interface for chat and streaming. Drop in your API key and start making requests against the models you already tested in the chat.

Per-token pricing

Tokens are the only thing we charge for on the API, and the public chat stays free, so what you pay follows what you run rather than a seat count or a reserved instance.

Per-instance security

Rate limiting, IP allowlisting, a geo-aware firewall, and an automatic lock after repeated unauthorized attempts. Every instance is isolated to your organization.

Usage tracking and budgets

Every request is logged with its token counts and response time, and each instance can carry a total or monthly token budget that stops it before it overruns.

Free chat

No account, no message cap, no card on file, and it stays that way

Per token

Tokens are the only thing the API charges for

US-based

Our own compute, so you know where prompts are processed

One base URL

Keep the OpenAI SDK and the code you already wrote

How Carpathian uses AI across our platform

We run the same AI infrastructure we offer to customers on our own platform. This is how we use it internally to keep the platform secure, reduce manual review, and catch abuse before it becomes a problem.

Multi-layer spam and bot detection

Every contact form submission and new signup runs through a multi-layer detection pipeline. The first layer is heuristic, analyzing character entropy, vowel ratios, consonant runs, bigram frequency, and word coverage against common English to catch gibberish, keyboard mashing, and bot-generated text. Submissions that clear it move to a model we host on our own infrastructure, which reads the full submission in context and returns a structured decision with a confidence score and its reasoning.

Pipeline

layer 1heuristics: entropy, bigrams, coverage

layer 2hosted model, structured verdict

verdictapprove, flag, or rejectapprove

Automated account moderation

When the review layer flags a new signup, the system acts on the confidence score it returned. A high score suspends the account outright, a borderline score places a temporary hold that gives the admin team time to review by hand, and a low score leaves the account active while the admins get a notification carrying the full submission and the reasoning behind the flag.

Action by confidence

highpermanent suspensionsuspended

borderlinetemporary hold, manual reviewon hold

lowlogged, admin notifiedactive

Progressive IP blocking

Repeated abuse from the same address escalates rather than jumping straight to a permanent block. Each new ban lengthens the last, running from ten minutes out to a full day, and the fifth makes the block permanent. Temporary bans expire and clean themselves up, so a legitimate visitor who trips a false positive is not locked out for good.

Escalation ladder

first ban10 minutes, expires on its own

second ban30 minutes

third ban1 hour

fourth ban24 hours

fifth banpermanent

Geo-aware AI firewall

Every new IP that reaches an AI inference endpoint gets a GeoIP lookup. The system records the country and region, notifies the instance admins, and holds the address pending until someone approves or blocks it. Monitor mode logs everything and blocks nothing, enforce mode turns away unknown addresses by default, and three blocked attempts inside five minutes lock the instance until an admin releases it.

Firewall queue, monitor mode

198.51.100.24US / Iowaapproved

203.0.113.9DE / Hessepending

192.0.2.77SG / Centralblocked

Common questions about Carpathian AI

What teams ask before they move from the chat to the API.

Is Carpathian AI chat free?

Yes. The public chat has no message quota, no token metering, and no card on file, and it does not require an account to start. It is free and it stays free.

Is the API compatible with the OpenAI SDK?

Yes. Point your client base URL at https://api.carpathian.ai/ai, authenticate with the x-api-key header, and the OpenAI SDKs and client libraries you already use keep working without code changes.

Where do the models run?

On Carpathian compute in the United States, the same infrastructure that serves the public chat, so your prompts are not forwarded to a third-party API.

How is AI usage priced?

Per token. The public chat costs nothing, and API usage is charged on the tokens you send and receive, with every request logged with its token counts and response time.

Can I restrict which IP addresses call my endpoint?

Yes. Each instance carries its own rate limit and IP allowlist, and a geo-aware firewall that runs in monitor mode or enforce mode. Three blocked attempts inside five minutes lock the instance and notify your admins.

Can I bring my own provider keys?

Yes. The proxy accepts keys for the providers you already run, so those requests pass through the same logging, rate limiting, and firewall controls as the models we host.

Help building AI into your product

Hosting gives you the endpoint. If you want help designing the feature around it, our software team builds AI search, assistants, and automation into products, and works with the providers you already run, including OpenAI and Anthropic.

See AI Development

Start with the chat, move to the API when you need it

The chat is free and needs no account. When your application needs an endpoint, create an account and point your OpenAI client at Carpathian.

Create Free Account See Pricing Talk to an Engineer