Railbase
GPTClaude

Connecting a model

Point Railbase at a cloud model, a local server, or a model file on your own hardware.

Updated

Railbase has an AI engine in the core, but it ships with no model of its own. Until you connect one, the AI screens show a banner — "No AI provider is configured." Connecting a model happens on one screen: Settings → AI, the LLM tab (/_/settings/ai/llm).

AI is a free part of the core — Railbase does not bill AI usage and applies no quota. Connect a cloud model and you pay that provider directly; connect a local one and the compute is your own.

What's on the LLM tab

From the top:

  • This host — what this machine can run locally: platform, CPUs, RAM, free disk, GPU, and a Tier badge (external, tiny, small, standard, strong). external means too little RAM for a usable on-device model, so a cloud or endpoint provider is the way to go.
  • Detected local endpoints — shown only when a local OpenAI-compatible server is actually reachable.
  • Local models (local.native) — native runtime artifacts installed on this host.
  • Connected providers — every provider you've connected, each with a status badge, a Test button and a Delete button.
  • Two connect forms: Connect an external provider (BYOK) and Connect a local endpoint.

Connect a cloud provider

In Connect an external provider (BYOK):

  1. Provider is fixed to anthropic — the external cloud provider this version supports. The field is read-only.
  2. Model — optional. Left empty, claude-sonnet-5 is used.
  3. API key — your own key (sk-ant-…). Connect stays disabled until you type one.

Press Connect. The new entry appears under Connected providers with the provider name, the model, and a short hint of the key — its last four characters, nothing more.

Note

The key is write-only. It is stored in the core-owned _ai_providers collection, read only by the AI engine, and never returned to a screen or an API response: the provider list carries key_set plus that four-character hint and no key. The configuration change is audited as ai.provider.upsert recording the provider and whether a key is set — never the key. There is no "reveal" and no in-place key edit on this screen. To replace a key, delete the entry and connect again.

The form always adds an entry (it sends no provider id), so pressing Connect twice gives you two. Delete the ones you don't want.

Connect a local endpoint

Connect a local endpoint points Railbase at an OpenAI-compatible model server you run yourself — Ollama, LM Studio, a llama.cpp server, apfel, or Railbase's own apple-fm-bridge. No key is involved.

  1. Base URL — for example http://127.0.0.1:11435.
  2. Model — the model name that server expects, for example apple-foundation.

Railbase talks the OpenAI chat wire format (POST /v1/chat/completions) to that address, over a separate HTTP client that permits loopback and LAN addresses — the client used for external cloud calls rejects them. Entries created here are recorded under the provider name local.

Let Railbase find the server for you

You usually don't have to type a base URL at all. Each time the tab loads, the core probes the well-known loopback ports on the Railbase host — 127.0.0.1:11435 (apple-fm-bridge), 127.0.0.1:11434 (Ollama), 127.0.0.1:1234 (LM Studio) — with GET /health, falling back to GET /v1/models. Anything that answers appears in the Detected local endpoints card with its base URL and first model, and one click on Connect wires it up: the entry is stored under the probe's name (apple-fm-bridge, ollama, lm-studio) with that first model.

Note

Detection only surfaces candidates — nothing is connected automatically, and the card renders nothing when no local server is reachable.

Use the model built into a Mac

tools/apple-fm-bridge in the Railbase repo is a small Swift sidecar that exposes the Mac's built-in Apple Foundation Model (Apple Intelligence, on-device) as an OpenAI-compatible HTTP server, so Railbase can drive it exactly like Ollama or LM Studio — no API key, no downloaded weights. It exists because the core is pure Go and cannot link the FoundationModels framework directly.

It needs macOS 26+ on Apple Silicon with Apple Intelligence enabled, and the Swift toolchain from the Xcode command-line tools:

swiftc -O -o apple-fm-bridge main.swift
APPLE_FM_BRIDGE_PORT=11435 ./apple-fm-bridge

The bridge binds 127.0.0.1 only, never off-host. Once it's running, reload the LLM tab and it shows up under Detected local endpoints — or add it by hand with base_url = http://127.0.0.1:11435.

Note

Apple Foundation Models has no embeddings endpoint, so this bridge serves generation only; anything that needs embeddings needs a different backend. It also reports token counts as zero, so those calls contribute nothing to the Usage tab's token columns.

Test a connection

Each row under Connected providers has a Test button. It runs a one-token ping generation through that exact stored configuration and reports Connection OK plus the model label the backend answered with. A failure surfaces the provider's own error — a bad key, an unreachable endpoint, a model that isn't loaded. A test is a probe, not a business call: it is neither rate-limited nor recorded in the AI audit trail. The probe reaches every provider kind through the same client the generation path uses — a local.endpoint provider on loopback or your LAN tests exactly like a cloud one.

Choose which provider generates

With several providers connected, the generator is the first configured one unless you name a preference. That preference lives on the Policies tab, in Default provider — type a provider name (for example anthropic) or a provider id, and leave it empty to keep the first-configured behaviour. It is stored as the ai.provider.default policy key.

Switch a provider off

The LLM tab itself offers Delete, which removes the entry and the key stored with it. To stop using a provider without losing its configuration, use the Policies tab:

Control Effect
Provider governance → Blocked Blocks a whole provider name for this scope. Blocked providers are skipped by every AI call; the entry and its key stay.
External model egress → Disabled Master switch for calls to an external cloud provider. Local endpoint providers keep working.

About scope

An admin session has no tenant, so everything you connect here is stored at the site scope — the operator-wide default. A tenant that has configured no providers of its own resolves to these, so one key set once serves every tenant.

Model files on this host

The Local models (local.native) card lists native runtime artifacts installed on this machine and installs new ones. An install takes an Artifact ID, a Kind (weights or library), and for weights a Model label and Tier, plus a Download URL and the SHA-256 from the signed catalog. The download is streamed to a content-addressed store under <data-dir>/storage/ai_runtime and verified against that hash — a mismatch is rejected with nothing left on disk. Installing artifacts registers them on the host; it does not by itself create a provider entry (the two connect forms create cloud and local endpoint providers).

See also Operating from the admin for the other day-two screens.

Was this page helpful?Thanks for your feedback!
← PreviousThe schedulerNext →AI agents and approvals