# Connecting a model > Point Railbase at a cloud model, a local server, or a model file on your own hardware. _Updated: 2026-07-21_ Railbase has an AI engine in the core, but it ships with no model of its own. Until you connect one, the AI screens show a banner — *"No AI provider is configured."* Connecting a model happens on one screen: **Settings → AI**, the **LLM** tab (`/_/settings/ai/llm`). AI is a **free part of the core** — Railbase does not bill AI usage and applies no quota. Connect a cloud model and you pay that provider directly; connect a local one and the compute is your own. ## What's on the LLM tab From the top: - **This host** — what this machine can run locally: platform, CPUs, RAM, free disk, GPU, and a **Tier** badge (`external`, `tiny`, `small`, `standard`, `strong`). `external` means too little RAM for a usable on-device model, so a cloud or endpoint provider is the way to go. - **Detected local endpoints** — shown only when a local OpenAI-compatible server is actually reachable. - **Local models (local.native)** — native runtime artifacts installed on this host. - **Connected providers** — every provider you've connected, each with a status badge, a **Test** button and a **Delete** button. - Two connect forms: **Connect an external provider (BYOK)** and **Connect a local endpoint**. ## Connect a cloud provider In **Connect an external provider (BYOK)**: 1. **Provider** is fixed to `anthropic` — the external cloud provider this version supports. The field is read-only. 2. **Model** — optional. Left empty, `claude-sonnet-5` is used. 3. **API key** — your own key (`sk-ant-…`). **Connect** stays disabled until you type one. Press **Connect**. The new entry appears under **Connected providers** with the provider name, the model, and a short hint of the key — its last four characters, nothing more. > [!NOTE] > **The key is write-only.** It is stored in the core-owned `_ai_providers` > collection, read only by the AI engine, and never returned to a screen or an API > response: the provider list carries `key_set` plus that four-character hint and > no key. The configuration change is audited as `ai.provider.upsert` recording the > provider and whether a key is set — never the key. There is no "reveal" and no > in-place key edit on this screen. To replace a key, delete the entry and connect > again. The form always **adds** an entry (it sends no provider id), so pressing Connect twice gives you two. Delete the ones you don't want. ## Connect a local endpoint **Connect a local endpoint** points Railbase at an OpenAI-compatible model server you run yourself — Ollama, LM Studio, a llama.cpp server, apfel, or Railbase's own `apple-fm-bridge`. No key is involved. 1. **Base URL** — for example `http://127.0.0.1:11435`. 2. **Model** — the model name that server expects, for example `apple-foundation`. Railbase talks the OpenAI chat wire format (`POST /v1/chat/completions`) to that address, over a separate HTTP client that permits loopback and LAN addresses — the client used for external cloud calls rejects them. Entries created here are recorded under the provider name `local`. ## Let Railbase find the server for you You usually don't have to type a base URL at all. Each time the tab loads, the core probes the well-known loopback ports **on the Railbase host** — `127.0.0.1:11435` (apple-fm-bridge), `127.0.0.1:11434` (Ollama), `127.0.0.1:1234` (LM Studio) — with `GET /health`, falling back to `GET /v1/models`. Anything that answers appears in the **Detected local endpoints** card with its base URL and first model, and one click on **Connect** wires it up: the entry is stored under the probe's name (`apple-fm-bridge`, `ollama`, `lm-studio`) with that first model. > [!NOTE] > Detection only surfaces candidates — nothing is connected automatically, and > the card renders nothing when no local server is reachable. ## Use the model built into a Mac `tools/apple-fm-bridge` in the Railbase repo is a small Swift sidecar that exposes the Mac's built-in Apple Foundation Model (Apple Intelligence, on-device) as an OpenAI-compatible HTTP server, so Railbase can drive it exactly like Ollama or LM Studio — no API key, no downloaded weights. It exists because the core is pure Go and cannot link the `FoundationModels` framework directly. It needs macOS 26+ on Apple Silicon with **Apple Intelligence enabled**, and the Swift toolchain from the Xcode command-line tools: ```sh swiftc -O -o apple-fm-bridge main.swift APPLE_FM_BRIDGE_PORT=11435 ./apple-fm-bridge ``` The bridge binds `127.0.0.1` only, never off-host. Once it's running, reload the LLM tab and it shows up under **Detected local endpoints** — or add it by hand with `base_url = http://127.0.0.1:11435`. > [!NOTE] > Apple Foundation Models has **no embeddings endpoint**, so this bridge serves > generation only; anything that needs embeddings needs a different backend. It > also reports token counts as zero, so those calls contribute nothing to the > Usage tab's token columns. ## Test a connection Each row under **Connected providers** has a **Test** button. It runs a one-token `ping` generation through that exact stored configuration and reports **Connection OK** plus the model label the backend answered with. A failure surfaces the provider's own error — a bad key, an unreachable endpoint, a model that isn't loaded. A test is a probe, not a business call: it is neither rate-limited nor recorded in the AI audit trail. The probe reaches every provider kind through the same client the generation path uses — a `local.endpoint` provider on loopback or your LAN tests exactly like a cloud one. ## Choose which provider generates With several providers connected, the generator is the **first configured** one unless you name a preference. That preference lives on the **Policies** tab, in **Default provider** — type a provider **name** (for example `anthropic`) or a provider **id**, and leave it empty to keep the first-configured behaviour. It is stored as the `ai.provider.default` policy key. ## Switch a provider off The LLM tab itself offers **Delete**, which removes the entry and the key stored with it. To stop using a provider without losing its configuration, use the **Policies** tab: | Control | Effect | |---|---| | **Provider governance** → **Blocked** | Blocks a whole provider **name** for this scope. Blocked providers are skipped by every AI call; the entry and its key stay. | | **External model egress** → **Disabled** | Master switch for calls to an external cloud provider. Local endpoint providers keep working. | ## About scope An admin session has no tenant, so everything you connect here is stored at the **site scope** — the operator-wide default. A tenant that has configured no providers of its own resolves to these, so one key set once serves every tenant. ## Model files on this host The **Local models (local.native)** card lists native runtime artifacts installed on this machine and installs new ones. An install takes an **Artifact ID**, a **Kind** (`weights` or `library`), and for weights a **Model label** and **Tier**, plus a **Download URL** and the **SHA-256** from the signed catalog. The download is streamed to a content-addressed store under `/storage/ai_runtime` and verified against that hash — a mismatch is rejected with nothing left on disk. Installing artifacts registers them on the host; it does not by itself create a provider entry (the two connect forms create cloud and local endpoint providers). See also [Operating from the admin](operations) for the other day-two screens.