How to Self-Host Robin Behind a Private LLM

To run AI code review with zero external API calls, you point Robin at a private LLM endpoint instead of a public one — set the base URL to your own model server, and your diff is reviewed without ever leaving your network. Because Robin is bring-your-own-key, “self-hosting the model” is mostly a configuration change: the same Action that calls OpenAI can call a local Ollama or vLLM server on your infrastructure. The one extra piece for true isolation is running the CI job on a self-hosted runner too, so the code is never checked out on someone else’s machine. This guide walks through the whole setup.

Key Takeaways

• Robin’s bring-your-own-key design means pointing at a private LLM is a config change, not a fork.
• Set LLM_BASE_URL to your local model endpoint (e.g. an Ollama server) and the API key to ollama.
• For full network isolation, run the GitHub Actions job on a self-hosted runner as well as the model.
• You own model quality and uptime, but no diff ever leaves your infrastructure.

Why Point Robin at a Private LLM?

The default BYOK setup already keeps the tool vendor out of your data path — your diff goes only to the endpoint you configure. Self-hosting the model goes one step further: the endpoint is inside your own network, so the diff never reaches any external provider at all. For teams under strict data-residency rules, in regulated industries, or simply unwilling to send source code anywhere outside, this is the configuration that satisfies “code must not leave our infrastructure.”

Robin is built for this. Its API key input even defaults to ollama, signaling that a self-hosted Ollama endpoint is a first-class target, not an afterthought.

Step 1: Stand Up a Private Model Endpoint

Run an OpenAI-compatible model server inside your network. The common choices:

• Ollama — the simplest path. Install it on a machine with a capable GPU (or Apple Silicon with enough unified memory), pull a code-capable model (llama3.2, codellama:13b, or similar), and it serves an OpenAI-compatible API on port 11434.
• vLLM — for higher throughput on a GPU cluster, serving open-weight models with an OpenAI-compatible endpoint.

The key requirement is that the endpoint speaks the OpenAI chat-completions format, which both of the above do. Note that smaller open-weight models give solid first-pass review; larger ones (that need more serious hardware) close the gap with frontier models on complex logic.

Step 2: Point Robin at It

This is the configuration change. Set Robin’s secrets to target your endpoint rather than a public API:

• LLM_BASE_URL — your model server’s URL, e.g. http://your-server:11434/v1
• LLM_API_KEY — ollama for a local Ollama endpoint (no real key needed)
• LLM_MODEL — the model you pulled, e.g. llama3.2 or codellama:13b

That’s it on the Robin side. The Action now sends review requests to your internal endpoint instead of an external provider. Everything else about how Robin works — diff filtering, size caps, inline comments, the summary — is unchanged.

Step 3: Use a Self-Hosted Runner for True Isolation

Here’s the piece teams miss. Even with a private model, a GitHub-hosted runner checks out your code on GitHub’s infrastructure before your workflow runs — so the diff has technically left your network, even if the model call stays internal. For genuine end-to-end isolation, run the CI job on a self-hosted runner inside the same network as your model.

With both pieces in place:

1. A self-hosted runner checks out the code inside your network.
2. Robin composes the review request and sends it to your internal model endpoint.
3. No external network call is made; the diff never crosses your boundary.

This two-part setup (private model and self-hosted runner) is what compliance-driven teams actually need when the requirement is absolute.

Step 4: Tune for Your Hardware

Self-hosted models on modest hardware can be slower than cloud APIs. A couple of Robin’s knobs help:

• max-diff-size caps how much of a large diff is sent, keeping requests within your model’s practical limits.
• llm-timeout-ms sets the request timeout; raise it if large diffs on a slower local model occasionally time out.

Start with a smaller model to validate the workflow end to end, then move to a larger one if review depth falls short for your codebase.

The Trade-Off, Honestly

Self-hosting maximizes privacy and eliminates per-token cost, but you take on real operational work: the model, the GPU, updates, and performance tuning are now yours. For many teams, the middle path — BYOK pointed at a vetted external provider with a data-processing agreement — delivers most of the control with none of the model ops. Reach for full self-hosting when your policy genuinely requires that code never leave your network.

In our experience maintaining Robin, the smoothest adoptions start on the BYOK middle path and tighten to fully self-hosted only when a compliance requirement makes it necessary — and because it’s a one-line endpoint change, that transition doesn’t mean re-tooling.

Frequently Asked Questions

Does self-hosting Robin mean my code never leaves my network?

Only if you self-host both the model and the CI runner. A private model endpoint keeps the model call internal, but a GitHub-hosted runner still checks out your code on GitHub’s infrastructure first. With a self-hosted runner plus a private model, no diff crosses your network boundary.

What model should I run locally for code review?

Start with a code-capable open-weight model like codellama:13b or a recent llama variant via Ollama — solid for first-pass review and runnable on a single high-end machine. Move to a larger model on stronger hardware if you need deeper reasoning on complex diffs.

Is self-hosting worth it versus just using a BYOK provider?

It depends on your requirements. If your policy demands code never leave your network, self-hosting is the answer and worth the operational cost. If you can send code to a vetted provider under a DPA, the BYOK middle path gives most of the control without running a model.

Where to Go From Here

The Robin documentation covers the exact configuration for a custom endpoint, including pointing at a local model and tuning diff size and timeouts for self-hosted setups. If you’re weighing self-hosted against other approaches first, the self-hosted vs SaaS comparison lays out the trade-offs.