Open Source Project #246: rec — Recursive's Purpose-Built Local Inference Engine for Running Flint on a Mac, Claiming 1.4–2.6x Over llama.cpp

Introduction

"Our lightning-fast inference engine."

This is #246 in the "One Open Source Project a Day" series. Today's project is rec (recursive-oss/rec).

For running models locally there are already mature options: llama.cpp, Ollama, LM Studio. rec takes a different path: it doesn't try to be general; it serves one model and one kind of scenario. It's an inference engine Recursive wrote specifically for its own Flint language model, aiming to make coding agents run fast enough on a Mac, locally.

A caveat up front: this is a very new and small project (151 Stars and 6 commits at the time of writing), so judgments should be restrained. This article focuses on how to use it, where its boundaries are, and which claims are currently just the project's own.

What You'll Learn

• rec's positioning: why a "purpose-built engine" rather than yet another llama.cpp
• Hardware requirements and installation
• The two usage modes: rec launch and rec serve
• How to connect the OpenAI- and Anthropic-compatible local APIs
• The command overview
• The test conditions behind the performance claim, and how to read it

Prerequisites

• Experience with local LLM tools (any of Ollama, llama.cpp, LM Studio)
• Familiarity with calling OpenAI-compatible APIs
• A basic idea of coding-agent harnesses such as OpenCode or pi

Project Background

Overview

rec is a purpose-built inference engine for running Recursive's Flint language model on your own machine, designed for single-session use. The docs position Flint as "fast, local agentic coding" — a model that can run locally or in the cloud and plug into coding tools (harnesses).

rec does three things:

1. Downloads the model automatically and runs inference efficiently on a Mac
2. Configures and launches a coding harness (pi or OpenCode) for you
3. Exposes local OpenAI- and Anthropic-compatible APIs for your own clients

Team

• Organization: recursive-oss (Recursive)
• The docs say Flint was built by the company's own "autoresearch" system, the "Eureka Machine"
• Individual authors and contributors: not shown on the project page
• Creation date: not stated

Project Stats

• ⭐ GitHub Stars: 151
• 🍴 Forks: 16
• 📝 Commits: 6
• 🐛 Issues: 7, Pull requests: 4
• 📄 License: Apache-2.0
• 📚 Docs: docs.recursive.com

Main Features

Core Purpose

On Apple Silicon Macs, run Flint locally faster than a generic runtime would, and wire it into a coding agent along the way.

Use Cases

1. A local coding agent

   • rec launch pi starts the model and a harness in one command, and code never leaves the machine.

2. A local model for your own client

   • Connect any compatible client through the OpenAI-compatible /v1/chat/completions endpoint.

3. A local model for Anthropic-protocol tools

   • Use the /v1/messages endpoint and point ANTHROPIC_BASE_URL at the local service.

4. Comparing local inference throughput

   • Benchmark single-session decoding against llama.cpp on the same machine.

Quick Start

Hardware and environment requirements

• An Apple Silicon Mac: M4 Pro, M5 Pro, or M5 Max
• At least 48 GB of unified memory
• About 30 GB of free disk space
• Node.js 22 and npm if you use pi

Install

curl -fsSL https://raw.githubusercontent.com/recursive-oss/rec/main/install.sh | bash

rec installs to ~/.local/bin and is added to PATH for new terminals; other data lives in ~/.local/share/rec.

Launch

rec launch pi        # preferred
rec launch opencode

The first run downloads the model and harness. The service keeps running after you close the terminal; stop it with rec stop. The harnesses rec manages are isolated from any you installed yourself, and rec exec runs commands inside that environment:

rec exec pi install npm:PACKAGE

The project warns explicitly: rec is optimized for a single session. For multiple agents in parallel, see the other ways of running the model in the official docs.

Using Your Own Harness

Start the service first (keep the terminal open; once flint is ready. appears, connect from another terminal):

rec serve

To see which environment variables and command line rec launch uses, print them without running:

rec launch --args pi
rec launch --args opencode

Swap in your own harness path. To keep your own settings and sessions, remove the variables that point at rec's settings and sessions.

Call the OpenAI-compatible API directly:

curl -fsS http://127.0.0.1:11500/v1/chat/completions \
  -H "Authorization: Bearer $(rec auth token)" \
  -H 'Content-Type: application/json' \
  -d '{"model":"flint","messages":[{"role":"user","content":"Say hello."}],"max_tokens":128}'

Configuration for other clients:

• OpenAI-compatible: base URL http://127.0.0.1:11500/v1, model flint, API key from rec auth token
• Anthropic-compatible: endpoint http://127.0.0.1:11500/v1/messages; point ANTHROPIC_BASE_URL at http://127.0.0.1:11500; send the token as a bearer or x-api-key; model is again flint

Core Features

1. An engine dedicated to Flint

   • It doesn't try to support every model, which leaves room for targeted optimization.

2. A one-command local coding setup

   • rec launch downloads the model, then configures and starts pi or OpenCode.

3. Two compatible APIs

   • OpenAI and Anthropic protocols, so existing toolchains connect locally.

4. Isolated harnesses

   • Harnesses managed by rec don't interfere with your system installs.

5. Local authentication

   • The service requires a key from rec auth token rather than leaving an open local port.

6. Full command-line management

   • Start/stop, status, model download, environment check, update, and uninstall.

Command Overview

Use rec COMMAND --help for each command's options.

Advantages

The project's own performance claim: on an M4 Pro, throughput is 1.4–2.6x that of llama.cpp, across context lengths from 1K to 252K.

Test conditions:

A qualitative comparison with alternatives (the project page gives no formal comparison; this is my summary based on its positioning):

When is it worth trying?

• You have a qualifying Mac and want a local coding agent
• You want to see for yourself how much faster a purpose-built engine can be than a generic runtime
• You need both OpenAI and Anthropic local interfaces

Deep Dive

The trade-off: dedicated, not general

The most interesting thing about rec is its trade-off. A general inference runtime must support many model architectures, quantization formats, and backends, at the cost of being hard to push to the limit for any one model. rec inverts that: only Flint, only Apple Silicon, only single-session decoding.

The benefits and costs are both explicit:

Project structure

The root shows cli, engine, packaging, and tests directories, plus go.mod, .golangci.yml, and ruff.toml. That suggests at least a CLI/engine split and Go plus Python tooling. Language breakdown and the actual engine implementation aren't given on the project page — this is inference from file names.

What the engine does internally, and why it's faster than llama.cpp, is explained neither on the project page nor on the docs site, so I won't speculate. To learn the mechanics you'd need to read the engine source or wait for an official write-up.

How to read "1.4–2.6x"

This is the project's own result. A few things to keep in mind:

• Decoding only, no prefill. In coding-agent use, prefill over long contexts often dominates, and this number excludes it.
• One machine. Only M4 Pro 48 GB data is given.
• The opponent is llama.cpp. Version, build options, and quantization details aren't provided.
• Different models. rec runs Flint; whether the comparison used the same model and quantization isn't stated.

The safer reading: treat it as a lead worth re-measuring yourself, not a verdict.

Limitations and things to watch

• High hardware bar. Only M4 Pro, M5 Pro, and M5 Max, with 48 GB unified memory and about 30 GB of disk.
• Model-bound. It's Flint's engine, not a general runtime.
• Single session. The project explicitly says to use another setup for parallel agents.
• Very early stage. 6 commits; interfaces and behavior may change.
• curl | bash install. For production or sensitive environments, read install.sh first.
• Limited info on the docs site. Model architecture, context length, and release dates aren't on the docs home page; check the model card.

Links and Resources

Official

• 🌟 GitHub: https://github.com/recursive-oss/rec
• 📚 Docs: https://docs.recursive.com
• 🐛 Issue Tracker: https://github.com/recursive-oss/rec/issues

Related

• Flint model card and release status: see the model card and releases pages on the docs site
• Other ways to run Flint listed by the docs: Ollama, Unsloth, LM Studio, llama.cpp, plus Hugging Face Transformers, SGLang, and vLLM

Summary and Outlook

Key Takeaways

1. A dedicated engine: rec optimizes only for Flint, Apple Silicon, and single sessions, trading generality for speed.
2. Out of the box: one command installs it, and rec launch pi gives you a local coding-agent environment.
3. Friendly interfaces: OpenAI- and Anthropic-compatible local APIs, with local authentication.
4. Read the performance claim carefully: 1.4–2.6x is self-reported, decode-only, and from a single machine — worth re-measuring yourself.

Who It's For

• Developers with a 48 GB+ high-end Mac who want a local coding agent
• Engineers interested in the design trade-offs of local inference engines
• Evaluators who want to compare dedicated engines with general runtimes

Suggested Path

Run rec doctor to confirm your machine qualifies, then rec launch pi for the end-to-end flow. To verify the performance claim, reproduce it on one machine under the stated conditions (4-bit, 1,024 output tokens, five runs per context length) and record prefill time separately.

Explore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.

Visit my personal site for more insights and interesting products.