Open Source Project #194: deepseek-harness — DeepSeek's AI Agent Development Framework, Everything is a Plugin + PTC Programmatic Composition + Full Trajectory Tracing, 137k Stars

Introduction

The name includes "harness," but this has nothing to do with EleutherAI's lm-evaluation-harness (a model evaluation framework). Different project, different team, different purpose.

This is article #194 in the "One Open Source Project a Day" series. Today's project is deepseek-harness — DeepSeek AI's agent development framework, built around the design principle "Everything is a Plugin."

One sentence: it turns every AI agent capability into a swappable plugin, then offers four fundamentally different execution modes — including PTC, which replaces sequential tool calls with a TypeScript program that composes operations in a single execution.

137,100 Stars, 13,800 Forks. MIT license, TypeScript-primary, Node.js runtime. Currently in Developer Preview — breaking changes possible.

What You'll Learn

• What "harness" means here (not benchmark — constraint layer)
• The Cordis "everything is a plugin" architecture
• PTC mode: the model writes a TypeScript program instead of calling tools step-by-step
• The four operating modes and when to use each
• Append-only session logs and trajectory tracing
• Single-command launch of the full agent dev environment

Prerequisites

• Basic understanding of AI agents and tool calling
• Familiarity with TypeScript/Node.js
• Plugin system design concepts helpful but not required

Background: What "Harness" Means

In engineering, a "harness" isn't a benchmark tool — it's a constraint layer that wraps a system and gives it structured boundaries and capabilities.

A horse harness doesn't restrict the horse; it channels its power precisely. deepseek-harness does the same for AI agents: the plugin constraint layer defines what an agent can do, how it does it, and what records it leaves behind.

Most agent frameworks hardwire tools, memory, planning, and sub-agent logic together. Change one piece and others may break. deepseek-harness's answer: the kernel does exactly one thing — manage plugin registration, dependencies, and lifecycle. Every capability is a plugin. Swapping the tool plugin doesn't touch the planning plugin. Replacing the model backend doesn't affect the UI plugin.

Cordis Architecture: Everything is a Plugin

deepseek-harness runs on Cordis — a TypeScript framework designed for pluggable applications. The Cordis team published a paper on a "spatiotemporal composability" programming paradigm.

Cordis's core constraint: the kernel contains no business logic. It only handles:

• Plugin registration and unloading
• Plugin dependency declaration and resolution
• Plugin lifecycle management (load, ready, unload)
• Service provision and consumption

Every deepseek-harness capability follows this pattern:

Replace the model backend: swap plugin-model-openai for plugin-model-anthropic. Nothing else changes. Disable sub-agents: unload plugin-subagents. No other code needs touching.

Community plugins use the GitHub Topic dsh-plugin — searchable directly on GitHub.

Four Operating Modes

This is deepseek-harness's most distinctive design decision. Different tasks need different capability sets. Rather than one "universal" configuration, the framework ships four presets.

Standard Mode

Full toolchain: file editing, persistent shell, vector retrieval, skills library, planner, goal tracking, sub-agent dispatch, workflow orchestration.

Good for: real engineering tasks requiring the complete agent capability stack.

npx @deepseek-ai/dsh web    # Web UI, Standard mode by default
npx @deepseek-ai/dsh        # CLI mode

PTC Mode: Programmatic Tool Composition

PTC stands for Program That Calls. It's deepseek-harness's most architecturally interesting execution mode.

Standard tool calling flows like this:

Agent calls tool_A → waits for result → calls tool_B → waits → calls tool_C ...

Every step requires a model decision. Sequential execution. No control flow between calls.

PTC mode flows like this:

Agent writes a TypeScript program → program composes tool_A, tool_B, tool_C → single execution

That TypeScript program can use full programming constructs:

• if/else: branch on intermediate results
• for/while: iterate over multiple files or data items
• try/catch: handle tool call errors
• Parallel calls: Promise.all([toolA(), toolB()]) — run multiple operations simultaneously

// Example program generated in PTC mode
async function analyzeProject() {
  const files = await listFiles({ pattern: "src/**/*.ts" });
  
  const analyses = await Promise.all(
    files.map(f => readFile({ path: f }))
  );
  
  const hasTests = files.some(f => f.includes(".test."));
  if (!hasTests) {
    await createFile({
      path: "src/__tests__/basic.test.ts",
      content: generateTestTemplate(analyses)
    });
  }
  
  return summarize(analyses);
}

For tasks with conditional branching and parallelizable steps, PTC outperforms sequential tool calling on both efficiency and debuggability. The full execution plan is visible upfront; the program can be inspected before it runs.

Minimal Mode

Keeps only persistent bash and str_replace_editor. Everything else unloaded.

Purpose: benchmarking. Measure baseline model capability in a clean environment, uncontaminated by a complex tool stack. SWE-bench and similar code evaluation runs typically use this mode.

Creative Mode

Supports runtime plugin inspection and in-memory plugin trials — load a new plugin, observe its behavior, unload it if unsatisfactory, no process restart required.

Good for: developing new plugins, debugging plugin interactions, building custom agent presets.

Full Trajectory Tracing

This is deepseek-harness's core investment in engineering reliability.

All session logs use an append-only format, recording every event:

• Complete system prompt content
• Model chain-of-thought (CoT)
• Each tool call: function name, arguments, return value
• Sub-agent dispatch: which parent started which child, with what context
• Every context injection: which plugin injected what information

Append-only means each event lands at the end of the log. Nothing is modified or overwritten. This enables:

# Resume from a specific step
dsh resume --session <id> --from-step 15
 
# Fork a new execution branch from step 10
dsh fork --session <id> --at-step 10
 
# Replay the entire session (for debugging or validation)
dsh replay --session <id>
 
# Inspect the full context source at step 8
dsh trajectory --session <id> --step 8

The trajectory view breaks down attribution by source: which tool call's return value led to which conclusion, which plugin injected what context, what decision the model made at which step.

For bug reproduction, this mechanism is close to essential. When a standard agent run fails, you see the final result but have to infer what happened in between. With deepseek-harness's full logs, you can pinpoint "step 12: bash tool returned non-zero exit code; step 13: model made a false assumption from that."

Getting Started

One-Line Web UI Launch

npx @deepseek-ai/dsh web
# → Web UI at http://localhost:3080

No pre-installation needed. npx pulls the package automatically.

CLI Mode

npx @deepseek-ai/dsh "Analyze the dependency relationships in this TypeScript project"

Configure Model Backend

# DeepSeek models
export DEEPSEEK_API_KEY=your_key
npx @deepseek-ai/dsh web --model deepseek-coder
 
# OpenAI-compatible endpoint
export OPENAI_API_KEY=your_key
npx @deepseek-ai/dsh web --model gpt-4o

Select Operating Mode

npx @deepseek-ai/dsh web --mode ptc       # PTC programmatic mode
npx @deepseek-ai/dsh web --mode minimal   # Minimal benchmark mode
npx @deepseek-ai/dsh web --mode creative  # Creative plugin trial mode

Install Custom Plugins

# Install a community plugin
pnpm add @dsh-community/plugin-github-tools
 
# Register in config
# dsh.config.ts
export default {
  plugins: [
    require('@dsh-community/plugin-github-tools')
  ]
}

Architecture Comparison

Use Cases

Code tasks: Standard mode. Full toolchain — file editing, shell execution, vector retrieval, sub-agents working across multiple modules in parallel.

Model evaluation: Minimal mode. Clean environment, bash + editor only. Run SWE-bench or custom code benchmarks without tool-stack interference.

Multi-step data processing: PTC mode. The model writes a processing program — batch file handling, parallel API calls, conditional branching — orders of magnitude more efficient than sequential tool calls for the right tasks.

Plugin development: Creative mode. Load a new plugin at runtime, observe behavior, hot-unload if it doesn't work. No process restart cycle for each debug iteration.

Agent behavior debugging: Any mode. Trajectory view traces the full context source for any decision, so you can find exactly where a failure originated.

Project Links

• GitHub: deepseek-ai/deepseek-harness
• Website: deepseek.com/harness
• Community plugins: Search GitHub Topic dsh-plugin
• Cordis framework: github.com/cordiverse/cordis
• Team: DeepSeek AI, Hangzhou

Summary

deepseek-harness isn't building a better agent. It's building a better agent development environment.

The Cordis "everything is a plugin" constraint solves a common engineering problem in agent frameworks: capability components coupled together, so changing one breaks others. When every capability is an independent plugin, replacement, testing, and composition are all traceable operations.

PTC mode is a design decision worth thinking through separately. Standard tool calling is "take one step at a time." PTC is "write the plan first, then execute it all at once." For tasks with conditional branching and parallelizable operations, PTC wins on both efficiency and debuggability. You can see the full execution plan before it runs.

Full trajectory tracing turns the agent from a black box into an inspectable system. Every decision step has a record. Every record has a source. Every failure is precisely reproducible.

137,100 Stars during Developer Preview suggests DeepSeek's timing read is right: model capability is sufficient; what's missing is the engineering infrastructure that makes agents reliable to run and debuggable when they aren't.

Explore PrimeSkills — A marketplace for handpicked AI Agents and skills. Each is validated in real enterprise workflows, stripping away hype and keeping only what truly works.

Welcome to my Homepage for more useful insights and interesting products.