One Open Source Project a Day (Part 48): Agent-Reach - Give Your AI Agent Internet Capabilities, Zero API Fees Supporting Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu

Introduction

"Give your AI agent eyes to see the entire internet."

This is Part 48 of the "One Open Source Project a Day" series. Today's project is Agent-Reach (GitHub).

AI agents can help you write code, edit documents, manage projects—but when you ask them to find something online, they're stuck: can't watch YouTube videos, can't search Twitter (API costs money), Reddit returns 403, XiaoHongShu won't open, Bilibili won't connect... Agent-Reach is a scaffolding tool that gives AI agents internet capabilities with one click: supports Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu, Douyin, LinkedIn, WeChat Official Accounts, Weibo, RSS and more platforms, zero API fees, one-click installation, compatible with all agents that can run command lines like Claude Code, OpenClaw, Cursor, Windsurf.

Why it matters:

• 🚀 One-Click Installation: Copy one sentence to your agent, automatically completes all configuration
• 💰 Zero API Fees: All tools are open-source and free, no paid API dependencies
• 🔌 Pluggable Architecture: Each channel is independent, can be replaced if unsatisfied
• 🤖 Compatible with All Agents: Claude Code, OpenClaw, Cursor, Windsurf, and more
• 🔒 Privacy & Security: Cookies stored locally only, not uploaded or shared, code fully open-source
• 🩺 Built-in Diagnostics: agent-reach doctor command checks all channel statuses
• 🔄 Continuous Updates: Tracks platform changes, auto-fixes when platforms block

What You'll Learn

• Agent-Reach's positioning and "scaffolding" design philosophy
• Supported platforms and upstream tool selection (xreach, yt-dlp, Jina Reader, Exa, etc.)
• One-click installation process and configuration mechanism
• Pluggable architecture: How to replace upstream tools
• Security mechanisms: Cookie management, privacy protection
• Comparison with similar tools (LangChain Tools, CrewAI Tools, etc.)

Prerequisites

• Basic understanding of AI agents
• Basic command-line tool usage
• Basic knowledge of web scraping and API calls (optional)

Project Background

Project Overview

Agent-Reach is a scaffolding tool for giving AI agents internet capabilities. It's not a framework, but helps you complete tool selection and configuration: after installation, agents directly call upstream tools (xreach CLI, yt-dlp, mcporter, gh CLI, etc.) without going through Agent-Reach's wrapper layer.

Core Features:

• Scaffolding Positioning: Not a framework, but a configuration tool that helps select and configure upstream tools
• Zero API Fees: All tools are open-source and free, no paid API dependencies
• One-Click Installation: Copy one sentence to your agent, automatically completes all configuration
• Pluggable Architecture: Each channel is independent, can be replaced if unsatisfied
• Compatible with All Agents: Any agent that can run command lines can use it
• Continuous Updates: Tracks platform changes, auto-fixes when platforms block

Core Problems Solved:

• AI agents cannot access internet content (YouTube, Twitter, Reddit, etc.)
• Platform API costs are high (Twitter API costs $0.005 per read)
• Need to configure tools one by one, install dependencies, adjust configurations, high trial-and-error cost
• Platform anti-scraping and API changes require continuous maintenance

Target Users:

• Developers using AI agents (Claude Code, OpenClaw, Cursor, etc.)
• Users who need agents to access internet content
• Developers who don't want to pay for APIs
• Teams that need continuous tool configuration maintenance

Author/Team

• Author: Panniantong (GitHub)
• Background: Pure vibe coding project, author uses it daily, will continue maintaining
• Philosophy: Contribute to Web 4.0 infrastructure
• Contact: pnt01@foxmail.com, @Neo_Reidlab

Project Statistics

• ⭐ GitHub Stars: ~8.2k
• 🍴 Forks: ~593
• 📦 Version: v1.3.0 (actively maintained, 180+ commits)
• 📄 License: MIT
• 🌐 Documentation: GitHub README
• 💬 Community: GitHub Issues

Tech Stack:

• Language: Python (91.5%), Shell (8.5%)
• Python Version: 3.8+
• Core Dependencies: xreach, yt-dlp, Jina Reader, Exa, mcporter, feedparser, etc.

Core Features

Core Purpose

Agent-Reach's core purpose is to give AI agents internet capabilities with one click, enabling agents to:

1. Read Web Pages: Read any webpage content via Jina Reader
2. Search Twitter: Search tweets via xreach CLI, zero API fees
3. Extract YouTube Subtitles: Extract video subtitles and metadata via yt-dlp
4. Access GitHub: Read repositories, search, create issues/PRs via gh CLI
5. Access Bilibili: Extract subtitles and search via yt-dlp
6. Access Reddit: Free search via Exa, or configure proxy to read posts
7. Access XiaoHongShu: Read, search, post via xiaohongshu-mcp
8. Access Douyin: Parse videos, get watermark-free downloads via douyin-mcp-server
9. Access LinkedIn: Read public pages via linkedin-scraper-mcp
10. Access WeChat Official Accounts: Search and read via wechat-article-for-ai
11. Access Weibo: Hot topics, search, user feeds, comments
12. Subscribe to RSS: Parse RSS/Atom feeds via feedparser
13. Web Search: AI semantic search via Exa (MCP access, free, no key needed)

Use Cases

1. Content Research and Analysis

   • "Help me see what this YouTube tutorial is about" → Extract subtitles and metadata
   • "Search how people evaluate this product on Twitter" → Search tweets
   • "Go to Reddit to see if anyone encountered the same bug" → Search and read posts

2. Code and Project Research

   • "What does this GitHub repository do? What do the issues say?" → Read repository and issues
   • "Search for LLM frameworks on GitHub" → Search repositories

3. Social Media Monitoring

   • "Help me see the reputation of this product on XiaoHongShu" → Search and read notes
   • "See discussions about this topic on Weibo" → Search and read Weibo

4. Information Aggregation

   • "Help me subscribe to these RSS feeds, notify me when updated" → Parse RSS feeds
   • "Search the web for latest LLM framework comparisons" → AI semantic search

5. Content Creation Assistance

   • "What does this video say, help me summarize" → Extract subtitles and summarize
   • "Help me see what this webpage says" → Read webpage content

Quick Start

One-Click Installation:

Copy this sentence to your AI agent (Claude Code, OpenClaw, Cursor, etc.):

帮我安装 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md

The agent will automatically:

1. Install CLI tool: pip install agent-reach
2. Install system dependencies: Node.js, gh CLI, mcporter, xreach, etc.
3. Configure search engine: Access Exa via MCP (free, no API key needed)
4. Detect environment: Determine if local computer or server, provide configuration suggestions
5. Register SKILL.md: Install usage guide in agent's skills directory

Update:

帮我更新 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md

Safe Mode:

帮我安装 Agent Reach（安全模式）：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
安装时使用 --safe 参数

Diagnostics:

agent-reach doctor

One command tells you the status of each channel (which works, which doesn't, how to fix).

Usage Examples:

After installation, just tell the agent:

• "Help me see this link" → curl https://r.jina.ai/URL read any webpage
• "What does this GitHub repository do" → gh repo view owner/repo
• "What does this video say" → yt-dlp --dump-json URL extract subtitles
• "Help me see this tweet" → xreach tweet URL --json
• "Subscribe to this RSS" → feedparser parse
• "Search for LLM frameworks on GitHub" → gh search repos "LLM framework"

No need to remember commands. After reading SKILL.md, the agent knows what to call.

Key Features

1. Supported Platforms

   • 🌐 Web: Jina Reader reads any webpage (ready to use)
   • 📺 YouTube: yt-dlp subtitle extraction + video search (ready to use)
   • 📡 RSS: feedparser reads any RSS/Atom feed (ready to use)
   • 🔍 Web Search: Exa AI semantic search (MCP access, free, no key needed)
   • 📦 GitHub: gh CLI reads public repositories + search (ready to use), private repos need login
   • 🐦 Twitter/X: xreach reads single tweet (ready to use), search needs Cookie configuration
   • 📺 Bilibili: yt-dlp local subtitle extraction + search (ready to use), server needs proxy
   • 📖 Reddit: Exa free search (ready to use), reading posts needs proxy
   • 📕 XiaoHongShu: xiaohongshu-mcp read, search, post (needs configuration)
   • 🎵 Douyin: douyin-mcp-server video parsing, watermark-free download (needs configuration)
   • 💼 LinkedIn: linkedin-scraper-mcp reads public pages (ready to use), profile details need configuration
   • 💬 WeChat Official Accounts: wechat-article-for-ai search + read (ready to use)
   • 📰 Weibo: Hot topics, search, user feeds, comments (ready to use)
   • 🎙️ Xiaoyuzhou Podcast: Whisper transcription (needs configuration)

2. Pluggable Architecture

   • Each channel is backed by an independent upstream tool
   • Can be replaced if unsatisfied: web.py → Jina Reader (can replace with Firecrawl, Crawl4AI)
   • twitter.py → xreach (can replace with Nitter, official API)
   • youtube.py → yt-dlp (can replace with YouTube API, Whisper)
   • github.py → gh CLI (can replace with REST API, PyGithub)

3. Security Mechanisms

   • 🔒 Local Credential Storage: Cookies and tokens only stored in ~/.agent-reach/config.yaml, file permissions 600
   • 🛡️ Safe Mode: agent-reach install --safe won't automatically modify system
   • 👀 Fully Open-Source: Code transparent, can be reviewed anytime
   • 🔍 Dry Run: agent-reach install --dry-run previews all operations
   • 🍪 Cookie Security Recommendations: Use dedicated small accounts, don't use main account (account ban risk)

4. Compatibility

   • Compatible with all agents that can run command lines: Claude Code, OpenClaw, Cursor, Windsurf, Codex, etc.
   • OpenClaw users need to enable exec permission first: openclaw config set tools.profile "coding"

5. Continuous Updates

   • Tracks platform changes (auto-fixes when platforms block)
   • Adds new channels (adds when requested)
   • Regularly updates underlying tools to latest versions (yt-dlp, xreach, Jina Reader, etc.)

Project Advantages

Why Choose Agent-Reach?

• One-click installation: Copy one sentence to your agent, automatically completes all configuration
• Zero API fees: All tools are open-source and free, no paid API dependencies
• Pluggable architecture: Each channel is independent, can be replaced if unsatisfied
• Continuous updates: Tracks platform changes, auto-fixes when platforms block
• Built-in diagnostics: agent-reach doctor command checks all channel statuses

Deep Dive

Architecture Design

Agent-Reach uses a scaffolding + pluggable architecture design:

Core Design Philosophy:

Agent-Reach is a scaffolding (scaffolding), not a framework.

When you set up a new agent environment, you always spend time finding tools, installing dependencies, adjusting configurations—what to use for Twitter? How to bypass Reddit blocks? How to extract YouTube subtitles? You have to go through this every time.

What Agent-Reach does is simple: Helps you complete these tool selection and configuration tasks.

After installation, agents directly call upstream tools (xreach CLI, yt-dlp, mcporter, gh CLI, etc.) without going through Agent-Reach's wrapper layer.

Architecture Structure:

Agent-Reach
├── channels/ (Channel modules)
│   ├── web.py          → Jina Reader (replaceable)
│   ├── twitter.py      → xreach (replaceable)
│   ├── youtube.py      → yt-dlp (replaceable)
│   ├── github.py       → gh CLI (replaceable)
│   ├── bilibili.py     → yt-dlp (replaceable)
│   ├── reddit.py       → JSON API + Exa (replaceable)
│   ├── xiaohongshu.py  → mcporter MCP (replaceable)
│   ├── douyin.py       → mcporter MCP (replaceable)
│   ├── linkedin.py     → linkedin-mcp (replaceable)
│   ├── wechat.py       → camoufox+miku (replaceable)
│   ├── rss.py          → feedparser (replaceable)
│   ├── exa_search.py   → mcporter MCP (replaceable)
│   └── __init__.py     → Channel registration (for doctor detection)
├── config/ (Configuration management)
│   └── config.yaml     → Cookie, token storage (local, permissions 600)
├── scripts/ (Installation scripts)
│   └── install.sh      → One-click installation script
└── docs/ (Documentation)
    ├── install.md      → Installation guide
    └── update.md       → Update guide

Each channel file is only responsible for:

• Detecting if corresponding upstream tool is available (check() method)
• Providing status information to agent-reach doctor
• Actual reading and searching is done by agents directly calling upstream tools

Upstream Tool Selection

Current Selection:

📌 These are "current selections". Not satisfied? Just replace the corresponding file. This is exactly what scaffolding means.

Installation Process

One-Click Installation Process:

1. Agent Receives Instruction: User copies installation instruction to agent
2. Download Installation Guide: Agent reads install.md
3. Execute Installation Script:
   • pip install agent-reach (install CLI tool)
   • Detect and install system dependencies (Node.js, gh CLI, mcporter, xreach, etc.)
   • Configure search engine (access Exa via MCP)
   • Detect environment (local vs server)
4. Register SKILL.md: Install usage guide in agent's skills directory
5. Complete: Agent can now use all upstream tools

Safe Mode:

agent-reach install --safe

Won't automatically modify system, only lists what's needed, you decide whether to install.

Dry Run:

agent-reach install --dry-run

Preview all operations, make no changes.

Configuration Mechanism

Cookie Configuration:

For platforms requiring cookies (Twitter, XiaoHongShu, etc.), prioritize using Chrome extension Cookie-Editor to export cookies, send to agent for configuration.

Unified process:

1. Browser login to platform
2. Cookie-Editor exports cookies
3. Send to agent: agent-reach configure twitter-cookies "your_cookies"

Proxy Configuration:

Platforms like Reddit and Bilibili may be blocked on servers, configure residential proxy:

agent-reach configure proxy http://user:pass@ip:port

Recommend Webshare ($1/month). Local computers generally won't encounter this issue.

GitHub Login:

Tell agent "Help me login to GitHub", agent will guide you through GitHub CLI authentication.

Diagnostic Tool

agent-reach doctor:

One command checks all channel statuses:

• ✅ Which channels are available
• ❌ Which channels are unavailable
• 🔧 How to fix unavailable channels

Example output:

✅ Web (Jina Reader): Available
✅ YouTube (yt-dlp): Available
✅ RSS (feedparser): Available
✅ GitHub (gh CLI): Available
⚠️ Twitter (xreach): Not configured (need Cookie)
⚠️ Reddit: Need proxy for server IP
❌ 小红书: Not installed (need Docker)

Uninstall Mechanism

agent-reach uninstall

Will clear:

• ~/.agent-reach/ (including all tokens/cookies)
• Each agent's skill files
• MCP configurations in mcporter

Options:

# Preview only, don't actually delete
agent-reach uninstall --dry-run
 
# Only delete skill files, keep token configuration (for reinstall)
agent-reach uninstall --keep-config

Uninstall Python package itself: pip uninstall agent-reach

Project Links and Resources

Official Resources

• 🌟 GitHub: https://github.com/Panniantong/Agent-Reach
• 📚 Documentation: GitHub README
• 💬 Community: GitHub Issues
• 📧 Email: pnt01@foxmail.com
• 🐦 Twitter/X: @Neo_Reidlab

Related Resources

• Jina Reader: github.com/jina-ai/reader
• yt-dlp: github.com/yt-dlp/yt-dlp
• xreach: npmjs.com/package/xreach-cli
• Exa: exa.ai
• mcporter: github.com/steipete/mcporter
• xiaohongshu-mcp: github.com/xpzouying/xiaohongshu-mcp
• douyin-mcp-server: github.com/yzfly/douyin-mcp-server

Target Audience

• AI Agent Users: Developers using Claude Code, OpenClaw, Cursor, Windsurf, etc.
• Content Researchers: Need agents to access internet content for research
• Cost-Sensitive Users: Developers who don't want to pay for APIs
• Multi-Platform Needs: Users needing to access multiple platforms
• Rapid Prototyping: Want to quickly give agents internet capabilities

Learning Value:

• ✅ Scaffolding tool design philosophy and practices
• ✅ Pluggable architecture implementation
• ✅ Upstream tool selection and integration
• ✅ Cookie and proxy configuration management
• ✅ Agent tool ecosystem construction
• ✅ Continuous maintenance and update strategies

Visit my homepage for more useful knowledge and interesting products