> ## Documentation Index
> Fetch the complete documentation index at: https://docs.anysite.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Anysite CLI

> Command-line toolkit for web data extraction, batch processing, dataset pipelines, and analytics

<div style={{ textAlign: "center", margin: "2rem 0" }}>
  <img src="https://mintcdn.com/horizondatawave/8MjJq6bqhA5EMT4i/public/images/anysite-cli-logo.jpg?fit=max&auto=format&n=8MjJq6bqhA5EMT4i&q=85&s=540885001143f94558acc0e4f81a968c" alt="Anysite CLI" style={{ maxWidth: "280px", display: "inline-block" }} width="1024" height="1024" data-path="public/images/anysite-cli-logo.jpg" />
</div>

## What is Anysite CLI?

Anysite CLI is a powerful command-line toolkit that gives you full control over web data — from extraction to analysis to database loading. It supports **2,400+ endpoints** across 450+ data sources including LinkedIn, Instagram, Twitter/X, Reddit, YouTube, ProductHunt, Google, GitHub, Amazon, SEC EDGAR, and more.

<Card title="GitHub Repository" icon="github" href="https://github.com/anysiteio/anysite-cli">
  Source code, issues, and contributions — MIT licensed
</Card>

<CardGroup cols={2}>
  <Card title="Single API Calls" icon="bolt" href="/cli/api-calls">
    Make instant API requests with flexible output formats and field filtering
  </Card>

  <Card title="Batch Processing" icon="layer-group" href="/cli/batch-processing">
    Process thousands of inputs in parallel with rate limiting and error handling
  </Card>

  <Card title="Dataset Pipelines" icon="diagram-project" href="/cli/datasets/overview">
    Build declarative YAML workflows with chained dependencies and scheduling
  </Card>

  <Card title="Database Integration" icon="database" href="/cli/database/connections">
    Load data into SQLite, PostgreSQL, or ClickHouse with auto-schema and diff-sync
  </Card>

  <Card title="Database Discovery" icon="binoculars" href="/cli/database/discovery">
    Auto-discover database structure and browse saved catalogs
  </Card>

  <Card title="LLM Analysis" icon="brain" href="/cli/llm-analysis">
    Classify, summarize, enrich, and deduplicate data using AI models
  </Card>

  <Card title="SQL Querying" icon="chart-simple" href="/cli/querying">
    Query collected datasets with DuckDB SQL and generate analytics
  </Card>

  <Card title="Agent Protocol" icon="code" href="/cli/agent-protocol">
    Structured JSON output, exit codes, and discovery for AI agents
  </Card>

  <Card title="Agent Examples" icon="robot" href="/cli/agents/data-agent">
    AI agents for automated data collection and executive search
  </Card>
</CardGroup>

## Quick Start

Get up and running in 60 seconds:

<Steps>
  <Step title="Install the CLI">
    ```bash theme={null}
    curl -fsSL https://raw.githubusercontent.com/anysiteio/anysite-cli/main/install.sh | bash
    ```

    Or via pip: `pip install anysite-cli` — see [Installation](/cli/installation) for all methods.
  </Step>

  <Step title="Configure your API key">
    Get your API key from [app.anysite.io](https://app.anysite.io) and configure it:

    ```bash theme={null}
    anysite config set api_key YOUR_API_KEY
    ```
  </Step>

  <Step title="Update the schema cache">
    ```bash theme={null}
    anysite schema update
    ```
  </Step>

  <Step title="Make your first request">
    ```bash theme={null}
    anysite api /api/linkedin/user user=satyanadella
    ```
  </Step>
</Steps>

## Key Capabilities

| Feature                    | Description                                                                                                     |
| -------------------------- | --------------------------------------------------------------------------------------------------------------- |
| **2,400+ Endpoints**       | Access data from LinkedIn, Instagram, Twitter/X, Reddit, YouTube, ProductHunt, Google, GitHub, Amazon, and more |
| **Flexible Output**        | JSON, JSONL, CSV, and table formats with field selection and exclusion                                          |
| **Batch Processing**       | Parallel execution with rate limiting and configurable error handling                                           |
| **Dataset Pipelines**      | YAML-based multi-source workflows with dependency chains                                                        |
| **Database Loading**       | Auto-schema creation, upsert, and diff-sync for SQLite, PostgreSQL, and ClickHouse                              |
| **LLM Analysis**           | Classify, summarize, enrich, generate, match, and deduplicate records                                           |
| **Incremental Collection** | Resume from where you left off with cursor tracking                                                             |
| **Database Discovery**     | Auto-discover database structure, LLM-enriched catalogs                                                         |
| **Agent Protocol**         | Auto-JSON in pipes, structured errors, exit codes, discovery payload                                            |
| **Scheduling**             | Cron-based automation with webhook notifications                                                                |

## How It Fits in the Anysite Ecosystem

<CardGroup cols={3}>
  <Card title="REST API" icon="api" href="/api-reference">
    Direct HTTP API for programmatic integration into your applications
  </Card>

  <Card title="MCP Server" icon="server" href="/mcp-server/tools">
    AI agent integration via Model Context Protocol for Claude, Cursor, and other tools
  </Card>

  <Card title="n8n Nodes" icon="workflow" href="/n8n-nodes/installation">
    Visual workflow automation with drag-and-drop nodes
  </Card>
</CardGroup>

## Claude Code Skill

Want AI to build and run pipelines for you? Install the anysite-cli skill for Claude Code:

```bash theme={null}
/plugin marketplace add https://github.com/anysiteio/agent-skills
/plugin install anysite-cli@anysite-skills
```

The skill gives Claude Code full knowledge of all anysite commands — describe what data you need in natural language and let the agent handle the rest. See [Installation](/cli/installation#claude-code-skill) for details.

<Tip>
  Anysite CLI is ideal when you need programmatic control, reproducible pipelines, or want to integrate data extraction into scripts and CI/CD workflows.
</Tip>