> ## Documentation Index
> Fetch the complete documentation index at: https://docs.meter.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Introduction

> AI-powered web scraping with intelligent change detection to keep your data fresh

# Welcome to Meter

Meter is an AI-powered web scraping platform that helps you extract data from websites, monitor them for changes, and get notified only when meaningful content updates occur.

<Note>
  **API-first extraction, fully automated.** Meter automatically discovers and extracts data from hidden APIs when available—delivering cleaner, faster, more reliable results. When APIs aren't available, it falls back to intelligent HTML extraction. You get the best of both worlds without any extra configuration.
</Note>

## What makes Meter different?

Meter solves the core problem with traditional web scraping: **wasted resources from re-processing unchanged data**. Whether you're building a RAG system, monitoring competitor prices, or tracking news articles, Meter ensures you only process what's actually new.

<CardGroup cols={2}>
  <Card title="Scrape Any Site, Antibot or Not" icon="shield-check">
    Cloudflare, PerimeterX, DataDome—we handle it. Our antibot bypass means you can scrape sites that block traditional scrapers. No more 403s or CAPTCHAs.
  </Card>

  <Card title="Automatic API Discovery" icon="radar">
    JavaScript-heavy sites? No problem. Meter automatically detects hidden APIs,
    handles authentication tokens, and extracts data directly from the source—no
    brittle DOM scraping required.
  </Card>

  <Card title="Generate Once, Scrape Forever" icon="wand-magic-sparkles">
    Use AI to generate extraction strategies once. All future scrapes use fast, reliable CSS selectors—no recurring LLM costs.
  </Card>

  <Card title="Structural Content Diffing" icon="code-compare">
    Detect meaningful content changes using content hashing and structural signatures. Stop wasting compute on layout updates and noise.
  </Card>

  <Card title="95% Cost Reduction" icon="piggy-bank">
    Cut re-embedding costs for RAG systems by up to 95%. Only update your vector database when content actually changes.
  </Card>
</CardGroup>

## How it works

<Steps>
  <Step title="Describe What to Extract">
    Give Meter a URL and a plain English description: "Extract product names and prices" or "Get article headlines and authors."
  </Step>

  <Step title="AI Generates Strategy">
    Meter's AI analyzes the page and chooses the optimal extraction method. For
    traditional pages, it creates CSS selectors. For JavaScript-heavy sites, it
    automatically discovers APIs and generates direct extraction code.
  </Step>

  <Step title="Monitor on Schedule">
    Set up automated monitoring (hourly, daily, cron-based). Meter scrapes using the saved strategy—no LLM costs.
  </Step>

  <Step title="Get Only Real Changes">
    Receive webhooks or poll for changes. Meter's diffing detects structural content changes, filtering out layout noise and timestamps.
  </Step>
</Steps>

## Use cases

<AccordionGroup>
  <Accordion title="RAG Systems & Vector Databases">
    Keep your embeddings fresh without wasting tokens. Meter detects content changes and triggers re-embedding only for updated content, reducing costs by up to 95%.
  </Accordion>

  <Accordion title="Price Monitoring">
    Track competitor prices, product availability, or market trends. Get notified instantly when prices change—ignore layout updates and ads.
  </Accordion>

  <Accordion title="Content Aggregation">
    Monitor news sites, blogs, or forums for new articles. Detect new posts while filtering out timestamp changes and layout shifts.
  </Accordion>

  <Accordion title="Job Board Tracking">
    Scrape job listings and get alerts when new positions appear. Perfect for talent teams or job search automation.
  </Accordion>
</AccordionGroup>

## Key concepts

Before diving in, familiarize yourself with these core concepts:

* **[Strategy](/concepts/strategies)**: An AI-generated extraction plan that defines how to scrape a website
* **[Job](/concepts/jobs)**: A single execution of a scrape using a strategy
* **[Schedule](/concepts/schedules)**: Automated recurring scrapes at specified intervals
* **[Workflow](/concepts/workflows)**: DAG-based pipelines that chain multiple strategies together
* **[Content Diffing](/concepts/change-detection)**: Intelligent change detection using content hashing and structural signatures

## Ready to start?

<CardGroup cols={2}>
  <Card title="Quick Start" icon="rocket" href="/quickstart">
    Generate your first strategy and run a scrape in 5 minutes
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/python/installation">
    Explore the Python SDK and REST API documentation
  </Card>

  <Card title="Integration Guides" icon="book-open" href="/guides/overview">
    Learn how to integrate Meter with your RAG pipeline
  </Card>

  <Card title="Examples" icon="flask" href="/guides/examples/ecommerce-monitoring">
    See real-world examples and copy-paste code
  </Card>
</CardGroup>

## Current status

<Note>
  Meter is currently in **early beta**. The API is functional and stable for core features, but breaking changes may occur. We're actively developing new features and improvements.
</Note>

* ✅ AI strategy generation
* ✅ Scheduled monitoring (cron and interval)
* ✅ Webhooks (push-based notifications)
* ✅ Pull-based change detection
* ✅ Content diffing (content hashing + structural signatures)
* ✅ Antibot bypass (feature-gated—[contact us](mailto:mckinnon@meter.sh))
* ✅ LLM summaries (feature-gated—[contact us](mailto:mckinnon@meter.sh))
* ✅ Automatic API discovery for JavaScript-heavy sites
* ✅ Workflows (DAG-based multi-step pipelines)
* 🚧 Semantic similarity detection (roadmap)

## Need help?

Email me at [mckinnon@meter.sh](mailto:mckinnon@meter.sh)