Skip to main content

​
Welcome to Meter

Meter is an AI-powered web scraping platform that helps you extract data from websites, monitor them for changes, and get notified only when meaningful content updates occur.
API-first extraction, fully automated. Meter automatically discovers and extracts data from hidden APIs when available—delivering cleaner, faster, more reliable results. When APIs aren’t available, it falls back to intelligent HTML extraction. You get the best of both worlds without any extra configuration.

​
What makes Meter different?

Meter solves the core problem with traditional web scraping: wasted resources from re-processing unchanged data. Whether you’re building a RAG system, monitoring competitor prices, or tracking news articles, Meter ensures you only process what’s actually new.

Scrape Any Site, Antibot or Not

Cloudflare, PerimeterX, DataDome—we handle it. Our antibot bypass means you can scrape sites that block traditional scrapers. No more 403s or CAPTCHAs.

Automatic API Discovery

JavaScript-heavy sites? No problem. Meter automatically detects hidden APIs, handles authentication tokens, and extracts data directly from the source—no brittle DOM scraping required.

Generate Once, Scrape Forever

Use AI to generate extraction strategies once. All future scrapes use fast, reliable CSS selectors—no recurring LLM costs.

Structural Content Diffing

Detect meaningful content changes using content hashing and structural signatures. Stop wasting compute on layout updates and noise.

95% Cost Reduction

Cut re-embedding costs for RAG systems by up to 95%. Only update your vector database when content actually changes.

​
How it works

1

Describe What to Extract

Give Meter a URL and a plain English description: “Extract product names and prices” or “Get article headlines and authors.”
2

AI Generates Strategy

Meter’s AI analyzes the page and chooses the optimal extraction method. For traditional pages, it creates CSS selectors. For JavaScript-heavy sites, it automatically discovers APIs and generates direct extraction code.
3

Monitor on Schedule

Set up automated monitoring (hourly, daily, cron-based). Meter scrapes using the saved strategy—no LLM costs.
4

Get Only Real Changes

Receive webhooks or poll for changes. Meter’s diffing detects structural content changes, filtering out layout noise and timestamps.

​
Use cases

Keep your embeddings fresh without wasting tokens. Meter detects content changes and triggers re-embedding only for updated content, reducing costs by up to 95%.
Track competitor prices, product availability, or market trends. Get notified instantly when prices change—ignore layout updates and ads.
Monitor news sites, blogs, or forums for new articles. Detect new posts while filtering out timestamp changes and layout shifts.
Scrape job listings and get alerts when new positions appear. Perfect for talent teams or job search automation.

​
Key concepts

Before diving in, familiarize yourself with these core concepts:
  • Strategy: An AI-generated extraction plan that defines how to scrape a website
  • Job: A single execution of a scrape using a strategy
  • Schedule: Automated recurring scrapes at specified intervals
  • Workflow: DAG-based pipelines that chain multiple strategies together
  • Content Diffing: Intelligent change detection using content hashing and structural signatures

​
Ready to start?

Quick Start

Generate your first strategy and run a scrape in 5 minutes

API Reference

Explore the Python SDK and REST API documentation

Integration Guides

Learn how to integrate Meter with your RAG pipeline

Examples

See real-world examples and copy-paste code

​
Current status

Meter is currently in early beta. The API is functional and stable for core features, but breaking changes may occur. We’re actively developing new features and improvements.
  • ✅ AI strategy generation
  • ✅ Scheduled monitoring (cron and interval)
  • ✅ Webhooks (push-based notifications)
  • ✅ Pull-based change detection
  • ✅ Content diffing (content hashing + structural signatures)
  • ✅ Antibot bypass (feature-gated—contact us)
  • ✅ LLM summaries (feature-gated—contact us)
  • ✅ Automatic API discovery for JavaScript-heavy sites
  • ✅ Workflows (DAG-based multi-step pipelines)
  • 🚧 Semantic similarity detection (roadmap)

​
Need help?

Email me at mckinnon@meter.sh