Skip to main content

​
Bulk Upload

Bulk Upload lets you generate extraction strategies for many URLs in a single batch. Upload a CSV file, a TXT file, or paste URLs directly — Meter processes them in parallel and groups the resulting strategies together.

​
Overview

Instead of creating strategies one at a time, Bulk Upload lets you:
  1. Provide a list of URLs (tens or hundreds)
  2. Define what to extract (description, output schema, filters)
  3. Optionally enable auto-scheduling with webhooks
  4. Let Meter generate strategies for each URL in parallel
All generated strategies are automatically placed in a strategy group for easy management.

​
Input methods

​
CSV file

Upload a .csv file with URLs. Meter looks for a column named url, link, or website (case-insensitive). If none is found, it uses the first column. 
url,name
https://shop-a.com/products,Shop A
https://shop-b.com/products,Shop B
https://shop-c.com/products,Shop C

​
TXT file

Upload a .txt file with one URL per line: 
https://shop-a.com/products
https://shop-b.com/products
https://shop-c.com/products

​
Paste URLs

Paste URLs directly into the text area, one per line. URLs are automatically deduplicated, and https:// is added if no protocol is present.

​
Configuration options

​
Strategy template

Choose how strategies are generated:
  • New template — Define a strategy name, description, and optionally an output schema and filter. Each generated strategy is named "{Name} - domain.com".
  • Existing template — Select an existing strategy as a template. The description and output schema from the selected strategy are reused.

​
Output schema

Define the exact JSON structure for extraction results. See Output Schemas for details. 
{
  "product_name": "string",
  "price": "number",
  "in_stock": "boolean",
  "image_url": "string"
}
When an output schema is provided, all strategies in the batch produce results with the same field names and types.

​
Result filter

Apply a post-extraction filter to all generated strategies. Only items matching the filter conditions are kept in results.

​
Sub-URL discovery

When enabled, Meter discovers the best matching sub-page for each URL before generating a strategy. This is useful when you provide homepages but need a specific page type (e.g., a product listing page within each site).

​
Auto-scheduling

Enable automatic scheduling to create recurring scrape schedules for every generated strategy:
  • Interval — Every 1, 2, 6, 12, or 24 hours, or weekly
  • Webhook URL — Optional. Receive results via webhook for every scheduled run
  • Webhook type — standard, slack, or auto-detected from URL
  • Webhook secret — Optional. Leave empty to auto-generate a whsec_ secret shared across all schedules in the batch
  • Webhook metadata — Optional JSON object included in every webhook payload
When a webhook secret is auto-generated, it is displayed once in the results step. Store it securely — it won’t be shown again.

​
Strategy groups

All strategies generated in a bulk upload are automatically placed in a strategy group. You can provide a custom group name, or one is generated from the strategy name.

​
Processing

  • Up to 4 URLs are processed in parallel
  • If rate-limited (429 response), all workers pause for 60 seconds before retrying
  • You can cancel processing at any time — pending URLs are marked as cancelled
  • Free-tier accounts are limited to 10 total strategies

​
Results

After processing, you can see:
  • Success and failure counts for each URL
  • Links to individual strategies and the strategy group
  • Error messages for failed URLs
  • A Retry Failed button to re-process only the URLs that failed

​
Step-by-step walkthrough

1

Open Bulk Upload

Navigate to the dashboard and click Bulk Upload (or the upload icon).
2

Choose a template

Select New template and enter a strategy name and description, or select Existing template to reuse an existing strategy’s configuration.
3

Configure options

Optionally enable output schema, result filter, sub-URL discovery, and auto-scheduling.
4

Add URLs

Upload a CSV/TXT file or paste URLs directly. Review the parsed URL count and preview.
5

Start processing

Click Start and watch progress as strategies are generated for each URL.
6

Review results

Check successes and failures. Retry any failed URLs. Navigate to the strategy group to manage all strategies together.

​
Next steps

Strategy Groups

Manage bulk-uploaded strategies as a group

Output Schemas

Define consistent output structure

Filtering

Filter extraction results

Webhooks

Handle webhook notifications from scheduled strategies

​
Need help?

Email me at mckinnon@meter.sh