Skip to main content

​
Bulk Upload

Bulk Upload lets you generate extraction strategies for many URLs in a single batch. Upload a CSV file, a TXT file, or paste URLs directly — Meter processes them in parallel and groups the resulting strategies together.

​
Overview

Instead of creating strategies one at a time, Bulk Upload lets you:
  1. Provide a list of URLs (tens or hundreds)
  2. Define what to extract (description, output schema, filters)
  3. Optionally enable auto-scheduling with webhooks
  4. Let Meter generate strategies for each URL in parallel
All generated strategies are automatically placed in a strategy group for easy management.

​
Input methods

​
CSV file

Upload a .csv file with URLs. Meter looks for a column named url, link, or website (case-insensitive). If none is found, it uses the first column.

​
TXT file

Upload a .txt file with one URL per line:

​
Paste URLs

Paste URLs directly into the text area, one per line. URLs are automatically deduplicated, and https:// is added if no protocol is present.

​
Configuration options

​
Strategy template

Choose how strategies are generated:
  • New template — Define a strategy name, description, and optionally an output schema and filter. Each generated strategy is named "{Name} - domain.com".
  • Existing template — Select an existing strategy as a template. The description and output schema from the selected strategy are reused.

​
Output schema

Define the exact JSON structure for extraction results. See Output Schemas for details.
When an output schema is provided, all strategies in the batch produce results with the same field names and types.

​
Result filter

Apply a post-extraction filter to all generated strategies. Only items matching the filter conditions are kept in results.

​
Sub-URL discovery

When enabled, Meter discovers the best matching sub-page for each URL before generating a strategy. This is useful when you provide homepages but need a specific page type (e.g., a product listing page within each site).

​
Auto-scheduling

Enable automatic scheduling to create recurring scrape schedules for every generated strategy:
  • Interval — Every 1, 2, 6, 12, or 24 hours, or weekly
  • Webhook URL — Optional. Receive results via webhook for every scheduled run
  • Webhook type — standard, slack, or auto-detected from URL
  • Webhook secret — Optional. Leave empty to auto-generate a whsec_ secret shared across all schedules in the batch
  • Webhook metadata — Optional JSON object included in every webhook payload
When a webhook secret is auto-generated, it is displayed once in the results step. Store it securely — it won’t be shown again.

​
Strategy groups

All strategies generated in a bulk upload are automatically placed in a strategy group. You can provide a custom group name, or one is generated from the strategy name.

​
Processing

  • Up to 4 URLs are processed in parallel
  • If rate-limited (429 response), all workers pause for 60 seconds before retrying
  • You can cancel processing at any time — pending URLs are marked as cancelled
  • Free-tier accounts are limited to 10 total strategies

​
Results

After processing, you can see:
  • Success and failure counts for each URL
  • Links to individual strategies and the strategy group
  • Error messages for failed URLs
  • A Retry Failed button to re-process only the URLs that failed

​
Step-by-step walkthrough

1

Open Bulk Upload

Navigate to the dashboard and click Bulk Upload (or the upload icon).
2

Choose a template

Select New template and enter a strategy name and description, or select Existing template to reuse an existing strategy’s configuration.
3

Configure options

Optionally enable output schema, result filter, sub-URL discovery, and auto-scheduling.
4

Add URLs

Upload a CSV/TXT file or paste URLs directly. Review the parsed URL count and preview.
5

Start processing

Click Start and watch progress as strategies are generated for each URL.
6

Review results

Check successes and failures. Retry any failed URLs. Navigate to the strategy group to manage all strategies together.

​
Next steps

Strategy Groups

Manage bulk-uploaded strategies as a group

Output Schemas

Define consistent output structure

Filtering

Filter extraction results

Webhooks

Handle webhook notifications from scheduled strategies

​
Need help?

Email me at mckinnon@meter.sh