A URL in.
Useful content out.

Make your first request

Sign in, open API keys, and create a key. Store it in an environment variable. Set SCRAPEGOAT_URL to this deployment’s origin.

export SCRAPEGOAT_URL='https://your-deployment.example'
export SCRAPEGOAT_API_KEY='sr_your_key'

curl "$SCRAPEGOAT_URL/v1/fetch" \
  -H "Authorization: Bearer $SCRAPEGOAT_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://example.com",
    "formats": ["markdown"],
    "budget": {
      "cost_ceiling_micro_usd": 50000,
      "max_attempts": 3
    }
  }'

The response includes content, selected_strategy, attempts, validation, latency_ms, and the provider cost. Read the X-Charged-Micro-USD response header for the amount charged to your account, including the service fee.

Fetch a page

Send POST /v1/fetch with a JSON body. The hosted API uses the router’s fetch schema. These are the fields you will use most often.

Unknown fields, duplicate fields, null values, and trailing JSON are rejected. If you request a capability with no eligible route, the API returns a capability error. It does not silently remove the requirement.

Use the playground to build a request and copy the cURL code.

A key for each project

Use Authorization: Bearer sr_…. Keys work with the fetch and account-owned request-history endpoints. Browser sessions manage keys and payments; API keys cannot create other keys or access billing.

The full secret is displayed once when the key is created. Only a hash and masked prefix are stored. Create a replacement key, update your application, then revoke the old key. A revoked key cannot start new work. Already-admitted work can finish.

Follow the request

The response has an X-Request-ID. Use it to open the request in Activity, or call GET /v1/requests/{id} with your API key. This hosted endpoint returns account-owned metadata and attempt summaries, not stored page content.

You can supply a canonical req_ ULID in X-Request-ID. UUID v4 headers from proxies are also accepted and map to a stable canonical ID. Use the returned ID to read request history. An admitted ID cannot be used again. Duplicate or uncertain IDs return 409 so a possibly charged request is not replayed.

If a connection closes or a request is canceled, inspect Activity before retrying. A pending request can mean the final result is uncertain, including after a process restart. Reservations are not released automatically in that case.

Small, explicit budgets

One dollar equals 1,000,000 micro-USD. The service fee is 20% of the recorded provider cost. Charges round up to the next micro-USD and are capped at the provider budget plus the service fee. For a $0.05 provider budget, the maximum customer charge is $0.06.

Before work starts, the account reserves the budget plus the service fee. When the final cost is known, the unused amount returns to the available balance. Provider work can cost money even if the final request fails. Estimated charges are labeled as estimates; they are not represented as invoice measurements.

Unknown costs remain pending with their reservation held. Known zero cost stays zero. If a provider’s charge exceeds its reservation, the customer still pays no more than the configured budget plus the service fee.

Top-ups of $10, $50, or $100 use Stripe Checkout. The application adds prepaid balance only after a verified payment event. Returning from the checkout page does not prove that a payment completed. There is no monthly subscription or automatic top-up. Stripe handles card details, invoices, and the billing portal. Payment availability depends on deployment configuration.

Refunds remove the corresponding unused or spent balance. A refund or dispute can make the balance negative, which blocks new paid work until funds are restored. Contact the deployment operator for billing support and uncertain-cost review.

Limits and errors

Each account can have 20 active keys, 5 concurrently pending requests, and 60 admitted requests per minute. The hosted request body limit is 64 KiB and the encoded response limit is 16 MiB. Provider and router limits can be lower.

Data and request privacy

The console keeps your verified Google identity, sessions, hashed API keys, payment references, and account-owned request metadata. It stores normalized target hosts and resource paths, formats, timing, route names, outcomes, and cost evidence.

Page bodies are returned in the request response and are not saved by the console. Console history excludes target query values, cookies, and authorization header values. Provider and router operation data is kept in a separate private store. Each configured upstream provider processes the target URL and the supported request options needed to fetch it.

Only the owner’s signed-in session or active key can read that account’s request history. Authentication and card processing are handled by Google and Stripe, respectively. No third-party analytics or tracking scripts are loaded by this website.

This page describes the product’s data flow. The deployment operator must publish its own legal terms, privacy notice, retention policy, and contact details before a public launch.