crawlo.com
  • Architecture
  • Delivery
  • Volume
  • Compliance
  • API Docs
Home Architecture Delivery Volume Compliance API Docs
Product

Limits & Quotas

Last updated: 8 February 2025 · API v3

Rate Limits

Rate limits are applied per API key and vary by plan. When a rate limit is exceeded, the API returns 429 Too Many Requests with a Retry-After header indicating when the next request can be made.

PlanRequests / secondRequests / dayConcurrent connections
Starter510,0005
Growth25100,00025
Business100500,000100
EnterpriseCustomCustomCustom

Bandwidth Quotas

Each plan includes a monthly bandwidth allowance. Bandwidth is measured as the total size of response bodies delivered to the Client.

PlanMonthly bandwidthOverage rate
Starter10 GB€0.10 / GB
Growth100 GB€0.08 / GB
Business1 TB€0.05 / GB
EnterpriseCustomCustom

Request Limits

  • Maximum URL length: 2,048 characters.
  • Maximum payload size: 10 MB per response.
  • Request timeout: 60 seconds (configurable up to 120s on Business and Enterprise plans).
  • Batch size: up to 1,000 URLs per batch request.
  • Webhook payload: maximum 5 MB per delivery.

Data Retention

  • Data in transit: maximum 72 hours, then automatically deleted.
  • Webhook retries: up to 5 attempts over 24 hours with exponential backoff.
  • Failed jobs: retained for 48 hours for debugging, then deleted.

Test Environment

Test keys (crw_sk_test_...) have reduced limits to prevent accidental billing:

  • Maximum 100 requests per day.
  • Maximum 1 GB bandwidth per month.
  • Batch size limited to 10 URLs.

Monitoring Usage

Check your current usage and remaining quotas via the GET /v3/usage endpoint. Response headers on every request include:

  • X-RateLimit-Limit — maximum requests per second for your plan.
  • X-RateLimit-Remaining — requests remaining in the current window.
  • X-RateLimit-Reset — Unix timestamp when the rate limit resets.

Need higher limits? Contact sales@crawlo.com to discuss Enterprise plans with custom rate limits, dedicated infrastructure and priority support.

crawlo.com

Raw data extraction infrastructure.
We extract, you analyse.

Product

API DocumentationStatusChangelogLimits & Quotas

Company

ContactTerms of ServicePrivacy PolicyAcceptable UseCookies

Legacy Access

Suite 360Pricing ToolAbracadabra
Legal Notice: Crawlo.com is operated by CRAWLO DATA INTELLIGENCE LTD, Efesou 9, Paralimni 5280, Famagusta, Cyprus, as a licensee of third-party data extraction technology. Crawlo acts exclusively as a public data extraction infrastructure provider.
© 2025 CRAWLO DATA INTELLIGENCE LTD