Product
Limits & Quotas
Rate Limits
Rate limits are applied per API key and vary by plan. When a rate limit is exceeded, the API returns 429 Too Many Requests with a Retry-After header indicating when the next request can be made.
| Plan | Requests / second | Requests / day | Concurrent connections |
|---|---|---|---|
| Starter | 5 | 10,000 | 5 |
| Growth | 25 | 100,000 | 25 |
| Business | 100 | 500,000 | 100 |
| Enterprise | Custom | Custom | Custom |
Bandwidth Quotas
Each plan includes a monthly bandwidth allowance. Bandwidth is measured as the total size of response bodies delivered to the Client.
| Plan | Monthly bandwidth | Overage rate |
|---|---|---|
| Starter | 10 GB | €0.10 / GB |
| Growth | 100 GB | €0.08 / GB |
| Business | 1 TB | €0.05 / GB |
| Enterprise | Custom | Custom |
Request Limits
- Maximum URL length: 2,048 characters.
- Maximum payload size: 10 MB per response.
- Request timeout: 60 seconds (configurable up to 120s on Business and Enterprise plans).
- Batch size: up to 1,000 URLs per batch request.
- Webhook payload: maximum 5 MB per delivery.
Data Retention
- Data in transit: maximum 72 hours, then automatically deleted.
- Webhook retries: up to 5 attempts over 24 hours with exponential backoff.
- Failed jobs: retained for 48 hours for debugging, then deleted.
Test Environment
Test keys (crw_sk_test_...) have reduced limits to prevent accidental billing:
- Maximum 100 requests per day.
- Maximum 1 GB bandwidth per month.
- Batch size limited to 10 URLs.
Monitoring Usage
Check your current usage and remaining quotas via the GET /v3/usage endpoint. Response headers on every request include:
X-RateLimit-Limit— maximum requests per second for your plan.X-RateLimit-Remaining— requests remaining in the current window.X-RateLimit-Reset— Unix timestamp when the rate limit resets.
Need higher limits? Contact sales@crawlo.com to discuss Enterprise plans with custom rate limits, dedicated infrastructure and priority support.