01
Starts at your page
Kaiju reads the start page, then the pages it links to, up to your link depth.
Crawl
Crawl reads a start page and the pages it links to, up to the page and depth limits you set.
Free. No sign-up. No card.
/docs/1,240 wordsRead/docs/install860 wordsRead/docs/api2,010 wordsPartial/docs/changelogPast your limitHow it works
How Crawl moves from your start page.
01
Kaiju reads the start page, then the pages it links to, up to your link depth.
02
Only the paths you include, never the paths you skip. It stops at your page limit.
03
One result for each page, read like a Scrape result. Pages past your limit are listed, not read.
What you get back
Each step lights the part of the example that answers it. Pick one, or let it play.
{ "operationId": "9c4b1e27…", "state": "completed", "highWater": 3, "items": [ { "id": "d4a0ad0e…", "sequence": 1, "kind": "crawl", "url": "https://example.com/docs/", "status": "ok", "bytes": 5840, "bodyLink": "/v1/operations/9c4b1e27…/results/d4a0ad0e…" }, { "id": "669adfc5…", "sequence": 2, "kind": "crawl", "url": "https://example.com/docs/install", "status": "ok", "bytes": 7312, "bodyLink": "/v1/operations/9c4b1e27…/results/669adfc5…" }, { "id": "7e3e79c3…", "sequence": 3, "kind": "crawl", "url": "https://example.com/docs/api", "status": "ok", "bytes": 9126, "bodyLink": "/v1/operations/9c4b1e27…/results/7e3e79c3…" } ], "complete": true}Build a request
Change an option and the request changes with it. Run it here, free, then keep going in the playground.
End it with /* to keep the crawl under that path.
Every option has a default Change one and the request updates. Focus an option to read what it does.
POST /v1/crawl
{
"url": "https://northwind.example/docs/",
"includePaths": [
"/docs/"
]
}Free. No sign-up. Run it here, then keep going in the playground.
Which endpoint
Start from what you have. Each endpoint does one job.
You know the page you need.
POST /v1/scrape
{
"url": "https://northwind.example/blog/launch-week"
}You need the list of pages before you read them.
POST /v1/map
{
"url": "https://northwind.example",
"maxUrls": 100
}You need the text of many pages in one section.
POST /v1/crawl
{
"url": "https://northwind.example/docs/",
"maxPages": 5,
"maxDepth": 1
}You need named values, such as prices, not whole pages.
POST /v1/collect
// 503 capability_unavailable: not available yetCost and limits
Draft rates. We confirm them before release.
1 credit per page read
The draft rate for each page Kaiju reads. We confirm it before release.
Limits set by the service
Read GET /v1/capabilities for the current limits. The time limit covers the whole job, not each page.
0 credits
Pages outside your scope: not fetched, so not charged. Pages past your limit: not read, not charged, and listed for you.
Work out a whole job in the pricing estimator.
Try Crawl
The playground is free and needs no sign-up.
Analytics preferences
Analytics helps Kaiju understand which pages and actions are useful. Marketing tracking can identify business visitors for follow-up. Essential storage remains available either way.
Read the Cookie Policy and Privacy notice.