Map

List every page on a site first.

Map lists the page addresses of a site from its sitemaps and its links, and does not read the pages.

Free. No sign-up. No card.

Mapnorthwind.example
  • /Sitemap
  • /blog/Sitemap
  • /blog/launch-weekLink
  • /docs/Sitemap
  • /docs/installLink
  • /pricingSitemap
Found 6 addresses · read 0 pages

How it works

From one address to every page.

How Map finds the pages of a site.

01

Reads the sitemaps

Kaiju reads the sitemaps the site publishes, if it has any.

02

Follows the links

It also reads the links on the pages it opens, inside the scope you set.

03

Returns the list

Each address once, up to your limit. Map does not read the pages behind them.

What you get back

One entry for each address.

Each step lights the part of the example that answers it. Pick one, or let it play.

Step 1 of 4
GET/v1/operations/{id}/results
Example
{  "operationId": "5e2d9b14…",  "state": "completed",  "highWater": 3,  "items": [    {      "id": "63eb5535…",      "sequence": 1,      "kind": "map",      "url": "https://example.com/",      "status": "ok",      "bytes": 118,      "bodyLink": "/v1/operations/5e2d9b14…/results/63eb5535…"    },    {      "id": "efbdb029…",      "sequence": 2,      "kind": "map",      "url": "https://example.com/docs/",      "status": "ok",      "bytes": 126,      "bodyLink": "/v1/operations/5e2d9b14…/results/efbdb029…"    },    {      "id": "e7a09620…",      "sequence": 3,      "kind": "map",      "url": "https://example.com/pricing",      "status": "ok",      "bytes": 121,      "bodyLink": "/v1/operations/5e2d9b14…/results/e7a09620…"    }  ],  "complete": true}

Build a request

Every option, with its default.

Change an option and the request changes with it. Run it here, free, then keep going in the playground.

The example site is made up. Any public address works in the playground.

Main options
Maximum URLsDefault: 100 URLs · 1 to 250
URLs
Only URLs containingOptional · up to 200 characters
More options7
Discovery
Find pages fromDefault: Sitemaps and links
Scope
Include pathsUp to 10 paths, each starting with /
Skip pathsUp to 10 paths, each starting with /
Include subdomainsPlanned
Planned
Ignore query for duplicatesDefault: off
Page accessShared by Scrape, Map and Crawl.
Open pages withDefault: Automatic
Wait after page loadDefault: 0 ms · 0 to 30,000
ms
Time limitDefault: 30 s · 5 to 120
s
Reuse a saved copy up toPlannedNow: Fresh every run
Planned: 0 to 720 h
Mobile browserPlanned
Planned

Every option has a default Change one and the request updates. Focus an option to read what it does.

POST /v1/map
{
  "url": "https://northwind.example/"
}

Free. No sign-up. Run it here, then keep going in the playground.

Which endpoint

Pick the endpoint for the job.

Start from what you have. Each endpoint does one job.

You know the page you need.

You send
One page address
You get
The page as Markdown, with its source
Limit
One page for each job.
See Scrape
Smallest request
POST /v1/scrape
{
  "url": "https://northwind.example/blog/launch-week"
}

You need the list of pages before you read them.

You send
A site, or a section of it
You get
A list of page addresses
Limit
Set by the service. Read GET /v1/capabilities.
You are on this page
Smallest request
POST /v1/map
{
  "url": "https://northwind.example",
  "maxUrls": 100
}

You need the text of many pages in one section.

You send
A start page and your limits
You get
One result for each page read
Limit
Set by the service. Read GET /v1/capabilities.
See Crawl
Smallest request
POST /v1/crawl
{
  "url": "https://northwind.example/docs/",
  "maxPages": 5,
  "maxDepth": 1
}

You need named values, such as prices, not whole pages.

You send
Up to four pages and the fields you name
You get
One record with those values
Limit
Up to 4 pages and 20 fields for each job.
See Extract
Smallest request
POST /v1/collect
// 503 capability_unavailable: not available yet

Cost and limits

Listing costs less than reading.

Draft rates. We confirm them before release.

1 credit per 100 pages listed

Each page

The draft rate for each page Kaiju reads. We confirm it before release.

Limits set by the service

Each job

Read GET /v1/capabilities for the current limits. The time limit covers the whole job, not each page.

0 credits

Not charged

Pages outside your scope: not fetched, so not charged. Pages past your limit: not read, not charged, and listed for you.

Work out a whole job in the pricing estimator.

Try Map

Map a sample site now.

The playground is free and needs no sign-up.

Privacy choices

Optional website tracking

These choices control optional browser tracking on Kaiju's public website. The app asks separately for identified product analytics. Turning Analytics off here also stops optional app analytics. These choices do not change information you intentionally submit through a contact, evaluation, or meeting form.

Essential

Website delivery, security, forms, and this saved preference.

Always on