Map a website's URLs

Map

API Playground Get API key

Cost / request

$0.01

Response format

JSON

Protocol

REST

About this endpoint

Map a website through agntdata to get a list of its URLs quickly. Optionally order results by relevance to a search term. Ideal for discovering a site's structure before scraping specific pages.

Use cases

Discovering all pages on a site before deep scraping

Finding the most relevant pages on a domain for a query

Building a crawl plan for an ingestion pipeline

Pricing

$0.01per successful request

Credits are deducted from your agntdata balance on each successful API call. Failed requests (4xx/5xx) are not charged.

Start for free

Related endpoints

/scrape

/search

Start using this endpoint

Generate your API key and make your first call in under a minute. Free tier available with transparent pricing.

Generate API key View all endpoints

site map URL discovery API

website URL list API

crawl planning API

AI agents web data API

web scraping API

web search API for AI agents

scrape to markdown API

Back to Web