Crawl Engine
Maps the site and creates the shared website intelligence model.
Updated August 2026
Overview
The Crawl Engine discovers and fetches pages from the site, then structures what it found into the shared intelligence model used by every later stage.
What it collects
- Discovered URLs and their relationships.
- Response status, redirects, and canonical signals.
- Page structure, headings, and on-page content.
- Internal link graph data.
Crawl limits
The number of pages crawled per audit is capped by your plan.
| Plan | Sites | Audits | Pages per audit | Notes |
|---|---|---|---|---|
| Free | 1 | 1 | 5 | Preview of the top 3 issues |
| Starter | 1 | 25 / month | 25 | Full report access |
| Growth | 5 | 100 / month | 100 | Full report access |
| Agency | 20 | 300 / month | 500 | Multi-site workflows |