Estimate crawl budget consumption and how long a full crawl takes.
Crawl budget is finite, and URLs that should not be indexed consume it. Faster server responses increase the rate search engines are willing to crawl, which is why performance affects indexing. On large sites crawl waste is the main reason new content takes weeks to be indexed.
Crawl Budget
Days for a full crawl = crawlable URLs ÷ useful pages crawled per day
Days for a full crawl = crawlable URLs ÷ useful pages crawled per day Crawl budget is finite, and URLs that should not be indexed consume it. Faster server responses increase the rate search engines are willing to crawl, which is why performance affects indexing.
On large sites crawl waste is the main reason new content takes weeks to be indexed.
This calculator takes 4 inputs: Total crawlable URLs, Pages crawled per day, Share of crawl spent on low-value URLs, Average server response time. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.