Loading scenario...
Hint 1 — locked
Hint 2 — locked
Hint 3 — locked
Web Crawler
Problem
Build a scalable web crawler that discovers, downloads, and indexes web pages across the internet. It must crawl billions of pages while respecting robots.txt, avoiding duplicate URLs, and storing crawled content for a search index.
Functional Requirements
Non-Functional
Prerequisites
Design your Web Crawler
Drag components from the Brief panel → switch to Components tab
Click a dimension to see what's missing. Score updates as you design.