Completed
Web Spider
Solo Developer
A distributed web crawler in Rust with a Redis-backed BFS queue, URL normalisation and deduplication, and configurable concurrency and depth, orchestrated with Docker Compose.
- Rust
- Redis
- Docker
Work
Case studies covering the problem, the constraints, and how each system was put together.
1 projects shown