web crawler

This article walks through how to approach a Web Crawler system design problem in interviews, including scale estimation, architectural choices, and trade-offs. This system design problem comes quite a lot in the interviews - and for a reason - this simple high level problem comes in many different flavours and has the potential to get deeper-and-deeper and it gets harder … [Read more...] about System Design – Building Large Scale Web Crawler