I recently began using DeepCrawl after putting it off for awhile. I quickly saw the advantage of not only being able to crawl much larger sites (since desktop applications begin to use up a lot of memory once your crawl exceeds a certain number of URLs). I was happy with my decision to begin using …
Read More