Ive found that sites are less likely to block you from scraping them if you vary the user agent you send in your scrape requests. Being able to set 1 agent is good, but being able to set 10 or 20 and have the application choose one at random when requesting a target url would help reduce the chances of being blocked.