I'm sure I'm not the only one that creates scripts to scrap sites. Feel free to share ones you've made here.Viper.py
This script is for viperteen.com. I used it to download around 200GB within a few hours. It will crawl a specified page, create a folder for the model name and dump all the images in the set folder.Requires
: Python2, BeautifulSoup4 and RequestsUsage:
python viper.py [AGENCY] [START_PAGE] [MAX_PAGE] [MODEL_INDEX]Example:
python viper.py ttl 1 6 0(Note: Get the agency name from the url.
This means the script will crawl "[url]viperteen.com/non-nude/ttl/*new-1/[/url]" up to page 6, starting at the first model on the page.(model index 0) Occasionally the script may fuck up due to server issues, in which case you can just restart it where you left off.
I'd also be happy to fulfill any other customized scrappers/site rippers requests, if for whatever reasons BID/IGH/DownthemALL don't suffice.