Proxy scrapping softwares

[Vector]

Not sure if this will get you thousands of proxies but i wrote a wrapper for a certain proxy scraper and i made it so you can very easily sort on geolocation. Get it on your box like so:

Code:

wget -O proxies.sh https://gist.githubusercontent.com/NullArray/35e3d894fe896ee1d7d8088a3d8175b7/raw/4756ba136456896512c26ee79f23c247d299aee0/geo-sorter.sh

If you want to use it to gather a large amount of proxies use something like the shell script i wrote for you below.

Code:

#!/bin/bash
#____   ____             __                
#\   \ /   /____   _____/  |_  ___________
# \   Y   // __ \_/ ___\   __\/  _ \_  __ \
#  \     /\  ___/\  \___|  | (  <_> )  | \/
#   \___/  \___  >\___  >__|  \____/|__|  
#              \/     \/    
##############################################

# Coloring scheme for notifications
ESC="\x1b["
RESET=$ESC"39;49;00m"
CYAN=$ESC"33;36m"
RED=$ESC"31;01m"
GREEN=$ESC"32;01m"



# Warning
function warning(){
    echo -e "\n$RED [!] $1 $RESET\n"
    }

# Green notification
function notification(){
    echo -e "\n$GREEN [+] $1 $RESET\n"
    }

# Cyan notification
function notification_b(){
    echo -e "\n$CYAN [-] $1 $RESET\n"
    }

function main(){
    notification_b "Enter your country of choice"
    read -p "Country: " country
        
    clear && notification "Running scraper on loop. Press CTRL+C to stop"
    while i=0
    do
        python3 fetch.py --country='$country' --max-latency=4 --anonymity='elite|anonymous' --output=proxies.log
        cat proxies.log >> unsorted.log && rm proxies.log
        cat unsorted.log | sort -u >> unique_proxies.txt

        notification_b "Starting delay, 15 minutes remaining"
        sleep 300 && clear
        notification_b "10 Minutes remaining"
        sleep 300 && clear
        notification_b "5 Minutes remaining"
        sleep 300 && clear
        notification "New iteration initiated."
    done
    }

if [[ -d "fetch-some-proxies" ]]; then
    cd /home/$USER/fetch-some-proxies/
    main
else
    warning "Proxy fetcher not found."
    printf "Cloning latest version from Github.\n" && sleep 1  
    
    git clone https://github.com/stamparm/fetch-some-proxies.git
    
    notification "Done." && sleep 1
    cd /home/$USER/fetch-some-proxies/
    main
fi

The reason i put in a 15 minute delay, is so that the remote resource has some time to reload a new batch. It will automatically sort out duplicates and will loop forever until you manually stop it.

Not sure if the Python script that does the heavy lifting allows you to enter an argument to get just SOCKS5 proxies, but configured as t is it will get SOCKS4 and SOCKS5. Not sure how essential the fact the proxies are SOCKS5 is to your operation.