Project: GreySec Data Scraper Program
#11
Hey there, I've actually already made something that scrapes the site, making a new folder for each forum/subforum/thread and a different text document for each post, adding the posts author and date of creation as well as it's content. It's functional but I've got more ideas for what I might want to add to it once I get around to it.

I've also got another little bot for writing posts and I intend to add PM handling to it as well. This was all created using Python and Selenium mainly as a way for me to browse the site from my terminal because I'm just weird like that! :-)
Reply
#12
(05-13-2020, 08:07 PM)EpochRoot Wrote: Hey there, I've actually already made something that scrapes the site, making a new folder for each forum/subforum/thread and a different text document for each post, adding the posts author and date of creation as well as it's content. It's functional but I've got more ideas for what I might want to add to it once I get around to it.

I've also got another little bot for writing posts and I intend to add PM handling to it as well. This was all created using Python and Selenium mainly as a way for me to browse the site from my terminal because I'm just weird like that! :-)
 
That's great because I've kind of moved on from this anyway lol. Browsing GS from the terminal would be really cool actually. I use the command line a lot so that would be really nice.
Reply
#13
I'd be happy to share the source once I've got it properly set up and once I've squared away some other stuff, with permission from the owner of course!
Reply
#14
(05-13-2020, 10:48 PM)EpochRoot Wrote: I'd be happy to share the source once I've got it properly set up and once I've squared away some other stuff, with permission from the owner of course!

I'd be very interested in this. If you don't want to release it for everyone though... you're free to PM me about it instead Smile
Reply
#15
(05-16-2020, 10:28 PM)Insider Wrote:
(05-13-2020, 10:48 PM)EpochRoot Wrote: I'd be happy to share the source once I've got it properly set up and once I've squared away some other stuff, with permission from the owner of course!

I'd be very interested in this. If you don't want to release it for everyone though... you're free to PM me about it instead :)
 
Yeah, PM me too ;)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  GreySec Social thread. Vector 163 115,134 06-11-2020, 11:25 PM
Last Post: Insider
  How did you find GreySec? Insider 8 1,120 06-11-2020, 11:11 PM
Last Post: Insider
  GreySec Scraper Tool Release Dismal_0x8 4 1,019 06-09-2020, 12:20 AM
Last Post: Insider
  GreySec Forum Statistics (Full Stats) Dismal_0x8 7 973 06-02-2020, 11:15 AM
Last Post: Insider