Project: GreySec Data Scraper Program
#11
Hey there, I've actually already made something that scrapes the site, making a new folder for each forum/subforum/thread and a different text document for each post, adding the posts author and date of creation as well as it's content. It's functional but I've got more ideas for what I might want to add to it once I get around to it.

I've also got another little bot for writing posts and I intend to add PM handling to it as well. This was all created using Python and Selenium mainly as a way for me to browse the site from my terminal because I'm just weird like that! :-)
Reply
#12
(05-13-2020, 08:07 PM)EpochRoot Wrote: Hey there, I've actually already made something that scrapes the site, making a new folder for each forum/subforum/thread and a different text document for each post, adding the posts author and date of creation as well as it's content. It's functional but I've got more ideas for what I might want to add to it once I get around to it.

I've also got another little bot for writing posts and I intend to add PM handling to it as well. This was all created using Python and Selenium mainly as a way for me to browse the site from my terminal because I'm just weird like that! :-)
 
That's great because I've kind of moved on from this anyway lol. Browsing GS from the terminal would be really cool actually. I use the command line a lot so that would be really nice.
Reply
#13
I'd be happy to share the source once I've got it properly set up and once I've squared away some other stuff, with permission from the owner of course!
Reply
#14
(05-13-2020, 10:48 PM)EpochRoot Wrote: I'd be happy to share the source once I've got it properly set up and once I've squared away some other stuff, with permission from the owner of course!

I'd be very interested in this. If you don't want to release it for everyone though... you're free to PM me about it instead Smile
Reply
#15
(05-16-2020, 10:28 PM)Insider Wrote:
(05-13-2020, 10:48 PM)EpochRoot Wrote: I'd be happy to share the source once I've got it properly set up and once I've squared away some other stuff, with permission from the owner of course!

I'd be very interested in this. If you don't want to release it for everyone though... you're free to PM me about it instead :)
 
Yeah, PM me too ;)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  GreySec Social thread. Vector 206 369,553 07-04-2021, 03:29 PM
Last Post: Insider
  UK members make sure to opt out of this NHS data grab XUnit 4 4,774 06-25-2021, 12:32 PM
Last Post: XUnit
  GreySec Communication Channels (IRC.. etc) Insider 18 50,198 03-27-2021, 08:12 AM
Last Post: KattStof
  Does anyone know where to find the SHEIN data dump? InfinityDark 0 4,194 03-11-2021, 04:10 PM
Last Post: InfinityDark