Scribd
Upload
23 views2 pages

Automating Onion Link Scraping with Python

The document outlines a Python script for automating the scraping of .onion links from the Ahmia search engine. It takes a user-defined query, formats it for the search URL, retrieves the content, and extracts unique .onion links using regular expressions. The links are then saved to a text file, and the first five links are printed to the console.

Uploaded by

joshyarit07
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
23 views2 pages

Automating Onion Link Scraping with Python

The document outlines a Python script for automating the scraping of .onion links from the Ahmia search engine. It takes a user-defined query, formats it for the search URL, retrieves the content, and extracts unique .onion links using regular expressions. The links are then saved to a text file, and the first five links are printed to the console.

Uploaded by

joshyarit07
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Onion Links Automation Using Python

import requests

import random

import re

def scrape(newdata):

yourquery = newdata

if " " in yourquery:

yourquery = [Link](" ", "+")

url = "[Link]

requests = [Link](url)

content = [Link]

regexquery = "\w\.onion"

mineddata = [Link](regexquery, content)

n = [Link](1, 9999)

filename = "sites{}.txt".format(str(n))

print("Saving to .... ", filename)


mineddata = list(set(mineddata))

for k in mineddata:

with open(filename, "a") as newfile:

k = k + "\n"

[Link](k)

print("All the files written to a text file : ", filename)

with open(filename) as input_file:

head = [next(input_file) for x in range(5)]

contents = "\n".join(map(str, head))

print(contents)

newdata= input("Enter the query to search for : ")

scrape(newdata)

You might also like