Abdulmohsen AbdulmohsenA
AbdulmohsenA
/ scrape_arabic_books.py
Created
June 26, 2024 22:15
Code to rapidly scrape Arabic books from المكتبة الشاملة
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # THIS CODE IS INTENDED TO SCRAPE ARABIC BOOKS FROM https://shamela.ws | |
| import requests | |
| from bs4 import BeautifulSoup | |
| import pandas as pd | |
| from concurrent.futures import ThreadPoolExecutor | |
| def scrape_page(book_code, page): | |
| '''Scrape a page from a book in Shamela website: https://shamela.ws''' | |
| url = f'https://shamela.ws/book/{book_code}/{page}' |