Skip to content

Instantly share code, notes, and snippets.

@edson-github

edson-github/Web scrape e-commerce site.py

Forked from chayb/Web scrape e-commerce site.py
Created February 2, 2023 14:11
Show Gist options
  • Save edson-github/22b18a7251d35007faee81a21e7d324a to your computer and use it in GitHub Desktop.
Save edson-github/22b18a7251d35007faee81a21e7d324a to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @chayb chayb revised this gist Aug 5, 2020. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion Web scrape e-commerce site.py
    View file
    Original file line number Diff line number Diff line change
    @@ -18,7 +18,7 @@
    time.sleep(3)
    productInfoList=webD.find_elements_by_class_name('a-size-mini')
    for el in productInfoList:
    if(el.text !=""):
    if(el.text !="" and el.text !="Sponsored"):
    pp2=el.find_element_by_tag_name('a')
    listOflinks.append(pp2.get_property('href'))
    try:
  2. @chayb chayb revised this gist Aug 3, 2020. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion Web scrape e-commerce site.py
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    #importing libraries*/
    #importing libraries
    import selenium
    from selenium import webdriver as wb
    from selenium.webdriver.support.ui import Select
  3. @chayb chayb created this gist Aug 3, 2020.
    30 changes: 30 additions & 0 deletions Web scrape e-commerce site.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,30 @@
    #importing libraries*/
    import selenium
    from selenium import webdriver as wb
    from selenium.webdriver.support.ui import Select
    import pandas as pd
    import time

    #Opening Chrome browser
    wbD=wb.Chrome('chromedriver.exe')

    #Opening webpage
    wbD.get('https://www.amazon.in/s?bbn=1389396031&rh=n%3A976419031%2Cn%3A%21976420031%2Cn%3A1389375031%2Cn%3A1389396031%2Cn%3A15747864031&dc&fst=as%3Aoff&qid=1596287247&rnid=1389396031&ref=lp_1389396031_nr_n_1')

    #Running loop to store the product links in a list
    listOflinks =[]
    condition =True
    while condition:
    time.sleep(3)
    productInfoList=webD.find_elements_by_class_name('a-size-mini')
    for el in productInfoList:
    if(el.text !=""):
    pp2=el.find_element_by_tag_name('a')
    listOflinks.append(pp2.get_property('href'))
    try:
    wbD.find_element_by_class_name('a-last').find_element_by_tag_name('a').get_property('href')
    wbD.find_element_by_class_name('a-last').click()
    except:
    condition=False

    len(listOflinks)