Selenium unable to discover open pages

131
August 03, 2021, at 03:10 AM

I am using web scraping for my project and it has worked perfectly on a windows system. Deploying it on ubuntu has worked perfectly the first time running my script but any time after that I receive the error

selenium.common.exceptions.WebDriverException: Message: unknown error: unable to discover open pages

Usually before this happens the script has no output for around a minute and 30 seconds before returning the error. Any help would be appreciated!

My code:

import selenium
from bs4 import BeautifulSoup, NavigableString
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
for i in range(5):
    URLS = []
    URLS.append('https://website.com/')
for i in range(1):
        options = webdriver.ChromeOptions()
        options.add_argument("--no-sandbox")
        options.add_argument("--headless")
        options.add_argument("start-maximized")
        options.add_argument("window-size=1900,1080")
        options.add_argument("disable-gpu")
        options.add_argument("--disable-software-rasterizer")
        options.add_argument("--disable-dev-shm-usage")
        options.add_argument('--remote-debugging-port=9222')
        options.add_argument('user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36')
        driver = webdriver.Chrome(executable_path="./drivers/chromedriver", options=options)
        driver.set_page_load_timeout(2)
        for url in URLS:
            try:
                webdriver.get(url)
                innerHTML = webdriver.page_source
                soup = BeautifulSoup(innerHTML, "html.parser")
                for i in soup.select("#ELEMENT"):
                    global ELEMENT
                    ELEMENT = i.text
                    print(ELEMENT)
                webdriver.close()
                webdriver.quit()
           
            except:
                pass
Answer 1

Not sure, but possibly you are using an outdated version of ChromeDriver.

READ ALSO
How to load Post Type fields with data from a remote API

How to load Post Type fields with data from a remote API

I need to bring the information of a Post Type from a Remote URL that show a json file

102
How to create blog app MERN with like instagram blog?

How to create blog app MERN with like instagram blog?

I learn REST api in NodejsI want to create blog app like instagram

120
How to use AJAX and Django Models to print dropdown query

How to use AJAX and Django Models to print dropdown query

I am attempting to use AJAX where the client selects a State from a dropdown list, then the counties within that state will appear in the next dropdown list

86
NewLine in the browser validation message

NewLine in the browser validation message

I have a input which is created using jquery

48