python qiitaを参考にしたbotの開発におけるlist index out of range問題

Question

参考にした記事

https://qiita.com/ogi-iii/items/cc9a7bc45f7b15001440

発生している問題・エラーメッセージ

IndexError               Traceback (most recent call last)
<ipython-input-1-5ea6d0fc191f> in <module>
     24     #choose a stock out of list
     25     stockClick=browser.find_elements_by_class_name("clickable")
---> 26     stockClick[num].find_element_by_tag_name("a").click()
     27 
     28     stockTable=browser.find_element_by_class_name("table_wrap")

IndexError: list index out of range

該当のソースコード

#ready for scraping
columnNames=[]
ETFComparisonsTable=[]
for num in range(0,48):
    browser.get("https://kabuoji3.com/stock/")
    stockSearch=browser.find_element_by_class_name("form_inputs")
    stockSearchForm=stockSearch.find_element_by_class_name("form_txt")
    stockSearchForm.send_keys("ETF")
    btnClick=browser.find_element_by_class_name("btn_submit")
    btnClick.click()

    #choose a stock out of list
    stockClick=browser.find_elements_by_class_name("clickable")
    stockClick[num].find_element_by_tag_name("a").click()

    stockTable=browser.find_element_by_class_name("table_wrap")
    stockLine=stockTable.find_elements_by_tag_name("tr")

    #price scraping with calculation
    if len(stockLine)==302:
        ETFComparisons=[]
        for i in range(2,152):
            stockETFPriceAfter=stockLine[i-1].find_elements_by_tag_name("td")
            stockETFPriceBefore=stockLine[i].find_elements_by_tag_name("td")
            ETFComparison=float(stockETFPriceAfter[6].text)-float(stockETFPriceBefore[6].text)
            ETFComparisons.append(ETFComparison)

        stockETFPriceAfter=stockLine[151].find_elements_by_tag_name("td")
        stockETFPriceBefore=stockLine[153].find_elements_by_tag_name("td")
        ETFComparison=float(stockETFPriceAfter[6].text)-float(stockETFPriceBefore[6].text)
        ETFComparisons.append(ETFComparison)

        for i in range(154,302):
            stockETFPriceAfter=stockLine[i-1].find_elements_by_tag_name("td")
            stockETFPriceBefore=stockLine[i].find_elements_by_tag_name("td")
            ETFComparison=float(stockETFPriceAfter[6].text)-float(stockETFPriceBefore[6].text)
            ETFComparisons.append(ETFComparison)

        ETFComparisonsTable.append(ETFComparisons)

        #pick up title
        stockTitleBox=browser.find_element_by_class_name("base_box_ttl")
        stockTitle=stockTitleBox.find_element_by_class_name("jp").text
        columnNames.append(stockTitle)

#making ETF table
ETFTable=pd.DataFrame(ETFComparisonsTable)
ETFTable=ETFTable.T
ETFTable.columns=columnNames
#checking ETF table
ETF.tail()

補足情報（FW/ツールのバージョンなど）

参考にしたQiitaの記事
https://qiita.com/ogi-iii/items/cc9a7bc45f7b15001440

jupyter lab

Comments

[YT0014]

https://kabuoji3.com/stock/のHTMLソースに依存しているので、変更があった場合、当然、動作しません。
確認したところ、現在、clickableクラスの要素はありませんでした。

Answer 1

いくつかエラーハンドリングが必要です。

    stockClick[num].find_element_by_tag_name("a").click()

の前に

    if num >= len(stockClick):
        continue

を追加。

        stockTable=browser.find_element_by_class_name("table_wrap")

の1行を

    try:
        stockTable=browser.find_element_by_class_name("table_wrap")
    except:
        continue

の4行に差し替え

Comments

[Ponimaru]

回答ありがとうございます。
今度はこのようなエラーが出ました。

```python
NoSuchElementException Traceback (most recent call last)
<ipython-input-1-8783f502fee9> in <module>
16 for num in range(0,48):
17 browser.get("https://kabuoji3.com/stock/")
---> 18 stockSearch=browser.find_element_by_class_name("form_inputs")
19 stockSearchForm=stockSearch.find_element_by_class_name("form_txt")
20 stockSearchForm.send_keys("ETF")

~/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py in find_element_by_class_name(self, name)
562 element = driver.find_element_by_class_name('foo')
563 """
--> 564 return self.find_element(by=By.CLASS_NAME, value=name)
565
566 def find_elements_by_class_name(self, name):

~/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py in find_element(self, by, value)
976 return self.execute(Command.FIND_ELEMENT, {
977 'using': by,
--> 978 'value': value})['value']
979
980 def find_elements(self, by=By.ID, value=None):

~/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
319 response = self.command_executor.execute(driver_command, params)
320 if response:
--> 321 self.error_handler.check_response(response)
322 response['value'] = self._unwrap_value(
323 response.get('value', None))

~/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
240 alert_text = value['alert'].get('text')
241 raise exception_class(message, screen, stacktrace, alert_text)
--> 242 raise exception_class(message, screen, stacktrace)
243
244 def _value_or_default(self, obj, key, default):

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".form_inputs"}
(Session info: chrome=83.0.4103.106)
```