質問編集履歴

2

全体的に変更しました

2020/05/02 08:48

投稿

k0gane
k0gane

スコア10

test CHANGED
@@ -1 +1 @@
1
- スクレイピングで入手できないデ
1
+ seleniumにてスクレイピングを実行したところエラーが
test CHANGED
@@ -12,7 +12,9 @@
12
12
 
13
13
  ```
14
14
 
15
- エラーメッセージ
15
+ raise exception_class(message, screen, stacktrace)
16
+
17
+ selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <div class="normal-container"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
16
18
 
17
19
  ```
18
20
 
@@ -24,11 +26,9 @@
24
26
 
25
27
  ```python
26
28
 
27
- import urllib3
29
+ from selenium import webdriver
28
30
 
29
- from bs4 import BeautifulSoup
31
+ from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
30
-
31
- import certifi
32
32
 
33
33
 
34
34
 
@@ -36,29 +36,25 @@
36
36
 
37
37
 
38
38
 
39
- sl_URL = "https://stellabms.xyz/s/sl/" + str(song_id)
39
+ while True:
40
40
 
41
- sl_http = urllib3.PoolManager(cert_reqs='CERT_REQUIRED',ca_certs=certifi.where())
41
+ sl_URL = "https://stellabms.xyz/s/sl/" + str(song_id)
42
42
 
43
- r = sl_http.request('GET', sl_URL)
43
+ driver.get(sl_URL)
44
44
 
45
- sl_soup = BeautifulSoup(r.data, "lxml")
45
+ sl = driver.find_elements_by_tag_name("div")
46
46
 
47
- sl_div = sl_soup.find("div")
47
+ sl_hrefs = [s.get_attribute("href") for s in sl]
48
48
 
49
+ for i in sl_hrefs:
50
+
49
- print(sl_div)
51
+ print(driver.get(i))
52
+
53
+
50
54
 
51
55
  ```
52
56
 
53
57
 
54
-
55
- 出力
56
-
57
- ```
58
-
59
- <div id="root"></div>
60
-
61
- ```
62
58
 
63
59
 
64
60
 
@@ -75,5 +71,3 @@
75
71
 
76
72
 
77
73
  Python==3.7.3
78
-
79
- beautifulsoup4==4.9.0

1

bs4とbeautifulsoup4を混同していました

2020/05/02 08:48

投稿

k0gane
k0gane

スコア10

test CHANGED
File without changes
test CHANGED
@@ -76,4 +76,4 @@
76
76
 
77
77
  Python==3.7.3
78
78
 
79
- bs4==0.0.1
79
+ beautifulsoup4==4.9.0