トップに関する質問 Python スクレイピング　株価取得 Beatiful Soup

編集履歴

質問編集履歴

更新

2020/09/13 12:43

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -311,3 +311,85 @@
 ValueError: No objects to concatenate
 ```
+```python
+以下が2framesを開いたエラー表記です。
+2424 ブラス
+2018
+No data
+2019
+No data
+---------------------------------------------------------------------------
+ValueError                                Traceback (most recent call last)
+<ipython-input-68-b8a81eea83a2> in <module>()
+     45     print(k,v)
+     46     dfs = get_dfs(k)
+---> 47     data = concatenate(dfs)
+     48     data.to_csv('{}-{}.csv'.format(k,v))
+2 frames
+<ipython-input-68-b8a81eea83a2> in concatenate(dfs)
+     29
+     30 def concatenate(dfs):
+---> 31     data = pd.concat(dfs,axis=0)
+     32     data = data.reset_index(drop=True)
+     33     col = ['始値','高値','安値','終値','出来高','終値調整']
+/usr/local/lib/python3.6/dist-packages/pandas/core/reshape/concat.py in concat(objs, axis, join, ignore_index, keys, levels, names, verify_integrity, sort, copy)
+    279         verify_integrity=verify_integrity,
+    280         copy=copy,
+--> 281         sort=sort,
+    282     )
+    283
+/usr/local/lib/python3.6/dist-packages/pandas/core/reshape/concat.py in __init__(self, objs, axis, join, keys, levels, names, ignore_index, verify_integrity, copy, sort)
+    327
+    328         if len(objs) == 0:
+--> 329             raise ValueError("No objects to concatenate")
+    330
+    331         if keys is None:
+ValueError: No objects to concatenate
+```

エラーを追加しました。

2020/09/13 12:42

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -253,3 +253,61 @@
 NameError: name 'code_list' is not defined
 ```
+```python
+以下がエラーです。次は上のこちらの部分に赤線がひいて
+ありました。→data = pd.concat(dfs,axis=0)
+2424 ブラス
+2019
+No data
+2020
+No data
+---------------------------------------------------------------------------
+ValueError                                Traceback (most recent call last)
+<ipython-input-67-3b125c0fc7ed> in <module>()
+     45     print(k,v)
+     46     dfs = get_dfs(k)
+---> 47     data = concatenate(dfs)
+     48     data.to_csv('{}-{}.csv'.format(k,v))
+2 frames
+/usr/local/lib/python3.6/dist-packages/pandas/core/reshape/concat.py in __init__(self, objs, axis, join, keys, levels, names, ignore_index, verify_integrity, copy, sort)
+    327
+    328         if len(objs) == 0:
+--> 329             raise ValueError("No objects to concatenate")
+    330
+    331         if keys is None:
+ValueError: No objects to concatenate
+```

更新

2020/09/13 12:22

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -221,3 +221,35 @@
 ```
+```python
+以下がエラーです。
+エラーが出た際、
+上のfor i in range(len(code_list)):のコードの部分に赤線が引かれていました。
+NameError                                 Traceback (most recent call last)
+<ipython-input-55-b9a24c2bba32> in <module>()
+     40
+     41 #複数のデータフレームをcsvで保存
+---> 42 for i in range(len(code_list)):
+     43     k = code_list.loc[i,'code']
+     44     v = code_list.loc[i,'name']
+NameError: name 'code_list' is not defined
+```

更新

2020/09/13 11:54

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -52,7 +52,7 @@
-csvfile = open('/content/drive/My Drive/google cora/codelist.csv')
+csvfile = open('/content/drive/My Drive/google cora/code_list.csv')

更新

2020/09/13 11:38

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -140,7 +140,7 @@
 #作成したコードリストを読み込む
-codelist = pd.read_csv('codelist.csv')
+code_list = pd.read_csv('code_list.csv')

更新

2020/09/13 11:36

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -140,7 +140,7 @@
 #作成したコードリストを読み込む
-code_list = pd.read_csv('code_list.csv')
+codelist = pd.read_csv('codelist.csv')

更新

2020/09/13 11:34

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -164,9 +164,11 @@
+```python
-```以下がエラーとして出てきました。
+以下がエラーとして出てきました。

更新

2020/09/13 11:31

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -160,11 +160,15 @@
     data.to_csv('{}-{}.csv'.format(k,v))
+```
-以下がエラーとして出てきました。
+```以下がエラーとして出てきました。
 ---------------------------------------------------------------------------

更新

2020/09/13 11:30

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -30,6 +30,10 @@
 ```python
@@ -207,3 +211,7 @@
 FileNotFoundError: [Errno 2] File codelist.csv does not exist: 'codelist.csv'
+```

現在の状況に更新しました

2020/09/13 11:27

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -48,175 +48,133 @@
-csvfile = open('/content/drive/My Drive/google cora/code_list.csv')
+csvfile = open('/content/drive/My Drive/google cora/codelist.csv')
 reader = csv.DictReader(csvfile)
 for row in reader:
- print(row)
+  print(row)
+from bs4 import BeautifulSoup
+import pandas as pd
+import requests
+from datetime import datetime
+def get_dfs(stock_number):
+    dfs = []
+    year = [2018,2019] #2017〜2019年までの株価データを取得
+    for y in year:
+        try:
+            print(y)
+            url = 'https://kabuoji3.com/stock/{}/{}/'.format(stock_number,y)
+            soup = BeautifulSoup(requests.get(url).content,'html.parser')
+            tag_tr = soup.find_all('tr')
+            head = [h.text for h in tag_tr[0].find_all('th')]
+            data = []
+            for i in range(1,len(tag_tr)):
+                data.append([d.text for d in tag_tr[i].find_all('td')])
+            df = pd.DataFrame(data, columns = head)
+            col = ['始値','高値','安値','終値','出来高','終値調整']
+            for c in col:
+                df[c] = df[c].astype(float)
+            df['日付'] = [datetime.strptime(i,'%Y-%m-%d') for i in df['日付']]
+            dfs.append(df)
+        except IndexError:
+            print('No data')
+    return dfs
+def concatenate(dfs):
+    data = pd.concat(dfs,axis=0)
+    data = data.reset_index(drop=True)
+    col = ['始値','高値','安値','終値','出来高','終値調整']
+    for c in col:
+        data[c] = data[c].astype(float)
+    return data
+#作成したコードリストを読み込む
+code_list = pd.read_csv('code_list.csv')
+#複数のデータフレームをcsvで保存
+for i in range(len(code_list)):
+    k = code_list.loc[i,'code']
+    v = code_list.loc[i,'name']
+    print(k,v)
+    dfs = get_dfs(k)
+    data = concatenate(dfs)
+    data.to_csv('{}-{}.csv'.format(k,v))
-以下が出てきたエラーです。
+以下がエラーとして出てきました。
 ---------------------------------------------------------------------------
-UnicodeDecodeError                        Traceback (most recent call last)
-<ipython-input-6-474e9fa87ab0> in <module>()
-      4 reader = csv.DictReader(csvfile)
-      5
-----> 6 for row in reader:
-      7  print(row)
-2 frames
-/usr/lib/python3.6/codecs.py in decode(self, input, final)
-    319         # decode input (taking the buffer into account)
-    320         data = self.buffer + input
---> 321         (result, consumed) = self._buffer_decode(data, self.errors, final)
-    322         # keep undecoded input until the next call
-    323         self.buffer = data[consumed:]
-UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 16: invalid start byte
-＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿
-以上までがエラー
-from bs4 import BeautifulSoup
-import pandas as pd
-import requests
-from datetime import datetime
-def get_dfs(stock_number):
-    dfs = []
-    year = [2018,2019] #2017〜2019年までの株価データを取得
-    for y in year:
-        try:
-            print(y)
-            url = 'https://kabuoji3.com/stock/{}/{}/'.format(stock_number,y)
-            soup = BeautifulSoup(requests.get(url).content,'html.parser')
-            tag_tr = soup.find_all('tr')
-            head = [h.text for h in tag_tr[0].find_all('th')]
-            data = []
-            for i in range(1,len(tag_tr)):
-                data.append([d.text for d in tag_tr[i].find_all('td')])
-            df = pd.DataFrame(data, columns = head)
-            col = ['始値','高値','安値','終値','出来高','終値調整']
-            for c in col:
-                df[c] = df[c].astype(float)
-            df['日付'] = [datetime.strptime(i,'%Y-%m-%d') for i in df['日付']]
-            dfs.append(df)
-        except IndexError:
-            print('No data')
-    return dfs
-def concatenate(dfs):
-    data = pd.concat(dfs,axis=0)
-    data = data.reset_index(drop=True)
-    col = ['始値','高値','安値','終値','出来高','終値調整']
-    for c in col:
-        data[c] = data[c].astype(float)
-    return data
-#作成したコードリストを読み込む
-code_list = pd.read_csv('code_list.csv')
-#複数のデータフレームをcsvで保存
-for i in range(len(code_list)):
-    k = code_list.loc[i,'code']
-    v = code_list.loc[i,'name']
-    print(k,v)
-    dfs = get_dfs(k)
-    data = concatenate(dfs)
-    data.to_csv('{}-{}.csv'.format(k,v))
-以下がエラーとして出てきました。
 ---------------------------------------------------------------------------
 FileNotFoundError                         Traceback (most recent call last)
-<ipython-input-2-75c0586c9f00> in <module>()
+<ipython-input-46-64a85a5b6de8> in <module>()
      37
      38 #作成したコードリストを読み込む
----> 39 code_list = pd.read_csv('code_list.csv')
+---> 39 codelist = pd.read_csv('codelist.csv')
      40
@@ -248,6 +206,4 @@
-FileNotFoundError: [Errno 2] File code_list.csv does not exist: 'code_list.csv'
+FileNotFoundError: [Errno 2] File codelist.csv does not exist: 'codelist.csv'
-```

画像の追加

2020/09/13 11:24

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -24,6 +24,10 @@
+![![イメージ説明](78084e398c508fc52913734b2396b51f.png)](0aa2c6503b87b174cf8bd61cefa8bddf.png)
 ```python

csvを読み込んだ際のエラーを追加しました

2020/09/13 10:22

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -28,6 +28,78 @@
 ```python
+from google.colab import drive
+drive.mount('/content/drive')
+Mounted at /content/drive
+import csv
+csvfile = open('/content/drive/My Drive/google cora/code_list.csv')
+reader = csv.DictReader(csvfile)
+for row in reader:
+ print(row)
+以下が出てきたエラーです。
+---------------------------------------------------------------------------
+UnicodeDecodeError                        Traceback (most recent call last)
+<ipython-input-6-474e9fa87ab0> in <module>()
+      4 reader = csv.DictReader(csvfile)
+      5
+----> 6 for row in reader:
+      7  print(row)
+2 frames
+/usr/lib/python3.6/codecs.py in decode(self, input, final)
+    319         # decode input (taking the buffer into account)
+    320         data = self.buffer + input
+--> 321         (result, consumed) = self._buffer_decode(data, self.errors, final)
+    322         # keep undecoded input until the next call
+    323         self.buffer = data[consumed:]
+UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 16: invalid start byte
+＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿
+以上までがエラー
 from bs4 import BeautifulSoup
 import pandas as pd

マークダウン記法にしました。

2020/09/13 10:15

投稿

dfmasap

スコア3

test CHANGED Viewed

File without changes

test CHANGED Viewed

@@ -21,6 +21,12 @@
 何も分からず恐縮ですが、ご回答いただけたら幸いです。
+```python
 from bs4 import BeautifulSoup
@@ -167,3 +173,5 @@
 FileNotFoundError: [Errno 2] File code_list.csv does not exist: 'code_list.csv'
+```