Q&A
このままではコードが読めないので、質問を編集し、</>(コードの挿入)ボタンを押し、出てくる’’’の枠の中にコードを貼り付けてください
前提
CSVデータを読み込んで、データ解析を試みようとしています。
実現したいこと
読み込んだデータが、カラム（行名）と認識されてしまいデータとして認識されない。
数値データをデータとして認識させたい。
発生している問題・エラーメッセージ
Signal Name \
2022/08/01 14:00 2022 8 1 5 0 0 6.79 217 81 43.32 12262 ** ---- ---- ---- ---- ---- ---- ---- ----
2022/08/01 14:01 ## ## ## ## ## ## 6.79 214 81 43.35 12487 ** ---- ---- ---- ---- ---- ---- ---- ----
2022/08/01 14:02 2022 8 1 5 2 0 6.79 214 81 43.32 12337 ** ---- ---- ---- ---- ---- ---- ---- ----
2022/08/01 14:03 2022 8 1 5 3 0 6.79 218 80 43.35 12412 ** ---- ---- ---- ---- ---- ---- ---- ----
2022/08/01 14:04 2022 8 1 5 4 0 6.79 213 80 43.35 12350 ** ---- ---- ---- ---- ---- ---- ---- ----
... ...
2022/08/31 15:55 2022 8 31 6 55 0 6.8 134 131 42.92 7437 -38 ---- ---- ---- ---- ---- ---- ---- ----
2022/08/31 15:56 2022 8 31 6 56 0 6.8 134 131 42.92 7362 -48 ---- ---- ---- ---- ---- ---- ---- ----
2022/08/31 15:57 2022 8 31 6 57 0 6.8 131 131 42.92 7437 -55 ---- ---- ---- ---- ---- ---- ---- ----
2022/08/31 15:58 2022 8 31 6 58 0 6.8 128 131 42.92 7437 -41 ---- ---- ---- ---- ---- ---- ---- ----
2022/08/31 15:59 2022 8 31 6 59 0 6.8 127 131 42.92 7425 -48 ---- ---- ---- ---- ---- ---- ---- ----
該当のソースコード
python
1#必要ライブラリーのインストール 2import numpy as np 3import scipy as sp 4import pandas as pd 5import pandas as df 6import os 7from pandas import Series, DataFrame 8from sklearn import datasets 9 10#可視化ライブラリー 11import matplotlib.pyplot as plt 12import matplotlib as mpl 13import seaborn as sns 14sns.set() 15%matplotlib inline 16from pylab import rcParams 17 18#少数第３位まで表示 19%precision 3 20 21#CSVデータの読み込む(UTF-8文字コードになっているので、日本語使えるASCIIにエンコードする。） 22#ここでは、最初の余計な３行を無視して読み込んでいる。 23An_Data = pd.read_csv('target.csv',encoding="shift-jis",skiprows=3) 24 25#最初の５行確認用コード（必要時に#を外す。） 26#An_Data.head() 27 28#最終行の確認コード（必要時に#を外す。） 29#An_Data.tail(1) 30 31#最終行の削除コード 32An_Data = An_Data[:-1]
試したこと
1.shift-jisをutf-8に変更したが、エラーとなった。
2.一番左の行（カラム名）を削除して、新しくカラム名を付与しようとしたが、
データ部分が削除され、カラムが消えない。
補足情報（FW/ツールのバージョンなど）
以下、読み込んでいるCSVデータの頭部分を記載します。
なお、改行部分を分かりやすく表記するため、■をいれてあります。
"データ単位","分単位"■
"開始日時","2022/08/01 14:00"■
"終了日時","2022/08/31 16:00"■
"Signal Name","LatestDataDate(YEAR)","LatestDataDate(MONTH)","LatestDataDate(DAY)","LatestDataDate(HOUR)","LatestDataDate(MIN)","LatestDataDate(SEC)","pH","電気信号","指標１","水温","指標２","指標３"■
"2022/08/01 14:00","2022","8","1","5","0","0","6.79","217","81","43.32","12262","","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"■
"2022/08/01 14:01","##","##","##","##","##","##","6.79","214","81","43.35","12487","","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"■
"2022/08/01 14:02","2022","8","1","5","2","0","6.79","214","81","43.32","12337","","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"■
"2022/08/01 14:03","2022","8","1","5","3","0","6.79","218","80","43.35","12412","","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"■
"2022/08/01 14:04","2022","8","1","5","4","0","6.79","213","80","43.35","12350","","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"■
"2022/08/01 14:05","2022","8","1","5","5","0","6.79","209","76","43.35","11325","","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"■
ご指摘ありがとうございます。 対応できたと思います。
csvもエラーになったあたりまでくらいは **ファイルの内容を** 正確に引用しませんか。
ご指摘ありがとうございます。おっしゃるとおりです。
ここでよいかわからないのですが、取り急ぎ貼り付けます。
"Signal Name","LatestDataDate(YEAR)","LatestDataDate(MONTH)","LatestDataDate(DAY)","LatestDataDate(HOUR)","LatestDataDate(MIN)","LatestDataDate(SEC)","pH","電気信号","指標１","水温","指標２","指標３"
"2022/08/01 14:00","2022","8","1","5","0","0","6.79","217","81","43.32","12262","**","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"
"2022/08/01 14:01","##","##","##","##","##","##","6.79","214","81","43.35","12487","**","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"
"2022/08/01 14:02","2022","8","1","5","2","0","6.79","214","81","43.32","12337","**","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"
"2022/08/01 14:03","2022","8","1","5","3","0","6.79","218","80","43.35","12412","**","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"
"2022/08/01 14:04","2022","8","1","5","4","0","6.79","213","80","43.35","12350","**","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"
"2022/08/01 14:05","2022","8","1","5","5","0","6.79","209","76","43.35","11325","**","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----","----"
ここには情報を載せないでください。質問を編集して普通に読めるように書きましょう。
（例えば、↑だと改行が本当にはどこに入っているのか判断できません）
わかりました！ ありがとうございます。
回答2件
0
ベストアンサー
以下のように、 ' と " を使い分けて、セルごと除去することで、不要部分である"----"のセルを削除することができた。
これにより、行列数のあった構造化データとなり、問題なくpandasで表として読みこむことができた。
python3
1fn = 'Target.csv' 2 3with open(fn, "r") as f: 4 s = f.read() 5 s = s.replace(',"----"','') 6 s = s.replace(',"0"','') 7 8with open(fn, "w") as f: 9 f.write(s)
投稿2022/09/14 00:33
退会済みユーザー
総合スコア0
0
投稿2022/09/06 02:48
総合スコア87774
ご回答ありがとうございます。
しかし、悔しいのですが、以下のエラーが返ってきます。
</>
An_Data = pd.read_csv('target.csv',encoding="shift-jis",skiprows=3, header = None)
#最初の５行確認用コード（必要時に#を外す。）
An_Data.head()
#最終行の確認コード（必要時に#を外す。）
An_Data.tail(1)
#最終行の削除コード
An_Data = An_Data[:-1]
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
エラーコード
ParserError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_14864/3403653594.py in <module>
----> 1 An_Data = pd.read_csv('target.csv',encoding="shift-jis",skiprows=3, header = None)
2
3 #最初の５行確認用コード（必要時に#を外す。）
4 An_Data.head()
5
~\Anaconda3\lib\site-packages\pandas\util\_decorators.py in wrapper(*args, **kwargs)
309 stacklevel=stacklevel,
310 )
--> 311 return func(*args, **kwargs)
312
313 return wrapper
~\Anaconda3\lib\site-packages\pandas\io\parsers\readers.py in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, error_bad_lines, warn_bad_lines, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)
584 kwds.update(kwds_defaults)
585
--> 586 return _read(filepath_or_buffer, kwds)
587
588
~\Anaconda3\lib\site-packages\pandas\io\parsers\readers.py in _read(filepath_or_buffer, kwds)
486
487 with parser:
--> 488 return parser.read(nrows)
489
490
~\Anaconda3\lib\site-packages\pandas\io\parsers\readers.py in read(self, nrows)
1045 def read(self, nrows=None):
1046 nrows = validate_integer("nrows", nrows)
-> 1047 index, columns, col_dict = self._engine.read(nrows)
1048
1049 if index is None:
~\Anaconda3\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py in read(self, nrows)
221 try:
222 if self.low_memory:
--> 223 chunks = self._reader.read_low_memory(nrows)
224 # destructive to chunks
225 data = _concatenate_chunks(chunks)
~\Anaconda3\lib\site-packages\pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader.read_low_memory()
~\Anaconda3\lib\site-packages\pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader._read_rows()
~\Anaconda3\lib\site-packages\pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows()
~\Anaconda3\lib\site-packages\pandas\_libs\parsers.pyx in pandas._libs.parsers.raise_parser_error()
ParserError: Error tokenizing data. C error: Expected 13 fields in line 5, saw 33
> Expected 13 fields in line 5, saw 33
と表示されていますので、5行目が33列になっています。
コメントありがとうございます。
最初に以下の３行をスキップして読み込んでいます。
"データ単位","分単位"
"開始日時","2022/08/01 14:00"
"終了日時","2022/08/31 16:00"
その後、列カラムがあり、３３列のデータがあります。
おそらく、カラム列数とデータ列数が合っていないのだと思います。
この要因は、行カラム数がまとめて１つとして認識されているために起こる現象と考えております。
しかし、解決方法が見いだせず。。。
あなたの回答
tips
プレビュー
