モデルが学習していないと思われるが解決策はありますでしょうか

Question

前提・実現したいこと

Transformersで感情を識別するモデルを作りたい

発生している問題・エラーメッセージ

ロスが下がっていない、実験結果も与えたテキストによらず似たようなアウトプットが出る。
恐らく、学習していないのではないかと思われる。

Epoch	Training Loss	Validation Loss	Runtime	Samples Per Second
1	No log	0.622909	0.594900	3.362000
2	No log	0.622928	0.631200	3.169000
3	No log	0.622966	0.610100	3.278000
4	No log	0.623018	0.590800	3.385000
5	No log	0.623088	0.625500	3.198000
6	No log	0.623177	0.614500	3.255000
7	No log	0.623285	0.604700	3.308000
8	No log	0.623407	0.599800	3.335000
9	No log	0.623546	0.609700	3.280000
10	No log	0.623706	0.604200	3.310000```

該当のソースコード

!pip install transformers[ja]==4.3.3 torch==1.8.1 sentencepiece==0.1.91

from google.colab import drive
import pandas as pd

drive.mount('/content/drive')

training_data = pd.read_csv('/content/drive/MyDrive/Texts/negaposi.csv')
training_data.head()

print(len(training_data["query"].unique()))
training_data[["title", "label"]].groupby("label").count()


from sklearn.model_selection import train_test_split
train_queries, val_queries, train_docs, val_docs, train_labels, val_labels = train_test_split(
    training_data["query"].tolist(),
    training_data["title"].tolist(),
    training_data["label"].tolist(),
    test_size=.2
)

from transformers import BertJapaneseTokenizer, BertForMaskedLM, pipeline

model_name = 'cl-tohoku/bert-base-japanese-whole-word-masking'
tokenizer = BertJapaneseTokenizer.from_pretrained(model_name)

train_encodings = tokenizer(train_queries, train_docs, truncation=True, padding='max_length', max_length=128)
val_encodings = tokenizer(val_queries, val_docs, truncation=True, padding='max_length', max_length=128)

import torch

class NPDataset(torch.utils.data.Dataset):
    def __init__(self, encodings, labels):
        self.encodings = encodings
        self.labels = labels

    def __getitem__(self, idx):
        item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
        item['labels'] = torch.tensor(self.labels[idx])
        return item

    def __len__(self):
        return len(self.labels)

train_dataset = NPDataset(train_encodings, train_labels)
val_dataset = NPDataset(val_encodings, val_labels)

model_name = "cl-tohoku/bert-base-japanese-whole-word-masking"
from transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained(model_name)

for param in model.base_model.parameters():
    param.requires_grad = False

from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir='/content/drive/MyDrive/Models/',          # output directory
    evaluation_strategy="epoch",     # Evaluation is done at the end of each epoch.
    num_train_epochs=10,              # total number of training epochs
    per_device_train_batch_size=16,  # batch size per device during training
    per_device_eval_batch_size=64,   # batch size for evaluation
    warmup_steps=500,                # number of warmup steps for learning rate scheduler
    weight_decay=0.01,               # strength of weight decay
    save_total_limit=1,              # limit the total amount of checkpoints. Deletes the older checkpoints.
)


trainer = Trainer(
    model=model,                         # the instantiated ???? Transformers model to be trained
    args=training_args,                  # training arguments, defined above
    train_dataset=train_dataset,         # training dataset
    eval_dataset=val_dataset             # evaluation dataset
)

trainer.train()

# 実験
nlp = pipeline("sentiment-analysis",model=model,tokenizer=tokenizer)
nlp("楽しかった")

試したこと

nlp("楽しかった")
nlp("悲しかった")

補足情報（FW/ツールのバージョンなど）

ここにより詳細な情報を記載してください。