質問編集履歴
5
やりたいこと、解決したい問題をより明確に
test
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
RuntimeError
|
|
1
|
+
RuntimeErrorが出て楽観的初期化を実装できない
|
test
CHANGED
|
@@ -164,6 +164,96 @@
|
|
|
164
164
|
|
|
165
165
|
```
|
|
166
166
|
|
|
167
|
+
### 発生している問題・エラーメッセージ
|
|
168
|
+
|
|
169
|
+
- Pytorchを利用して強化学習のAIを作成したい。
|
|
170
|
+
|
|
171
|
+
- そのために、楽観的初期化を実装したい。そのために、以下のソースコードを書いた。
|
|
172
|
+
|
|
173
|
+
```Python
|
|
174
|
+
|
|
175
|
+
#楽観的初期化の回数
|
|
176
|
+
|
|
177
|
+
StartInitializeTimes = 10
|
|
178
|
+
|
|
179
|
+
Need_Retain_First = True
|
|
180
|
+
|
|
181
|
+
|
|
182
|
+
|
|
183
|
+
for i in range(StartInitializeTimes):
|
|
184
|
+
|
|
185
|
+
target = torch.ones(DQNModel.Outputs,dtype=torch.float32,device=Devise)
|
|
186
|
+
|
|
187
|
+
out = Model_Target(torch.ones(DQNModel.Inputs,device=Devise))
|
|
188
|
+
|
|
189
|
+
loss = criterion(out,target)
|
|
190
|
+
|
|
191
|
+
optimizer.zero_grad()
|
|
192
|
+
|
|
193
|
+
loss.backward(retain_graph=Need_Retain_First)
|
|
194
|
+
|
|
195
|
+
Need_Retain_First = False
|
|
196
|
+
|
|
197
|
+
optimizer.step()
|
|
198
|
+
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
- しかし、以下のようなエラーが出ていて、実装できない。エラーを解決し、楽観的初期化を実装できるようにしたい。
|
|
202
|
+
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
Warning: Error detected in MmBackward. Traceback of forward call that caused the error:
|
|
206
|
+
|
|
207
|
+
File "C:\Users(ユーザー名)\Documents\MyFolder\CardBattle2\CardBattle\Agent_DQN.py", line 102, in <module>
|
|
208
|
+
|
|
209
|
+
out = Model_Target(torch.ones(DQNModel.Inputs,device=Devise))
|
|
210
|
+
|
|
211
|
+
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 550, in __call__
|
|
212
|
+
|
|
213
|
+
result = self.forward(*input, **kwargs)
|
|
214
|
+
|
|
215
|
+
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\container.py", line 100, in forward
|
|
216
|
+
|
|
217
|
+
input = module(input)
|
|
218
|
+
|
|
219
|
+
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 550, in __call__
|
|
220
|
+
|
|
221
|
+
result = self.forward(*input, **kwargs)
|
|
222
|
+
|
|
223
|
+
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\linear.py", line 87, in forward
|
|
224
|
+
|
|
225
|
+
return F.linear(input, self.weight, self.bias)
|
|
226
|
+
|
|
227
|
+
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\functional.py", line 1612, in linear
|
|
228
|
+
|
|
229
|
+
output = input.matmul(weight.t())
|
|
230
|
+
|
|
231
|
+
(print_stack at ..\torch\csrc\autograd\python_anomaly_mode.cpp:60)
|
|
232
|
+
|
|
233
|
+
Traceback (most recent call last):
|
|
234
|
+
|
|
235
|
+
File "C:\Users(ユーザー名)\Documents\MyFolder\CardBattle2\CardBattle\Agent_DQN.py", line 249, in <module>
|
|
236
|
+
|
|
237
|
+
loss.backward(retain_graph=Need_Retain)
|
|
238
|
+
|
|
239
|
+
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\tensor.py", line 198, in backward
|
|
240
|
+
|
|
241
|
+
torch.autograd.backward(self, gradient, retain_graph, create_graph)
|
|
242
|
+
|
|
243
|
+
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\autograd\__init__.py", line 100, in backward
|
|
244
|
+
|
|
245
|
+
allow_unreachable=True) # allow_unreachable flag
|
|
246
|
+
|
|
247
|
+
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [64, 5]],
|
|
248
|
+
|
|
249
|
+
which is output 0 of TBackward, is at version 21; expected version 19 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient.
|
|
250
|
+
|
|
251
|
+
The variable in question was changed in there or anywhere later. Good luck!
|
|
252
|
+
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
|
|
256
|
+
|
|
167
257
|
### 試したこと
|
|
168
258
|
|
|
169
259
|
以下のサイトを参考に、Agent_DQN.pyのoutに対して`out = out.clone()`してから`criterion()`の引数に入れるようにしたり、DQNModel.pyの`nn.LeakyReLU()`の引数に`inplace=True`を指定してみましたが、同様のエラーが発生しました。
|
4
入力ミスの修正
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
|
|
11
11
|
File "C:\Users(ユーザー名)\Documents\MyFolder\CardBattle2\CardBattle\Agent_DQN.py", line 102, in <module>
|
|
12
12
|
|
|
13
|
-
out = Model_Target(Input
|
|
13
|
+
out = Model_Target(torch.ones(DQNModel.Inputs,device=Devise))
|
|
14
14
|
|
|
15
15
|
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 550, in __call__
|
|
16
16
|
|
|
@@ -122,7 +122,7 @@
|
|
|
122
122
|
|
|
123
123
|
optimizer.zero_grad()
|
|
124
124
|
|
|
125
|
-
loss.backward(retain_graph=Need_Retain_
|
|
125
|
+
loss.backward(retain_graph=Need_Retain_First)
|
|
126
126
|
|
|
127
127
|
Need_Retain_First = False
|
|
128
128
|
|
3
入力ミスの修正
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -38,7 +38,7 @@
|
|
|
38
38
|
|
|
39
39
|
File "C:\Users(ユーザー名)\Documents\MyFolder\CardBattle2\CardBattle\Agent_DQN.py", line 249, in <module>
|
|
40
40
|
|
|
41
|
-
loss.backward(retain_graph=Need_Retain
|
|
41
|
+
loss.backward(retain_graph=Need_Retain)
|
|
42
42
|
|
|
43
43
|
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\tensor.py", line 198, in backward
|
|
44
44
|
|
|
@@ -107,6 +107,8 @@
|
|
|
107
107
|
#楽観的初期化の回数
|
|
108
108
|
|
|
109
109
|
StartInitializeTimes = 10
|
|
110
|
+
|
|
111
|
+
Need_Retain_First = True
|
|
110
112
|
|
|
111
113
|
|
|
112
114
|
|
2
URLが参照しやすくなるよう修正
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -170,7 +170,7 @@
|
|
|
170
170
|
|
|
171
171
|
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [4, 512, 16, 16]], which is output 0 of ConstantPadNdBackward, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True). · Issue #23 · NVlabs/FUNIT
|
|
172
172
|
|
|
173
|
-
URL : https://github.com/NVlabs/FUNIT/issues/23
|
|
173
|
+
URL : [https://github.com/NVlabs/FUNIT/issues/23](https://github.com/NVlabs/FUNIT/issues/23)
|
|
174
174
|
|
|
175
175
|
### 補足情報（FW/ツールのバージョンなど）
|
|
176
176
|
|
1
入力ミスの修正
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
|
|
11
11
|
File "C:\Users(ユーザー名)\Documents\MyFolder\CardBattle2\CardBattle\Agent_DQN.py", line 102, in <module>
|
|
12
12
|
|
|
13
|
-
out
|
|
13
|
+
out = Model_Target(Input_First.clone())
|
|
14
14
|
|
|
15
15
|
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 550, in __call__
|
|
16
16
|
|
|
@@ -38,7 +38,7 @@
|
|
|
38
38
|
|
|
39
39
|
File "C:\Users(ユーザー名)\Documents\MyFolder\CardBattle2\CardBattle\Agent_DQN.py", line 249, in <module>
|
|
40
40
|
|
|
41
|
-
loss
|
|
41
|
+
loss.backward(retain_graph=Need_Retain_1P)
|
|
42
42
|
|
|
43
43
|
File "C:\Users(ユーザー名)\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\tensor.py", line 198, in backward
|
|
44
44
|
|
|
@@ -118,9 +118,9 @@
|
|
|
118
118
|
|
|
119
119
|
loss = criterion(out,target)
|
|
120
120
|
|
|
121
|
-
optimizer
|
|
121
|
+
optimizer.zero_grad()
|
|
122
122
|
|
|
123
|
-
loss
|
|
123
|
+
loss.backward(retain_graph=Need_Retain_Init)
|
|
124
124
|
|
|
125
125
|
Need_Retain_First = False
|
|
126
126
|
|