pytorch 誤差逆伝播のエラー

Pythonの深層学習を行う際，誤差逆伝播の処理でエラーが生じてしまいます．
該当箇所のコードは以下の通りです．

python
1    synth_batch = iter(synth_images).next()
2    synth_batch = synth_batch.to(device)
3
4    ref_batch = RefNet(synth_batch)
5    batch_from_buffer = Buffer.get_from_buffer()
6    batch_from_buffer = torch.from_numpy(batch_from_buffer).to(device)
7
8    ref_batch[:bs//2] = batch_from_buffer
9
10    D_ref_out = DisNet(ref_batch).view(-1,2)
11    D_loss_ref = local_adv_loss(D_ref_out, fake_label(D_ref_out))
12
13    D_loss_ref.backward()

エラー文は以下のとおりです．コードの最終文に対してエラーが出ています．

python
1RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [256, 1, 35, 55]], which is output 0 of TanhBackward, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).