NNの神髄は目標関数を微分で極値に到達させる(これでback propagationの形でweightの最適化を行う)ことだと思います。
その微分の実現法として『自動微分』法が良く紹介されています。
しかし、『自動微分』の方法を調べても[例えば↓]、関数が数式(解析式,symbolic format)である例を説明されたばかりですが(これで"forward and reverse accumulation"を論ずる)、現実の関数は皆programming言語で定義されるわけですから、内容や形式が様々で、if文やloop文さえ入れている事もあり、解析的な数式表現とは全然異なりますね!
結局、**どうやって、『自動微分』が実装されたの?**という質問がずーと分かっていません。
この辺明るい方是非ご教授お願い申し上げます。
[例えば↑] <<Automatic differentiation>>
===================
【面白いidea 1】
What is Automatic Differentiation?
Automatic differentiation is really just a jumped-up chain rule.
When you implement a function on a computer, you only have a small number of primitive operations available (e.g. addition, multiplication, logarithm). Any complicated function, like log(x^2)/x.
is just a combination of these simple functions.
In other words, any complicated function f can be rewritten as the composition of a sequence of primitive functions fk:
f = f0∘f1∘f2∘…∘fn
Because each primitive function fk
has a simple derivative, we can use the chain rule to find df/dx pretty easily.
Although I've used a single-variable function f:R→R
as my example here, it's straightforward to extend this idea to multivariate functions f:Rn→Rm.
【面白いidea 2】
Automatic differentiation computes derivatives based on computational functions (which in turn are broken down into basic operations such as addition/subtraction and multipliation/division).
Since TensorFlow does differentiation based on a computation graph of operations, I'd intuitively say that it's automatic differentiation (I don't know of any other technique that would be appropriate here; I think the possibility that TensorFlow is converting the computation graph into a mathematical equation that is then parsed to compute the derivative of that equation is prob. out of question). The authors say "symbolic differentiation" in the TensorFlow whitepaper though -- however, I think this may be a misnomer similar to "Tensor" instead of "(multi-dimensional) data array" if you'd ask a mathematician.
回答1件
あなたの回答
tips
プレビュー
バッドをするには、ログインかつ
こちらの条件を満たす必要があります。
2018/08/18 05:12 編集
2018/08/18 09:54 編集
2018/08/18 13:53
2018/08/19 01:39 編集
2018/08/19 03:58