質問編集履歴
5
追加
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -20,7 +20,43 @@
|
|
|
20
20
|
|
|
21
21
|
＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝
|
|
22
22
|
|
|
23
|
-
【面白いidea】
|
|
23
|
+
**【面白いidea 1】**
|
|
24
|
+
|
|
25
|
+
What is Automatic Differentiation?
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
Automatic differentiation is really **just a jumped-up chain rule**.
|
|
30
|
+
|
|
31
|
+
When you implement a function on a computer, you only have a small number of primitive operations available (e.g. addition, multiplication, logarithm). Any complicated function, like log(x^2)/x.
|
|
32
|
+
|
|
33
|
+
is just a combination of these simple functions.
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
In other words, any complicated function f can be rewritten as the composition of a sequence of primitive functions fk:
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
f = f0∘f1∘f2∘…∘fn
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
Because each primitive function fk
|
|
48
|
+
|
|
49
|
+
has a simple derivative, we can use the chain rule to find df/dx pretty easily.
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
Although I've used a single-variable function f:R→R
|
|
54
|
+
|
|
55
|
+
as my example here, it's straightforward to extend this idea to multivariate functions f:Rn→Rm.
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
**【面白いidea 2】**
|
|
24
60
|
|
|
25
61
|
Automatic differentiation computes derivatives based on computational functions (which in turn are broken down into basic operations such as addition/subtraction and multipliation/division).
|
|
26
62
|
|
4
追加
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -15,3 +15,15 @@
|
|
|
15
15
|
|
|
16
16
|
|
|
17
17
|
[**例えば↑**] [**<<Automatic differentiation>>**](https://en.wikipedia.org/wiki/Automatic_differentiation)
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝
|
|
22
|
+
|
|
23
|
+
【面白いidea】
|
|
24
|
+
|
|
25
|
+
Automatic differentiation computes derivatives based on computational functions (which in turn are broken down into basic operations such as addition/subtraction and multipliation/division).
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
Since TensorFlow does differentiation based on a computation graph of operations, I'd intuitively say that it's **automatic differentiation** (I don't know of any other technique that would be appropriate here; I think the possibility that TensorFlow is **converting the computation graph into a mathematical equation** that is then **parsed** to compute the derivative of that equation is prob. out of question). The authors say "symbolic differentiation" in the TensorFlow whitepaper though -- however, I think this may be a **misnomer** similar to "Tensor" instead of "(multi-dimensional) data array" if you'd ask a mathematician.
|
3
追加
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
NNの神髄は目標関数を微分で極値に到達させることだと思います。
|
|
1
|
+
NNの神髄は目標関数を微分で極値に到達させる(これでback propagationの形でweightの最適化を行う)ことだと思います。
|
|
2
2
|
|
|
3
3
|
その微分の実現法として『自動微分』法が良く紹介されています。
|
|
4
4
|
|
2
訂正
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
その微分の実現法として『自動微分』法が良く紹介されています。
|
|
4
4
|
|
|
5
|
-
しかし、『自動微分』の方法を調べても[**例えば↓**]、関数が数式(解析式,symbolic format)である例を説明されたばかりですが(**これで"forward and reverse accumulation"を論ずる**)、現実の関数は皆
|
|
5
|
+
しかし、『自動微分』の方法を調べても[**例えば↓**]、関数が数式(解析式,symbolic format)である例を説明されたばかりですが(**これで"forward and reverse accumulation"を論ずる**)、現実の関数は皆programming言語で定義されるわけですから、内容や形式が様々で、if文やloop文さえ入れている事もあり、解析的な数式表現とは全然異なりますね！
|
|
6
6
|
|
|
7
7
|
|
|
8
8
|
|
1
訂正
test
CHANGED
|
File without changes
|
test
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
その微分の実現法として『自動微分』法が良く紹介されています。
|
|
4
4
|
|
|
5
|
-
しかし、『自動微分』の方法を調べても[**例えば↓**]、関数が数式(解析式,symbolic format)である例を説明されたばかりですが(**これで"forward and reverse accumulation"を論ずる**)、現実の関数は皆プログラム言語で定義されるわけですから、プログラ
|
|
5
|
+
しかし、『自動微分』の方法を調べても[**例えば↓**]、関数が数式(解析式,symbolic format)である例を説明されたばかりですが(**これで"forward and reverse accumulation"を論ずる**)、現実の関数は皆プログラム言語で定義されるわけですから、プログラミング言語で定義している関数の内容や形式は様々で、if文やloop文さえ入れている事もあり、解析的な数式のような表現と全然異なりますね！
|
|
6
6
|
|
|
7
7
|
|
|
8
8
|
|