teratail
回答率
85.37%
teratail 10th-anniversary
teratail 10th-anniversary
質問するログイン新規登録
トップに関する質問リジューム機能を使ったバルクインサート失敗

編集履歴

質問編集履歴

3

質問内容の更新

2018/12/05 02:29

投稿

raccoondog
raccoondog

スコア79

test CHANGED
File without changes
test CHANGED
@@ -181,3 +181,161 @@
181
181
 
182
182
 
183
183
  ```
184
+
185
+
186
+
187
+ 更新
188
+
189
+ ```ここに言語を入力
190
+
191
+ ※config_bq_incremental.yml
192
+
193
+ in:
194
+
195
+ type: oracle
196
+
197
+ path_prefix: null
198
+
199
+ driver_path: C:\Users\yazaki\Desktop\embulkDB\product\12.2.0\dbhome_1\sqldeveloper\jdbc\lib\ojdbc7.jar
200
+
201
+ host: A0430-02396
202
+
203
+ user: system
204
+
205
+ password: yazaki
206
+
207
+ database: ORCL
208
+
209
+ table: INPUT_TEST
210
+
211
+ incremental: true
212
+
213
+ incremental_columns: [id]
214
+
215
+ column_options:
216
+
217
+ ID: {type: string}
218
+
219
+ NUM: {type: string}
220
+
221
+ out: {type: bigquery, auth_method: json_key, json_keyfile: 'C:\Users\yazaki\Desktop\embulk\My
222
+
223
+ First Project-6041ffecfba7.json', project: fluid-emissary-216806, dataset: embulk_test,
224
+
225
+ auto_create_table: true, table: users, read_timeout_sec: 360000, open_timeout_sec: 360000,
226
+
227
+ send_timeout_sec: 360000, mode: append}
228
+
229
+
230
+
231
+
232
+
233
+ ※プレビュー
234
+
235
+
236
+
237
+ C:\Users\yazaki\Desktop\embulk>embulk-0.9.8.bat preview config_bq_incremental.yml
238
+
239
+ 2018-12-05 11:27:22.618 +0900: Embulk v0.9.8
240
+
241
+ 2018-12-05 11:27:23.304 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
242
+
243
+ 2018-12-05 11:27:25.457 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\yazaki.embulk\lib\gems"
244
+
245
+ 2018-12-05 11:27:26.190 +0900 [INFO] (main): Started Embulk v0.9.8
246
+
247
+ 2018-12-05 11:27:26.268 +0900 [INFO] (0001:preview): Loaded plugin embulk-input-oracle (0.9.3)
248
+
249
+ 2018-12-05 11:27:26.299 +0900 [INFO] (0001:preview): Connecting to jdbc:oracle:thin:@A0430-02396:1521:ORCL options {oracle.jdbc.ReadTimeout=1800000, user=system, password=***, oracle.net.CONNECT_TIMEOUT=300000}
250
+
251
+ 2018-12-05 11:27:26.674 +0900 [INFO] (0001:preview): Using JDBC Driver 12.1.0.2.0
252
+
253
+ 2018-12-05 11:27:26.783 +0900 [INFO] (0001:preview): Connecting to jdbc:oracle:thin:@A0430-02396:1521:ORCL options {oracle.jdbc.ReadTimeout=1800000, user=system, password=***, oracle.net.CONNECT_TIMEOUT=300000}
254
+
255
+ 2018-12-05 11:27:26.845 +0900 [INFO] (0001:preview): SQL: SELECT * FROM "INPUT_TEST" ORDER BY "ID"
256
+
257
+ 2018-12-05 11:27:26.861 +0900 [INFO] (0001:preview): > 0.02 seconds
258
+
259
+ +-----------+------------+------------+---------------+-------------------------+-------------------------+-----------------------------+-----------------------------+
260
+
261
+ | ID:string | NUM:string | STR:string | VARSTR:string | DT:timestamp | TIME0:timestamp | TIME6:timestamp | TIME9:timestamp |
262
+
263
+ +-----------+------------+------------+---------------+-------------------------+-------------------------+-----------------------------+-----------------------------+
264
+
265
+ | 179510 | 81596323 | OYYSHALN | PMTVRDKR | 2018-12-04 08:57:26 UTC | 2018-12-04 08:57:27 UTC | 2018-12-04 08:57:26.922 UTC | 2018-12-04 08:57:26.922 UTC |
266
+
267
+ | 179511 | 45967077 | EDMHMKSA | HCQMWTDO | 2018-12-04 08:57:26 UTC | 2018-12-04 08:57:27 UTC | 2018-12-04 08:57:26.969 UTC | 2018-12-04 08:57:26.969 UTC |
268
+
269
+ | 179512 | 45306398 | ZTGYAJZG | JGMEIMTI | 2018-12-04 08:57:26 UTC | 2018-12-04 08:57:27 UTC | 2018-12-04 08:57:26.969 UTC | 2018-12-04 08:57:26.969 UTC |
270
+
271
+
272
+
273
+ ※実行コマンド
274
+
275
+ C:\Users\yazaki\Desktop\embulk>embulk-0.9.8.bat run config_bq_incremental.yml -c diff.yml >> C:\Users\yazaki\Desktop\embulk\log\ORA_to_BQ_log004.txt
276
+
277
+
278
+
279
+
280
+
281
+ ※実行後のエラー
282
+
283
+ org.embulk.exec.PartialExecutionException: org.embulk.spi.DataException: Column type 'string' set at incremental_columns option is not supported
284
+
285
+ at org.embulk.exec.BulkLoader$LoaderState.buildPartialExecuteException(BulkLoader.java:339)
286
+
287
+ at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:565)
288
+
289
+ at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:34)
290
+
291
+ at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:352)
292
+
293
+ at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:349)
294
+
295
+ at org.embulk.spi.Exec.doWith(Exec.java:22)
296
+
297
+ at org.embulk.exec.BulkLoader.run(BulkLoader.java:349)
298
+
299
+ at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:161)
300
+
301
+ at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:292)
302
+
303
+ at org.embulk.EmbulkRunner.run(EmbulkRunner.java:156)
304
+
305
+ at org.embulk.cli.EmbulkRun.runSubcommand(EmbulkRun.java:436)
306
+
307
+ at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:91)
308
+
309
+ at org.embulk.cli.Main.main(Main.java:26)
310
+
311
+ Caused by: org.embulk.spi.DataException: Column type 'string' set at incremental_columns option is not supported
312
+
313
+ at org.embulk.input.jdbc.getter.AbstractColumnGetter.encodeToJson(org/embulk/input/jdbc/getter/AbstractColumnGetter.java:92)
314
+
315
+ at org.embulk.input.jdbc.AbstractJdbcInputPlugin$LastRecordStore.accept(org/embulk/input/jdbc/AbstractJdbcInputPlugin.java:441)
316
+
317
+ at org.embulk.input.jdbc.AbstractJdbcInputPlugin.run(org/embulk/input/jdbc/AbstractJdbcInputPlugin.java:490)
318
+
319
+ at org.embulk.exec.LocalExecutorPlugin$ScatterExecutor.runInputTask(org/embulk/exec/LocalExecutorPlugin.java:271)
320
+
321
+ at org.embulk.exec.LocalExecutorPlugin$ScatterExecutor.access$000(org/embulk/exec/LocalExecutorPlugin.java:196)
322
+
323
+ at org.embulk.exec.LocalExecutorPlugin$ScatterExecutor$1.call(org/embulk/exec/LocalExecutorPlugin.java:235)
324
+
325
+ at org.embulk.exec.LocalExecutorPlugin$ScatterExecutor$1.call(org/embulk/exec/LocalExecutorPlugin.java:232)
326
+
327
+ at java.util.concurrent.FutureTask.run(java/util/concurrent/FutureTask)
328
+
329
+ at java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor)
330
+
331
+ at java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor$Worker)
332
+
333
+ at java.lang.Thread.run(java/lang/Thread)
334
+
335
+
336
+
337
+ Error: org.embulk.spi.DataException: Column type 'string' set at incremental_columns option is not supported
338
+
339
+
340
+
341
+ ```

2

質問更新

2018/12/05 02:29

投稿

raccoondog
raccoondog

スコア79

test CHANGED
File without changes
test CHANGED
@@ -18,91 +18,129 @@
18
18
 
19
19
 
20
20
 
21
- ※config_bq.yml
21
+ ※2018/12/05更新
22
22
 
23
- ```
23
+ ```ここに言語を入力
24
24
 
25
- in: {type: oracle, driver_path: 'C:\Users\ｘｘｘｘ\Desktop\embulkDB\product\12.2.0\dbhome_1\sqldeveloper\jdbc\lib\ojdbc7.jar',
26
-
27
- host: A0430-02396, user: system, password: ｘｘｘｘ, database: ｘｘｘ, table: ｘｘｘｘ, incremental: true}
28
-
29
- out: {type: bigquery, auth_method: json_key, json_keyfile: 'C:\Users\ｘｘｘｘ\Desktop\embulk\My
30
-
31
- First Project-6041ffecfba7.json', project: ｘｘｘｘ, dataset: ｘｘｘｘ,
32
-
33
- auto_create_table: true, table: users, incremental: true}
25
+ ※Oracleテーブル定義
34
-
35
- ```
36
26
 
37
27
 
38
28
 
39
- ※実行時のエラー
29
+ ----------------- -------- ---
40
30
 
41
- ```
42
31
 
43
- C:\Users\ｘｘｘｘ\Desktop\embulk>embulk-0.9.8.bat run config_bq.yml -r resume_state.yml
44
32
 
45
- 2018-11-27 11:05:06.519 +0900: Embulk v0.9.8
33
+ ID NOT NULL NUMBER(8)
46
34
 
47
- 2018-11-27 11:05:07.564 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
35
+ NUM NUMBER(12,2
48
36
 
49
- 2018-11-27 11:05:11.457 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\ｘｘｘｘ.embulk\lib\gems"
37
+ STR CHAR(8)
50
38
 
51
- 2018-11-27 11:05:12.995 +0900 [INFO] (main): Started Embulk v0.9.8
39
+ VARSTR VARCHAR2(8)
52
40
 
53
- 2018-11-27 11:05:13.120 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-oracle (0.9.3)
41
+ DT DATE
54
42
 
55
- 2018-11-27 11:05:19.126 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-bigquery (0.4.9)
43
+ TIME0 TIMESTAMP(0
56
44
 
57
- 2018-11-27 11:05:19.176 +0900 [INFO] (0001:transaction): Connecting to jdbc:oracle:thin:@A0430-02396:1521:ORCL options {oracle.jdbc.ReadTimeout=1800000, user=system, password=***, oracle.net.CONNECT_TIMEOUT=300000}
45
+ TIME6 TIMESTAMP(6
58
46
 
59
- 2018-11-27 11:05:19.830 +0900 [INFO] (0001:transaction): Using JDBC Driver 12.1.0.2.0
47
+ TIME9 TIMESTAMP(9
60
48
 
61
- 2018-11-27 11:05:19.877 +0900 [INFO] (0001:transaction): Using primary keys as incremental_columns: [ID]
62
49
 
63
- 2018-11-27 11:05:19.971 +0900 [INFO] (0001:transaction): Using local thread executor with max_threads=8 / output tasks 4 = input tasks 1 * 4
64
50
 
65
- 2018-11-27 11:05:20.065 +0900 [INFO] (0001:transaction): embulk-output-bigquery: Get dataset... fluid-emissary-216806:embulk_test
66
51
 
67
- 2018-11-27 11:05:23.936 +0900 [INFO] (0001:transaction): embulk-output-bigquery: Create table... fluid-emissary-216806:embulk_test.LOAD_TEMP_b43a147c_fdcc_47e3_8b4e_08982a037c4c_users
68
52
 
69
- 2018-11-27 11:05:24.770 +0900 [INFO] (0001:transaction): {done: 0 / 1, running: 0}
53
+ ※config_bq.yml.liquid
70
54
 
71
- 2018-11-27 11:05:24.914 +0900 [INFO] (0022:task-0000): Connecting to jdbc:oracle:thin:@A0430-02396:1521:ORCL options {oracle.jdbc.ReadTimeout=1800000, user=system, password=***, oracle.net.CONNECT_TIMEOUT=300000}
72
55
 
73
- 2018-11-27 11:05:25.007 +0900 [INFO] (0022:task-0000): SQL: SELECT * FROM "INPUT_TEST" ORDER BY "ID"
74
56
 
75
- 2018-11-27 11:05:25.085 +0900 [INFO] (0022:task-0000): > 0.08 seconds
57
+ in:
76
58
 
77
- 2018-11-27 11:05:25.148 +0900 [INFO] (0022:task-0000): Fetched 500 rows.
59
+ type: oracle
78
60
 
79
- 2018-11-27 11:05:25.181 +0900 [INFO] (embulk-output-executor-0): embulk-output-bigquery: create C:/Users/ｘｘｘｘ/AppData/Local/Temp/embulk_output_bigquery_20181127-5668-19qsowq.5668.2000.csv
61
+ path_prefix: null
80
62
 
81
- 2018-11-27 11:05:25.181 +0900 [INFO] (0022:task-0000): Fetched 1,000 rows.
63
+ driver_path: C:\Users\yazaki\Desktop\embulkDB\product\12.2.0\dbhome_1\sqldeveloper\jdbc\lib\ojdbc7.jar
82
64
 
83
- 2018-11-27 11:05:25.197 +0900 [INFO] (embulk-output-executor-1): embulk-output-bigquery: create C:/Users/ｘｘｘｘ/AppData/Local/Temp/embulk_output_bigquery_20181127-5668-19qsowq.5668.2002.csv
65
+ host: A0430-02396
84
66
 
85
- 2018-11-27 11:05:25.197 +0900 [INFO] (embulk-output-executor-2): embulk-output-bigquery: create C:/Users/ｘｘｘｘ/AppData/Local/Temp/embulk_output_bigquery_20181127-5668-19qsowq.5668.2004.csv
67
+ user: system
86
68
 
87
- 2018-11-27 11:05:25.228 +0900 [INFO] (embulk-output-executor-3): embulk-output-bigquery: create C:/Users/ｘｘｘｘ/AppData/Local/Temp/embulk_output_bigquery_20181127-5668-19qsowq.5668.2006.csv
69
+ password: yazaki
88
70
 
89
- 2018-11-27 11:05:26.921 +0900 [INFO] (0022:task-0000): Fetched 2,000 rows.
71
+ database: ORCL
90
72
 
91
- 2018-11-27 11:05:27.490 +0900 [INFO] (0022:task-0000): Fetched 4,000 rows.
73
+ table: INPUT_TEST
92
74
 
93
- 2018-11-27 11:05:28.963 +0900 [INFO] (0022:task-0000): Fetched 8,000 rows.
75
+ incremental: true
94
76
 
95
- 2018-11-27 11:05:29.635 +0900 [INFO] (0001:transaction): {done: 1 / 1, running: 0}
77
+ incremental_columns: [id]
96
78
 
97
- 2018-11-27 11:05:29.635 +0900 [INFO] (0001:transaction): embulk-output-bigquery: Delete table... fluid-emissary-216806:embulk_test.LOAD_TEMP_b43a147c_fdcc_47e3_8b4e_08982a037c4c_users
79
+ skip_header_lines: 0
98
80
 
99
- 2018-11-27 11:05:30.041 +0900 [INFO] (main): Writing resume state to 'resume_state.yml'
81
+ allow_extra_columns: false
100
82
 
101
- 2018-11-27 11:05:30.104 +0900 [INFO] (main): Resume state is written. Run the transaction again with -r option to resume or use "cleanup" subcommand to delete intermediate data.
83
+ allow_optional_columns: true
102
84
 
103
- java.lang.RuntimeException: org.embulk.spi.DataException: Column type 'double' set at incremental_columns option is not supported
85
+ columns:
104
86
 
87
+ - {name: ID, type: string}
88
+
89
+ - {name: NUM, type: string}
90
+
91
+ - {name: STR, type: string}
92
+
93
+ - {name: VARSTR, type: string}
94
+
95
+ - {name: DT, type: string}
96
+
97
+ - {name: TIME0, type: timestamp, format: '%Y/%m/%d'}
98
+
99
+ - {name: TIME6, type: timestamp, format: '%Y/%m/%d'}
100
+
101
+ - {name: TIME9, type: timestamp, format: '%Y/%m/%d'}
102
+
103
+ out: {type: bigquery, auth_method: json_key, json_keyfile: 'C:\Users\yazaki\Desktop\embulk\My
104
+
105
+ First Project-6041ffecfba7.json', project: fluid-emissary-216806, dataset: embulk_test,
106
+
107
+ auto_create_table: true, table: users, read_timeout_sec: 360000, open_timeout_sec: 360000,
108
+
109
+ send_timeout_sec: 360000, mode: append}
110
+
111
+
112
+
113
+
114
+
115
+ ※実行コマンド
116
+
117
+ embulk-0.9.8.bat run config_bq.yml.liquid -c diff.yml >> C:\Users\yazaki\Desktop\embulk\log\ORA_to_BQ_log003.txt
118
+
119
+
120
+
121
+
122
+
123
+ ※実行後のエラー
124
+
125
+ org.embulk.exec.PartialExecutionException: org.embulk.spi.DataException: Column type 'double' set at incremental_columns option is not supported
126
+
127
+ at org.embulk.exec.BulkLoader$LoaderState.buildPartialExecuteException(BulkLoader.java:339)
128
+
129
+ at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:565)
130
+
131
+ at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:34)
132
+
133
+ at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:352)
134
+
135
+ at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:349)
136
+
137
+ at org.embulk.spi.Exec.doWith(Exec.java:22)
138
+
139
+ at org.embulk.exec.BulkLoader.run(BulkLoader.java:349)
140
+
141
+ at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:161)
142
+
105
- at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:317)
143
+ at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:292)
106
144
 
107
145
  at org.embulk.EmbulkRunner.run(EmbulkRunner.java:156)
108
146
 
@@ -142,116 +180,4 @@
142
180
 
143
181
 
144
182
 
145
- C:\Users\ｘｘｘｘ\Desktop\embulk>
146
-
147
-
148
-
149
183
  ```
150
-
151
-
152
-
153
- 状況更新
154
-
155
- ```ここに言語を入力
156
-
157
-
158
-
159
-
160
-
161
- ■Oracleテーブル定義
162
-
163
-
164
-
165
- SQL> desc INPUT_TEST
166
-
167
- 名前 NULL? 型
168
-
169
- ----------------------------------------- -------- ----------------------------
170
-
171
- ID NOT NULL NUMBER(8)
172
-
173
- NUM NUMBER(12,2)
174
-
175
- STR CHAR(8)
176
-
177
- VARSTR VARCHAR2(8)
178
-
179
- DT DATE
180
-
181
- TIME0 TIMESTAMP(0)
182
-
183
- TIME6 TIMESTAMP(6)
184
-
185
- TIME9 TIMESTAMP(9)
186
-
187
-
188
-
189
- SQL>
190
-
191
-
192
-
193
-
194
-
195
- ■Embulk定義
196
-
197
- in: {type: oracle, driver_path: 'C:\Users\yazaki\Desktop\embulkDB\product\12.2.0\dbhome_1\sqldeveloper\jdbc\lib\ojdbc7.jar',
198
-
199
- host: A0430-02396, user: system, password: yazaki, database: ORCL, table: INPUT_TEST, incremental: true, incremental_columns: id}
200
-
201
- out: {type: bigquery, auth_method: json_key, json_keyfile: 'C:\Users\yazaki\Desktop\embulk\My
202
-
203
- First Project-6041ffecfba7.json', project: fluid-emissary-216806, dataset: embulk_test,
204
-
205
- auto_create_table: true, table: users, read_timeout_sec: 360000, open_timeout_sec: 360000, send_timeout_sec: 360000, mode: insert}
206
-
207
-
208
-
209
- ■Embulk実行
210
-
211
-
212
-
213
- dos cmd > C:\Users\yazaki\Desktop\embulk\embulk-0.9.8.bat run C:\Users\yazaki\Desktop\embulk\config_bq.yml -c C:\Users\yazaki\Desktop\embulk\diff.yml >> C:\Users\yazaki\Desktop\embulk\ORA_to_BQ_log002
214
-
215
-
216
-
217
-
218
-
219
- ■ログ
220
-
221
- PS C:\Windows\System32\WindowsPowerShell\v1.0> cat "C:\Users\yazaki\Desktop\embulk\ORA_to_BQ_log002" -wait -tail 10
222
-
223
- 2018-12-04 17:57:27.580 +0900: Embulk v0.9.8
224
-
225
- 2018-12-04 17:57:29.234 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
226
-
227
- 2018-12-04 17:57:34.704 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\yazaki.embulk\lib\gems"
228
-
229
- 2018-12-04 17:57:36.763 +0900 [INFO] (main): Started Embulk v0.9.8
230
-
231
- 2018-12-04 17:57:36.919 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-oracle (0.9.3)
232
-
233
- 2018-12-04 17:57:44.440 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-bigquery (0.4.9)
234
-
235
- 2018-12-04 17:57:45.410 +0900: Embulk v0.9.8
236
-
237
- 2018-12-04 17:57:47.110 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
238
-
239
- 2018-12-04 17:57:52.525 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\yazaki.embulk\lib\gems"
240
-
241
- 2018-12-04 17:57:54.569 +0900 [INFO] (main): Started Embulk v0.9.8
242
-
243
- 2018-12-04 17:57:54.819 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-oracle (0.9.3)
244
-
245
- 2018-12-04 17:58:02.505 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-bigquery (0.4.9)
246
-
247
- 2018-12-04 17:58:03.163 +0900: Embulk v0.9.8
248
-
249
- 2018-12-04 17:58:04.333 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
250
-
251
- 2018-12-04 17:58:10.153 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\yazaki.embulk\lib\gems"
252
-
253
- 2018-12-04 17:58:12.368 +0900 [INFO] (main): Started Embulk v0.9.8
254
-
255
- 2018-12-04 17:58:12.555 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-oracle (0.9.3)
256
-
257
- ```

1

質問内容更新

2018/12/05 01:04

投稿

raccoondog
raccoondog

スコア79

test CHANGED
File without changes
test CHANGED
@@ -147,3 +147,111 @@
147
147
 
148
148
 
149
149
  ```
150
+
151
+
152
+
153
+ 状況更新
154
+
155
+ ```ここに言語を入力
156
+
157
+
158
+
159
+
160
+
161
+ ■Oracleテーブル定義
162
+
163
+
164
+
165
+ SQL> desc INPUT_TEST
166
+
167
+ 名前 NULL? 型
168
+
169
+ ----------------------------------------- -------- ----------------------------
170
+
171
+ ID NOT NULL NUMBER(8)
172
+
173
+ NUM NUMBER(12,2)
174
+
175
+ STR CHAR(8)
176
+
177
+ VARSTR VARCHAR2(8)
178
+
179
+ DT DATE
180
+
181
+ TIME0 TIMESTAMP(0)
182
+
183
+ TIME6 TIMESTAMP(6)
184
+
185
+ TIME9 TIMESTAMP(9)
186
+
187
+
188
+
189
+ SQL>
190
+
191
+
192
+
193
+
194
+
195
+ ■Embulk定義
196
+
197
+ in: {type: oracle, driver_path: 'C:\Users\yazaki\Desktop\embulkDB\product\12.2.0\dbhome_1\sqldeveloper\jdbc\lib\ojdbc7.jar',
198
+
199
+ host: A0430-02396, user: system, password: yazaki, database: ORCL, table: INPUT_TEST, incremental: true, incremental_columns: id}
200
+
201
+ out: {type: bigquery, auth_method: json_key, json_keyfile: 'C:\Users\yazaki\Desktop\embulk\My
202
+
203
+ First Project-6041ffecfba7.json', project: fluid-emissary-216806, dataset: embulk_test,
204
+
205
+ auto_create_table: true, table: users, read_timeout_sec: 360000, open_timeout_sec: 360000, send_timeout_sec: 360000, mode: insert}
206
+
207
+
208
+
209
+ ■Embulk実行
210
+
211
+
212
+
213
+ dos cmd > C:\Users\yazaki\Desktop\embulk\embulk-0.9.8.bat run C:\Users\yazaki\Desktop\embulk\config_bq.yml -c C:\Users\yazaki\Desktop\embulk\diff.yml >> C:\Users\yazaki\Desktop\embulk\ORA_to_BQ_log002
214
+
215
+
216
+
217
+
218
+
219
+ ■ログ
220
+
221
+ PS C:\Windows\System32\WindowsPowerShell\v1.0> cat "C:\Users\yazaki\Desktop\embulk\ORA_to_BQ_log002" -wait -tail 10
222
+
223
+ 2018-12-04 17:57:27.580 +0900: Embulk v0.9.8
224
+
225
+ 2018-12-04 17:57:29.234 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
226
+
227
+ 2018-12-04 17:57:34.704 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\yazaki.embulk\lib\gems"
228
+
229
+ 2018-12-04 17:57:36.763 +0900 [INFO] (main): Started Embulk v0.9.8
230
+
231
+ 2018-12-04 17:57:36.919 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-oracle (0.9.3)
232
+
233
+ 2018-12-04 17:57:44.440 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-bigquery (0.4.9)
234
+
235
+ 2018-12-04 17:57:45.410 +0900: Embulk v0.9.8
236
+
237
+ 2018-12-04 17:57:47.110 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
238
+
239
+ 2018-12-04 17:57:52.525 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\yazaki.embulk\lib\gems"
240
+
241
+ 2018-12-04 17:57:54.569 +0900 [INFO] (main): Started Embulk v0.9.8
242
+
243
+ 2018-12-04 17:57:54.819 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-oracle (0.9.3)
244
+
245
+ 2018-12-04 17:58:02.505 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-bigquery (0.4.9)
246
+
247
+ 2018-12-04 17:58:03.163 +0900: Embulk v0.9.8
248
+
249
+ 2018-12-04 17:58:04.333 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
250
+
251
+ 2018-12-04 17:58:10.153 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\yazaki.embulk\lib\gems"
252
+
253
+ 2018-12-04 17:58:12.368 +0900 [INFO] (main): Started Embulk v0.9.8
254
+
255
+ 2018-12-04 17:58:12.555 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-oracle (0.9.3)
256
+
257
+ ```