forked from docs/doc-exports
Su, Xiaomeng
483e5cc6cc
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com> Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
83 lines
6.8 KiB
HTML
83 lines
6.8 KiB
HTML
<a name="dli_02_0310"></a><a name="dli_02_0310"></a>
|
|
|
|
<h1 class="topictitle1">Creating and Submitting a Flink Job</h1>
|
|
<div id="body1602473883611"><div class="section" id="dli_02_0310__section851316282599"><h4 class="sectiontitle">Scenario Description</h4><p id="dli_02_0310__p1822151903">This section describes how to create and run a user-defined Flink job using APIs.</p>
|
|
</div>
|
|
<div class="section" id="dli_02_0310__section155144213214"><h4 class="sectiontitle">Constraints</h4><ul id="dli_02_0310__ul759612191669"><li id="dli_02_0310__li45963199610">It takes 6 to 10 minutes to start a job using a new queue for the first time.</li></ul>
|
|
</div>
|
|
<div class="section" id="dli_02_0310__section13856162971"><h4 class="sectiontitle">Involved APIs</h4><ul id="dli_02_0310__ul1862251633812"><li id="dli_02_0310__li462221673818"><a href="dli_02_0194.html">Creating a Queue</a>: Create a queue.</li><li id="dli_02_0310__li46223161388"><a href="dli_02_0130.html">Uploading a Package Group (Discarded)</a>: Upload the resource package required by the Flink custom job.</li><li id="dli_02_0310__li1137901761318"><a href="dli_02_0172.html">Querying Resource Packages in a Group (Discarded)</a>: Check whether the uploaded resource package is correct.</li><li id="dli_02_0310__li9641746152414"><a href="dli_02_0230.html">Creating a Flink Jar job</a> Create a user-defined Flink job.</li><li id="dli_02_0310__li2034416201925"><a href="dli_02_0233.html">Running Jobs in Batches</a>: Run a user-defined Flink job.</li></ul>
|
|
</div>
|
|
<div class="section" id="dli_02_0310__section2742155213719"><h4 class="sectiontitle">Procedure</h4><ol id="dli_02_0310__ol454620011213"><li id="dli_02_0310__li135461061218">Create a queue for general use. For details, see <a href="dli_02_0307.html">Creating a Queue</a>. In the request, set <strong id="dli_02_0310__b866451305419">resource_mode</strong> to <strong id="dli_02_0310__b1921715168545">1</strong> to create a dedicated queue.</li><li id="dli_02_0310__li117291344122510">Upload the resource package of the user-defined Flink job. For details, see <a href="dli_02_0309.html#dli_02_0309__li117291344122510">2</a>.</li><li id="dli_02_0310__li970315312304">Query resource packages in a group. For details, see <a href="dli_02_0309.html#dli_02_0309__li970315312304">3</a>.</li><li id="dli_02_0310__li125211016104016">Create a custom flink job.<ul id="dli_02_0310__ul26571210114414"><li id="dli_02_0310__li146576102444">API<p id="dli_02_0310__p10657111019442"><a name="dli_02_0310__li146576102444"></a><a name="li146576102444"></a>URI format: POST /v1.0/{<em id="dli_02_0310__i18298164714552">project_id</em>}/streaming/flink-jobs</p>
|
|
<ul id="dli_02_0310__ul56581107449"><li id="dli_02_0310__li17658191013442">Obtain the value of {<em id="dli_02_0310__i10529144915517">project_id</em>} from <a href="dli_02_0183.html">Obtaining a Project ID</a>.</li><li id="dli_02_0310__li3658201018442">For details about the request parameters, see <a href="dli_02_0028.html">Creating a Database (Discarded)</a>.</li></ul>
|
|
</li><li id="dli_02_0310__li10658410164410">Request example<ul id="dli_02_0310__ul14658161015442"><li id="dli_02_0310__li865851014449">Description: Create a user-defined Flink job in the project whose ID is <strong id="dli_02_0310__b2315144145612">48cc2c48765f481480c7db940d6409d1</strong>.</li><li id="dli_02_0310__li6658171074411">Example URL: POST https://{<em id="dli_02_0310__i353764845616">endpoint</em>}/v1.0/48cc2c48765f481480c7db940d6409d1/streaming/flink-jobs</li><li id="dli_02_0310__li165811084419">Body:<pre class="screen" id="dli_02_0310__screen3658131064420">{
|
|
"name": "test",
|
|
"desc": "job for test",
|
|
"queue_name": "testQueue",
|
|
"manager_cu_number": 1,
|
|
"cu_number": 2,
|
|
"parallel_number": 1,
|
|
"tm_cus": 1,
|
|
"tm_slot_num": 1,
|
|
"log_enabled": true,
|
|
"obs_bucket": "bucketName",
|
|
"smn_topic": "topic",
|
|
"main_class": "org.apache.flink.examples.streaming.JavaQueueStream",
|
|
"restart_when_exception": false,
|
|
"entrypoint": "javaQueueStream.jar",
|
|
"entrypoint_args":"-windowSize 2000 -rate3",
|
|
"dependency_jars": [
|
|
"myGroup/test.jar",
|
|
"myGroup/test1.jar"
|
|
],
|
|
"dependency_files": [
|
|
"myGroup/test.csv",
|
|
"myGroup/test1.csv"
|
|
]
|
|
}</pre>
|
|
</li></ul>
|
|
</li><li id="dli_02_0310__li2065912101446">Example response<pre class="screen" id="dli_02_0310__screen765961024418">{
|
|
"is_success": true,
|
|
"message": "A Flink job is created successfully.",
|
|
"job": {
|
|
"job_id": 138,
|
|
"status_name": "job_init",
|
|
"status_desc": ""
|
|
}
|
|
}</pre>
|
|
</li></ul>
|
|
</li><li id="dli_02_0310__li18904826102815">Run jobs in batches.<ul id="dli_02_0310__ul178791154132816"><li id="dli_02_0310__li1187919544287">API<p id="dli_02_0310__p198791154192815"><a name="dli_02_0310__li1187919544287"></a><a name="li1187919544287"></a>URI format: POST /v1.0/{<em id="dli_02_0310__i396792595714">project_id</em>}/streaming/jobs/run</p>
|
|
<ul id="dli_02_0310__ul158791254132815"><li id="dli_02_0310__li7879854172815">Obtain the value of {<em id="dli_02_0310__i6586133245719">project_id</em>} from <a href="dli_02_0183.html">Obtaining a Project ID</a>.</li><li id="dli_02_0310__li13879115411285">For details about the request parameters, see <a href="dli_02_0233.html">Running Jobs in Batches</a>.</li></ul>
|
|
</li><li id="dli_02_0310__li7879185418289">Request example<ul id="dli_02_0310__ul1287995452812"><li id="dli_02_0310__li118791554122819">Description: Run the jobs whose <strong id="dli_02_0310__b24295912573">job_id</strong> is <strong id="dli_02_0310__b1195014415820">298765</strong> and <strong id="dli_02_0310__b1150357135815">298766</strong> in the project whose ID is <strong id="dli_02_0310__b1951611195810">48cc2c48765f481480c7db940d6409d1</strong>.</li><li id="dli_02_0310__li6879145416284">Example URL: POST https://{<em id="dli_02_0310__i1862852495815">endpoint</em>}/v1.0/48cc2c48765f481480c7db940d6409d1/streaming/jobs/run</li><li id="dli_02_0310__li16880154182814">Body:<pre class="screen" id="dli_02_0310__screen488075411287">{
|
|
"job_ids": [131,130,138,137],
|
|
"resume_savepoint": true
|
|
}</pre>
|
|
</li></ul>
|
|
</li><li id="dli_02_0310__li198801754192819">Example response<pre class="screen" id="dli_02_0310__screen78801654182815">[
|
|
{
|
|
"is_success": "true",
|
|
"message": "The request for submitting DLI jobs is delivered successfully."
|
|
},
|
|
{
|
|
"is_success": "true",
|
|
"message": "The request for submitting DLI jobs is delivered successfully."
|
|
},
|
|
{
|
|
"is_success": "true",
|
|
"message": "The request for submitting DLI jobs is delivered successfully."
|
|
},
|
|
{
|
|
"is_success": "true",
|
|
"message": "The request for submitting DLI jobs is delivered successfully."
|
|
}
|
|
]</pre>
|
|
</li></ul>
|
|
</li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_02_0306.html">Getting Started</a></div>
|
|
</div>
|
|
</div>
|
|
|