forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
77 lines
5.2 KiB
HTML
77 lines
5.2 KiB
HTML
<a name="mrs_01_1050"></a><a name="mrs_01_1050"></a>
|
|
|
|
<h1 class="topictitle1">Completely Migrating Storm Services</h1>
|
|
<div id="body1590370637069"><div class="section" id="mrs_01_1050__section98244312213"><h4 class="sectiontitle">Scenarios</h4><p id="mrs_01_1050__p29121231220">This section describes how to convert and run a complete Storm topology developed using Storm API.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1050__section158251435222"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_1050__ol24171024182316"><li id="mrs_01_1050__li94171724102314"><span>Open the Storm service project, modify the POM file of the project, and add the reference of <strong id="mrs_01_1050__b541682492313">flink-storm_2.11</strong>, <strong id="mrs_01_1050__b4416102482316">flink-core</strong>, and <strong id="mrs_01_1050__b1541642414235">flink-streaming-java_2.11</strong>. The following figure shows an example.</span><p><p id="mrs_01_1050__p1416124142310"></p>
|
|
<pre class="screen" id="mrs_01_1050__screen16131132253017"><dependency>
|
|
<groupId>org.apache.flink</groupId>
|
|
<artifactId>flink-storm_2.11</artifactId>
|
|
<version>1.4.0</version>
|
|
<exclusions>
|
|
<exclusion>
|
|
<groupId>*</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
</exclusions>
|
|
</dependency></pre>
|
|
<pre class="screen" id="mrs_01_1050__screen421432282910"><dependency>
|
|
<groupId>org.apache.flink</groupId>
|
|
<artifactId>flink-core</artifactId>
|
|
<version>1.4.0</version>
|
|
<exclusions>
|
|
<exclusion>
|
|
<groupId>*</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
</exclusions>
|
|
</dependency></pre>
|
|
<div class="p" id="mrs_01_1050__p13142646192910"><pre class="screen" id="mrs_01_1050__screen957715183261"><dependency>
|
|
<groupId>org.apache.flink</groupId>
|
|
<artifactId>flink-streaming-java_2.11</artifactId>
|
|
<version>1.4.0</version>
|
|
<exclusions>
|
|
<exclusion>
|
|
<groupId>*</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
</exclusions>
|
|
</dependency></pre>
|
|
</div>
|
|
<div class="note" id="mrs_01_1050__note16417924152311"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1050__p5417142492318">If the project is not a non-Maven project, manually collect the preceding JAR packages and add them to the <em id="mrs_01_1050__i54175240234">classpath</em> environment variable of the project.</p>
|
|
</div></div>
|
|
</p></li></ol><ol start="2" id="mrs_01_1050__ol84155246231"><li id="mrs_01_1050__li34151524152314"><span>Modify the code for submission of the topology. The following uses WordCount as an example:</span><p><ol type="a" id="mrs_01_1050__ol14644141392611"><li id="mrs_01_1050__li96441813192615">Keep the structure of the Storm topology unchanged, including the Spout and Bolt developed using Storm API.</li></ol>
|
|
<pre class="screen" id="mrs_01_1050__screen19414202462310">TopologyBuilder builder = new TopologyBuilder();
|
|
builder.setSpout("spout", new RandomSentenceSpout(), 5);
|
|
builder.setBolt("split", new SplitSentenceBolt(), 8).shuffleGrouping("spout");
|
|
builder.setBolt("count", new WordCountBolt(), 12).fieldsGrouping("split", new Fields("word"));</pre>
|
|
<ol type="a" start="2" id="mrs_01_1050__ol464441362614"><li id="mrs_01_1050__li36442013122619">Modify the code for submission of the topology. An example is described as follows:</li></ol>
|
|
<pre class="screen" id="mrs_01_1050__screen241412472310">Config conf = new Config();
|
|
conf.setNumWorkers(3);
|
|
StormSubmitter.submitTopology("word-count", conf, builder.createTopology());</pre>
|
|
<p id="mrs_01_1050__p84141724182316">Perform the following operations:</p>
|
|
<pre class="screen" id="mrs_01_1050__screen641512415238">Config conf = new Config();
|
|
conf.setNumWorkers(3);
|
|
//converts Storm Config to StormConfig of Flink.
|
|
StormConfig stormConfig = new StormConfig(conf);
|
|
//Construct FlinkTopology using TopologBuilder of Storm.
|
|
FlinkTopology topology = FlinkTopology.createTopology(builder);
|
|
//Obtain the Stream execution environment.
|
|
StreamExecutionEnvironment env = topology.getExecutionEnvironment();
|
|
//Set StormConfig to the environment variable of Job to construct Bolt and Spout.
|
|
//If StormConfig is not required during the initialization of Bolt and Spout, you do not need to set this parameter.
|
|
env.getConfig().setGlobalJobParameters(stormConfig);
|
|
//Submit the topology.
|
|
topology.execute();</pre>
|
|
<ol type="a" start="3" id="mrs_01_1050__ol559131702617"><li id="mrs_01_1050__li10591417202617">After the package is repacked, run the following command to submit the package:<p id="mrs_01_1050__p1541592415235"><a name="mrs_01_1050__li10591417202617"></a><a name="li10591417202617"></a><strong id="mrs_01_1050__b17415172422318">flink run -class {MainClass} WordCount.jar</strong></p>
|
|
</li></ol>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1048.html">Migrating Storm Services to Flink</a></div>
|
|
</div>
|
|
</div>
|
|
|