forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
74 lines
14 KiB
HTML
74 lines
14 KiB
HTML
<a name="mrs_01_1060"></a><a name="mrs_01_1060"></a>
|
|
|
|
<h1 class="topictitle1">Configuring Non-encrypted Transmission</h1>
|
|
<div id="body1590374508401"><div class="section" id="mrs_01_1060__s0cb97aa89f1042c392b9fc24e7e79513"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1060__a92d808868c404364ab104117b6232a4c">This section describes how to configure Flume server and client parameters after the cluster and the Flume service are installed to ensure proper running of the service.</p>
|
|
<p id="mrs_01_1060__p141121017518">This section applies to MRS 3.<em id="mrs_01_1060__i151406210556">x</em> or later clusters.</p>
|
|
<div class="note" id="mrs_01_1060__ndd160f8933b9446596783f2a07aa2bcc"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p class="text" id="mrs_01_1060__af7d3f8067de140f2a767cc90c25c51b8">By default, the cluster network environment is secure and the SSL authentication is not enabled during the data transmission process. For details about how to use the encryption mode, see <a href="mrs_01_1069.html">Configuring the Encrypted Transmission</a>.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1060__sc533d51b187b45f6a2bcd50ddc39556a"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_1060__u6a6fc336add54ba58761fc908ff06fdc"><li id="mrs_01_1060__l72a28463ae7c4c799baf9e55efbaac0b">The cluster and Flume service have been installed.</li><li id="mrs_01_1060__l0422e75b03ea43a1b35a94aece09a7d5">The network environment of the cluster is secure.</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_1060__s9885d005d93044bbac2194705fc5bbb3"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_1060__o3c64252cfec843d8b8d8a8c26b08b111"><li id="mrs_01_1060__le96ae0c738ff4bdba4df51e7993798d4"><span>Configure the client parameters of the Flume role.</span><p><ol type="a" id="mrs_01_1060__o5d1c24e994ae43e2bd154e8d3dbb6256"><li id="mrs_01_1060__lfd8ae66cbb11466299503fa720020ebd">Use the Flume configuration tool on FusionInsight Manager to configure the Flume role client parameters and generate a configuration file.<ol class="substepthirdol" id="mrs_01_1060__oe7fc5bbc227745d2ac7b46b0ace2a18e"><li id="mrs_01_1060__l8dd968f25b3d4922abb3edd7ac864e78">Log in to FusionInsight Manager. Choose<strong id="mrs_01_1060__b0297635158"> Cluster</strong> > <strong id="mrs_01_1060__b530320315151">Services</strong> > <strong id="mrs_01_1060__b530314311153">Flume</strong> > <strong id="mrs_01_1060__b730393151519">Configuration Tool</strong>.</li><li id="mrs_01_1060__l41477adee2b542bd898ac4cbf283ded1">Set <strong id="mrs_01_1060__b6783713671035">Agent Name</strong> to <strong id="mrs_01_1060__b10998822221035">client</strong>. Select and drag the source, channel, and sink to be used to the GUI on the right, and connect them.<p id="mrs_01_1060__a091feaf306c74bd3a72bbb4a9b4e0830">For example, use SpoolDir Source, File Channel, and Avro Sink.</p>
|
|
</li><li id="mrs_01_1060__lfbfd4056beb04a41b3956cd1496886ba">Double-click the source, channel, and sink. Set corresponding configuration parameters by referring to <a href="#mrs_01_1060__table1953421416017">Table 1</a> based on the actual environment.<div class="note" id="mrs_01_1060__n6f884795f52645489f9cd872cf62be78"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_1060__uc07c46da58ed4ca381bda3a0eefdf4ec"><li id="mrs_01_1060__l9e5f446badd64ba2a171cfd8609c4bfc">If the client parameters of the Flume role have been configured, you can obtain the existing client parameter configuration file from <em id="mrs_01_1060__i4145102711035">client installation directory</em><strong id="mrs_01_1060__b13740975191035">/fusioninsight-flume-1.9.0/conf/properties.properties</strong> to ensure that the configuration is in concordance with the previous. Log in to FusionInsight Manager, choose <strong id="mrs_01_1060__b146852161618">Cluster</strong> > <strong id="mrs_01_1060__b57573587107">Services</strong> > <strong id="mrs_01_1060__b7446829111">Flume</strong> > <strong id="mrs_01_1060__b3357197121114">Configuration</strong> > <strong id="mrs_01_1060__b174123104119">Import</strong>, import the file, and modify the configuration items related to non-encrypted transmission.</li><li id="mrs_01_1060__lb018311b54284e728b7f99152cc19ce8">It is recommended that the numbers of Sources, Channels, and Sinks do not exceed 40 during configuration file import. Otherwise, the response time may be very long.</li></ul>
|
|
</div></div>
|
|
|
|
<div class="tablenoborder"><a name="mrs_01_1060__table1953421416017"></a><a name="table1953421416017"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1060__table1953421416017" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters to be modified for the Flume role client</caption><thead align="left"><tr id="mrs_01_1060__row11534141410012"><th align="left" class="cellrowborder" valign="top" width="16%" id="mcps1.3.3.2.1.2.1.1.1.3.3.2.4.1.1"><p id="mrs_01_1060__p11535111413017">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="55.00000000000001%" id="mcps1.3.3.2.1.2.1.1.1.3.3.2.4.1.2"><p id="mrs_01_1060__p12535161410019">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="28.999999999999996%" id="mcps1.3.3.2.1.2.1.1.1.3.3.2.4.1.3"><p id="mrs_01_1060__p253513149010">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1060__row19535131416017"><td class="cellrowborder" valign="top" width="16%" headers="mcps1.3.3.2.1.2.1.1.1.3.3.2.4.1.1 "><p id="mrs_01_1060__p15351414303">ssl</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="55.00000000000001%" headers="mcps1.3.3.2.1.2.1.1.1.3.3.2.4.1.2 "><p id="mrs_01_1060__p85351614403">Specifies whether to enable the SSL authentication. (You are advised to enable this function to ensure security.)</p>
|
|
<p id="mrs_01_1060__p453531410015">Only Sources of the Avro type have this configuration item.</p>
|
|
<ul id="mrs_01_1060__ul153513141809"><li id="mrs_01_1060__li14535514607"><strong id="mrs_01_1060__b7888980491035">true</strong> indicates that the function is enabled.</li><li id="mrs_01_1060__li753581417017"><strong id="mrs_01_1060__b10892039401035">false</strong> indicates that the function is not enabled.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.999999999999996%" headers="mcps1.3.3.2.1.2.1.1.1.3.3.2.4.1.3 "><p id="mrs_01_1060__p953516141019">false</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="mrs_01_1060__en-us_topic_0060028841_li28711884144">Click <strong id="mrs_01_1060__b14628654531035">Export</strong> to save the <strong id="mrs_01_1060__b12058647871035">properties.properties</strong> configuration file to the local server.</li></ol>
|
|
</li><li id="mrs_01_1060__l25df333186c14858bd8554a9407ab49c">Upload the <strong id="mrs_01_1060__b19717498081035">properties.properties</strong> file to <strong id="mrs_01_1060__b13322427521035">flume/conf/</strong> under the installation directory of the Flume client.</li></ol>
|
|
</p></li><li id="mrs_01_1060__led12e9da92e34b6bb134d4e8d55f87bd"><span>Configure the server parameters of the Flume role and upload the configuration file to the cluster.</span><p><ol type="a" id="mrs_01_1060__oebf6cfcb6dd4497987425fcbd0828492"><li id="mrs_01_1060__leac68130006945d09c4d7429c0ed4dea">Use the Flume configuration tool on the FusionInsight Manager portal to configure the server parameters and generate the configuration file.<ol class="substepthirdol" id="mrs_01_1060__o9bfd40d1591d452a9da6e48bb43384cd"><li id="mrs_01_1060__lcfddf79a865b476c9d406a1b3d165089">Log in to FusionInsight Manager. Choose<strong id="mrs_01_1060__b16289191410156"> Cluster</strong> > <strong id="mrs_01_1060__b1295814181510">Services</strong> > <strong id="mrs_01_1060__b5295191417151">Flume</strong> > <strong id="mrs_01_1060__b1829515149153">Configuration Tool</strong>.</li><li id="mrs_01_1060__l329ebff58c6e4be1ba007882a5c546bc">Set <strong id="mrs_01_1060__b1097611161663">Agent Name</strong> to <strong id="mrs_01_1060__b998218166616">server</strong>. Select and drag the source, channel, and sink to be used to the GUI on the right, and connect them.<p id="mrs_01_1060__a834342b660f14a9d893d98cb4e8a9f0e">For example, use Avro Source, File Channel, and HDFS Sink.</p>
|
|
</li><li id="mrs_01_1060__l764294befac24bdb8ab1a7f612c2a8b5">Double-click the source, channel, and sink. Set corresponding configuration parameters by referring to <a href="#mrs_01_1060__table1324711382017">Table 2</a> based on the actual environment.<div class="note" id="mrs_01_1060__na0b8a51a19cf4a26bfe868361d890e0d"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_1060__u3c502f2d5daf4a6d8694e5a08dee716d"><li id="mrs_01_1060__l85a4267f7cd44fa2ada08a148d289d16">If the server parameters of the Flume role have been configured, you can choose <strong id="mrs_01_1060__b18922143951617">Cluster</strong> > <strong id="mrs_01_1060__b15976065751035">Services</strong> > <strong id="mrs_01_1060__b6773185821035">Flume</strong> > <strong id="mrs_01_1060__b16397873401035">Instance</strong> on FusionInsight Manager. Then select the corresponding Flume role instance and click the <strong id="mrs_01_1060__b10447147731035">Download</strong> button behind the <span class="parmname" id="mrs_01_1060__parmname7672003841035"><b>flume.config.file</b></span> parameter on the <strong id="mrs_01_1060__b21018716781035">Instance Configurations</strong> page to obtain the existing server parameter configuration file. Choose <strong id="mrs_01_1060__b16903145331613">Cluster</strong> > <strong id="mrs_01_1060__b2266115621716">Service</strong> > <strong id="mrs_01_1060__b17327155818179">Flume</strong> > <strong id="mrs_01_1060__b1183916001810">Configurations</strong> > <strong id="mrs_01_1060__b84213331812">Import</strong>, import the file, and modify the configuration items related to non-encrypted transmission.</li><li id="mrs_01_1060__l9893349b35d444dc9ad0beaed31b18e5">It is recommended that the numbers of Sources, Channels, and Sinks do not exceed 40 during configuration file import. Otherwise, the response time may be very long.</li><li id="mrs_01_1060__l1657d86d3d5b4936b8ea1baea1bd72f9">A unique checkpoint directory needs to be configured for each File Channel.</li></ul>
|
|
</div></div>
|
|
|
|
<div class="tablenoborder"><a name="mrs_01_1060__table1324711382017"></a><a name="table1324711382017"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1060__table1324711382017" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Parameters to be modified for the Flume role server</caption><thead align="left"><tr id="mrs_01_1060__row192481438705"><th align="left" class="cellrowborder" valign="top" width="23.232323232323232%" id="mcps1.3.3.2.2.2.1.1.1.3.3.2.4.1.1"><p id="mrs_01_1060__p0248038604">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="45.45454545454545%" id="mcps1.3.3.2.2.2.1.1.1.3.3.2.4.1.2"><p id="mrs_01_1060__p102481538707">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="31.313131313131308%" id="mcps1.3.3.2.2.2.1.1.1.3.3.2.4.1.3"><p id="mrs_01_1060__p92487381606">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1060__row6248143810020"><td class="cellrowborder" valign="top" width="23.232323232323232%" headers="mcps1.3.3.2.2.2.1.1.1.3.3.2.4.1.1 "><p id="mrs_01_1060__p1224820381908">ssl</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="45.45454545454545%" headers="mcps1.3.3.2.2.2.1.1.1.3.3.2.4.1.2 "><p id="mrs_01_1060__p0248123815017">Specifies whether to enable the SSL authentication. (You are advised to enable this function to ensure security.)</p>
|
|
<p id="mrs_01_1060__p424810385017">Only Sources of the Avro type have this configuration item.</p>
|
|
<ul id="mrs_01_1060__ul19248133814010"><li id="mrs_01_1060__li32488382011"><strong id="mrs_01_1060__b1874554038">true</strong> indicates that the function is enabled.</li><li id="mrs_01_1060__li1524815387018"><strong id="mrs_01_1060__b699998174">false</strong> indicates that the function is not enabled.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="31.313131313131308%" headers="mcps1.3.3.2.2.2.1.1.1.3.3.2.4.1.3 "><p id="mrs_01_1060__p624815381300">false</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="mrs_01_1060__l8787b89531ab4cd1a676c5414a477fb9">Click <strong id="mrs_01_1060__b19559419681035">Export</strong> to save the <strong id="mrs_01_1060__b9901804491035">properties.properties</strong> configuration file to the local server.</li></ol>
|
|
</li><li id="mrs_01_1060__lcc8b162262da4c1fbc1d81471491b9bf">Log in to FusionInsight Manager and choose <strong id="mrs_01_1060__b27329185171">Cluster</strong> > <strong id="mrs_01_1060__b17529123817196">Services</strong> > <strong id="mrs_01_1060__b1643934301915">Flume</strong>. On the <strong id="mrs_01_1060__b14483174881920">Instances</strong> tab page, click <strong id="mrs_01_1060__b634411573197">Flume</strong>.</li><li id="mrs_01_1060__l1161c4a409f147619d9777f40a7a3683">Select the Flume role of the node where the configuration file is to be uploaded, choose <strong id="mrs_01_1060__b4860754211036">Instance Configurations</strong> > <strong id="mrs_01_1060__b21286565811036">Import</strong> beside the <span class="parmname" id="mrs_01_1060__parmname14000099201036"><b>flume.config.file</b></span>, and select the <strong id="mrs_01_1060__b3904656411036">properties.properties</strong> file.<div class="note" id="mrs_01_1060__n7d83ba7368a7433984a0625dfea03500"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_1060__u1fb8afeba96d47da992fc20b6e8212d8"><li class="text" id="mrs_01_1060__l63941783c6e141be83bb589743f3fb8a">An independent server configuration file can be uploaded to each Flume instance.</li><li class="text" id="mrs_01_1060__l1d035f47eef546c397c051fe96bfdfb9">This step is required for updating the configuration file. Modifying the configuration file on the background is an improper operation because the modification will be overwritten after configuration synchronization.</li></ul>
|
|
</div></div>
|
|
</li><li id="mrs_01_1060__ledb4b015c8a4449b9db31de33960b6c0">Click <strong id="mrs_01_1060__b7404234441036">Save</strong>, and then click <strong id="mrs_01_1060__b10852687471036">OK</strong>.</li><li id="mrs_01_1060__le5826f44d69c44e190606472518934d6">Click <strong id="mrs_01_1060__b6328728401036">Finish</strong>.</li></ol>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1059.html">Non-Encrypted Transmission</a></div>
|
|
</div>
|
|
</div>
|
|
|