forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
39 lines
2.8 KiB
HTML
39 lines
2.8 KiB
HTML
<a name="mrs_01_1692"></a><a name="mrs_01_1692"></a>
|
|
|
|
<h1 class="topictitle1">Why MapReduce Tasks Fails in the Environment with Multiple NameServices?</h1>
|
|
<div id="body8662426"><div class="section" id="mrs_01_1692__en-us_topic_0000001173949260_sec27629abc8c4b8aa175d2b22f2e74f7"><h4 class="sectiontitle">Question</h4><p id="mrs_01_1692__en-us_topic_0000001173949260_a3bf9a68b20d04306aa264b1e7acd184d">Why MapReduce or Yarn tasks using the viewFS function fail to be executed in the environment with multiple NameServices?</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1692__en-us_topic_0000001173949260_s758ad8f50e3b49ef9aaa033745f8f004"><h4 class="sectiontitle">Answer</h4><p id="mrs_01_1692__en-us_topic_0000001173949260_a220960dd334f48bf914d186df845b5bb">When viewFS is used, only directories mounted to viewFS can be accessed. Therefore, the most possible reason is that the configured path is not on the mount point of viewFS. Example:</p>
|
|
<pre class="screen" id="mrs_01_1692__en-us_topic_0000001173949260_se6c01255a43b4bf699e0ffab539b98b5"><property>
|
|
<name>fs.defaultFS</name>
|
|
<value>viewfs://ClusterX/</value>
|
|
</property>
|
|
<property>
|
|
<name>fs.viewfs.mounttable.ClusterX.link./folder1</name>
|
|
<value>hdfs://NS1/folder1</value>
|
|
</property>
|
|
<property>
|
|
<name>fs.viewfs.mounttable.ClusterX.link./folder2</name>
|
|
<value>hdfs://NS2/folder2</value>
|
|
</property></pre>
|
|
<p id="mrs_01_1692__en-us_topic_0000001173949260_a15436f8e049544e2be04f603b3dab943">In the MR configuration that depends on the HDFS, the mounted directory needs to be used.</p>
|
|
<p id="mrs_01_1692__en-us_topic_0000001173949260_a5db5a2d1640845ccb1dbbe74966418ba"><strong id="mrs_01_1692__en-us_topic_0000001173949260_b31175217619">Incorrect example</strong>:</p>
|
|
<pre class="screen" id="mrs_01_1692__en-us_topic_0000001173949260_s7a41e5fb136d482a8bd48fd2d0a41149"><property>
|
|
<name>yarn.app.mapreduce.am.staging-dir</name>
|
|
<value>/tmp/hadoop-yarn/staging</value>
|
|
</property></pre>
|
|
<p id="mrs_01_1692__en-us_topic_0000001173949260_a115f8e34f06e43b390cf9f202d898378">The root directory (/) cannot be accessed in viewFS.</p>
|
|
<p id="mrs_01_1692__en-us_topic_0000001173949260_a5add9087cb28418f8bfca69e484426ae"><strong id="mrs_01_1692__en-us_topic_0000001173949260_b829132519615">Correct example</strong>:</p>
|
|
<pre class="screen" id="mrs_01_1692__en-us_topic_0000001173949260_s1e6bad63cc7b47feace42139e7bd80ea"><property>
|
|
<name>yarn.app.mapreduce.am.staging-dir</name>
|
|
<value>/folder1/tmp/hadoop-yarn/staging</value>
|
|
</property></pre>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1690.html">FAQ</a></div>
|
|
</div>
|
|
</div>
|
|
|