forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
39 lines
2.4 KiB
HTML
39 lines
2.4 KiB
HTML
<a name="mrs_01_1799"></a><a name="mrs_01_1799"></a>
|
|
|
|
<h1 class="topictitle1">MapReduce Job Failed in Multiple NameService Environment</h1>
|
|
<div id="body1596092470816"><div class="section" id="mrs_01_1799__s90f17f4a33df43808ca34ad64490c038"><h4 class="sectiontitle">Question</h4><p id="mrs_01_1799__ae0c7a3ee8f6348928cf9265e92ad9847">MapReduce or Yarn job fails in multiple nameService environment using viewFS.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1799__s2d3b00cf5caf426194592c0e10a08972"><h4 class="sectiontitle">Answer</h4><p id="mrs_01_1799__a050931d5e67d48159868f5a1d646f523">When using viewFS only the mount directories are accessible, so the most possible cause is that the path configured is not in one of the mounted paths. For example:</p>
|
|
<pre class="screen" id="mrs_01_1799__s73a35f1b7d07448db3119311061a2993"><property>
|
|
<name>fs.defaultFS</name>
|
|
<value>viewfs://ClusterX/</value>
|
|
</property>
|
|
<property>
|
|
<name>fs.viewfs.mounttable.ClusterX.link./folder1</name>
|
|
<value>hdfs://NS1/folder1</value>
|
|
</property>
|
|
<property>
|
|
<name>fs.viewfs.mounttable.ClusterX.link./folder2</name>
|
|
<value>hdfs://NS2/folder2</value>
|
|
</property></pre>
|
|
<p id="mrs_01_1799__a4669189c41c145b7bd17511c9e814d0b">For all the MR properties which depends on HDFS, should use the paths inside mount folders.</p>
|
|
<p id="mrs_01_1799__a6ae17a1fa2164b09a89e089bfb9a6a5b"><strong id="mrs_01_1799__a99b32c7af2de44d08601c26581260de1">Incorrect:</strong></p>
|
|
<pre class="screen" id="mrs_01_1799__s412f58d64db84000ae0eeae25a6b7477"><property>
|
|
<name>yarn.app.mapreduce.am.staging-dir</name>
|
|
<value>/tmp/hadoop-yarn/staging</value>
|
|
</property></pre>
|
|
<p id="mrs_01_1799__a3cddb6ab38044d47af02cf242f0632d3">As the root folder (/) is not accessible in viewFS.</p>
|
|
<p id="mrs_01_1799__afa8266ae7b43493b987ad2cb0676fe8d"><strong id="mrs_01_1799__a5837476ae700494e9f688ebfcebfeaaf">Correct:</strong></p>
|
|
<pre class="screen" id="mrs_01_1799__sd7be1f993c2b45a1ae6b836ad97acaa7"><property>
|
|
<name>yarn.app.mapreduce.am.staging-dir</name>
|
|
<value>/folder1/tmp/hadoop-yarn/staging</value>
|
|
</property></pre>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1788.html">Common Issues About MapReduce</a></div>
|
|
</div>
|
|
</div>
|
|
|