forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
28 lines
6.1 KiB
HTML
28 lines
6.1 KiB
HTML
<a name="mrs_01_24117"></a><a name="mrs_01_24117"></a>
|
|
|
|
<h1 class="topictitle1">Hive Configuration Problems</h1>
|
|
<div id="body0000001104844650"><ul id="mrs_01_24117__ul74251315143915"><li id="mrs_01_24117__li7425161510399">The error message "java.lang.OutOfMemoryError: Java heap space." is displayed during Hive SQL execution.<p id="mrs_01_24117__p94761402111"><a name="mrs_01_24117__li7425161510399"></a><a name="li7425161510399"></a>Solution:</p>
|
|
<ul id="mrs_01_24117__ul1982154616385"><li id="mrs_01_24117__li182346193818">For MapReduce tasks, increase the values of the following parameters:<p id="mrs_01_24117__p11983157175111"><a name="mrs_01_24117__li182346193818"></a><a name="li182346193818"></a><strong id="mrs_01_24117__b161471854103817">set mapreduce.map.memory.mb=8192;</strong></p>
|
|
<p id="mrs_01_24117__p139834745118"><strong id="mrs_01_24117__b1617705453812">set mapreduce.map.java.opts=-Xmx6554M;</strong></p>
|
|
<p id="mrs_01_24117__p498377125116"><strong id="mrs_01_24117__b1017855463815">set mapreduce.reduce.memory.mb=8192;</strong></p>
|
|
<p id="mrs_01_24117__p1498307185119"><strong id="mrs_01_24117__b1417915493817">set mapreduce.reduce.java.opts=-Xmx6554M;</strong></p>
|
|
</li><li id="mrs_01_24117__li878017433916">For Tez tasks, increase the value of the following parameter:<p id="mrs_01_24117__p79831970516"><a name="mrs_01_24117__li878017433916"></a><a name="li878017433916"></a><strong id="mrs_01_24117__b076108183911">set hive.tez.container.size=8192;</strong></p>
|
|
</li></ul>
|
|
</li><li id="mrs_01_24117__li542571511399">After a column name is changed to a new one using the Hive SQL <strong id="mrs_01_24117__b2071184945018">as</strong> statement, the error message "Invalid table alias or column reference 'xxx'." is displayed when the original column name is used for compilation.<p id="mrs_01_24117__p1598367115115">Solution: Run the <strong id="mrs_01_24117__b3848147204011">set hive.cbo.enable=true;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__li542517150394">The error message "Unsupported SubQuery Expression 'xxx': Only SubQuery expressions that are top level conjuncts are allowed." is displayed during Hive SQL subquery compilation.<p id="mrs_01_24117__p1998314755117"><a name="mrs_01_24117__li542517150394"></a><a name="li542517150394"></a>Solution: Run the <strong id="mrs_01_24117__b89197114112">set hive.cbo.enable=true</strong><strong id="mrs_01_24117__b5884292416">;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__li114256155395">The error message "CalciteSubquerySemanticException [Error 10249]: Unsupported SubQuery Expression Currently SubQuery expressions are only allowed as Where and Having Clause predicates." is displayed during Hive SQL subquery compilation.<p id="mrs_01_24117__p1498319715510"><a name="mrs_01_24117__li114256155395"></a><a name="li114256155395"></a>Solution: Run the <strong id="mrs_01_24117__b66495485610">set hive.cbo.enable=true;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__li1542531519391">The error message "Error running query: java.lang.AssertionError: Cannot add expression of different type to set." is displayed during Hive SQL compilation.<p id="mrs_01_24117__p19832713516"><a name="mrs_01_24117__li1542531519391"></a><a name="li1542531519391"></a>Solution: Run the <strong id="mrs_01_24117__b0912144220569">set hive.cbo.enable=false;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__li12425115153916">The error message "java.lang.NullPointerException at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeStats$GenericUDAFNumericStatsEvaluator.init." is displayed during Hive SQL execution.<p id="mrs_01_24117__p18984173514"><a name="mrs_01_24117__li12425115153916"></a><a name="li12425115153916"></a>Solution: Run the <strong id="mrs_01_24117__b194101485578">set hive.map.aggr=false;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__li1042517156392">When <strong id="mrs_01_24117__b390832513585">hive.auto.convert.join</strong> is set to <strong id="mrs_01_24117__b83481428135814">true</strong> (enabled by default) and <strong id="mrs_01_24117__b43561042115812">hive.optimize.skewjoin</strong> is set to <strong id="mrs_01_24117__b1538434718587">true</strong>, the error message "ClassCastException org.apache.hadoop.hive.ql.plan.ConditionalWork cannot be cast to org.apache.hadoop.hive.ql.plan.MapredWork" is displayed.<p id="mrs_01_24117__p99846713516">Solution: Run the <strong id="mrs_01_24117__b10656858165910">set hive.optimize.skewjoin=false;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__li154251515163911">When <strong id="mrs_01_24117__b45115261904">hive.auto.convert.join</strong> is set to <strong id="mrs_01_24117__b02123291907">true</strong> (enabled by default), <strong id="mrs_01_24117__b1937311359016">hive.optimize.skewjoin</strong> is set to <strong id="mrs_01_24117__b1091110360012">true</strong>, and <strong id="mrs_01_24117__b15381105011013">hive.exec.parallel</strong> is set to <strong id="mrs_01_24117__b36217132117">true</strong>, the error message "java.io.FileNotFoundException: File does not exist:xxx/reduce.xml" is displayed.<p id="mrs_01_24117__p121221524184216">Solution:</p>
|
|
<ul id="mrs_01_24117__ul106001291423"><li id="mrs_01_24117__li1985053411421">Method 1: Switch the execution engine to Tez. For details, see <a href="mrs_01_1750.html">Switching the Hive Execution Engine to Tez</a>.</li><li id="mrs_01_24117__li13425165612423">Method 2: Run the <strong id="mrs_01_24117__b1961610306517">set hive.exec.parallel=false;</strong> statement.</li><li id="mrs_01_24117__li169581526194216">Method 3: Run the <strong id="mrs_01_24117__b437983717219">set hive.auto.convert.join=false;</strong> statement.</li></ul>
|
|
</li><li id="mrs_01_24117__li1435993319332">Eerror message "NullPointerException at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.mergeJoinComputeKeys" is displayed when Hive on Tez executes bucket map join.<p id="mrs_01_24117__p3368163310332"><a name="mrs_01_24117__li1435993319332"></a><a name="li1435993319332"></a>Solution: Run the <strong id="mrs_01_24117__b1145014376214">set tez.am.container.reuse.enabled=false;</strong> statement.</p>
|
|
</li></ul>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1752.html">Common Issues About Hive</a></div>
|
|
</div>
|
|
</div>
|
|
|