forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
27 lines
7.6 KiB
HTML
27 lines
7.6 KiB
HTML
<a name="mrs_01_24117"></a><a name="mrs_01_24117"></a>
|
|
|
|
<h1 class="topictitle1">Hive Configuration Problems</h1>
|
|
<div id="body32001227"><ul id="mrs_01_24117__en-us_topic_0000001173630694_ul74251315143915"><li id="mrs_01_24117__en-us_topic_0000001173630694_li7425161510399">The error message "java.lang.OutOfMemoryError: Java heap space." is displayed during Hive SQL execution.<p id="mrs_01_24117__en-us_topic_0000001173630694_p94761402111"><a name="mrs_01_24117__en-us_topic_0000001173630694_li7425161510399"></a><a name="en-us_topic_0000001173630694_li7425161510399"></a>Solution:</p>
|
|
<ul id="mrs_01_24117__en-us_topic_0000001173630694_ul1982154616385"><li id="mrs_01_24117__en-us_topic_0000001173630694_li182346193818">For MapReduce tasks, increase the values of the following parameters:<p id="mrs_01_24117__en-us_topic_0000001173630694_p11983157175111"><a name="mrs_01_24117__en-us_topic_0000001173630694_li182346193818"></a><a name="en-us_topic_0000001173630694_li182346193818"></a><strong id="mrs_01_24117__en-us_topic_0000001173630694_b161471854103817">set mapreduce.map.memory.mb=8192;</strong></p>
|
|
<p id="mrs_01_24117__en-us_topic_0000001173630694_p139834745118"><strong id="mrs_01_24117__en-us_topic_0000001173630694_b1617705453812">set mapreduce.map.java.opts=-Xmx6554M;</strong></p>
|
|
<p id="mrs_01_24117__en-us_topic_0000001173630694_p498377125116"><strong id="mrs_01_24117__en-us_topic_0000001173630694_b1017855463815">set mapreduce.reduce.memory.mb=8192;</strong></p>
|
|
<p id="mrs_01_24117__en-us_topic_0000001173630694_p1498307185119"><strong id="mrs_01_24117__en-us_topic_0000001173630694_b1417915493817">set mapreduce.reduce.java.opts=-Xmx6554M;</strong></p>
|
|
</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li878017433916">For Tez tasks, increase the value of the following parameter:<p id="mrs_01_24117__en-us_topic_0000001173630694_p79831970516"><a name="mrs_01_24117__en-us_topic_0000001173630694_li878017433916"></a><a name="en-us_topic_0000001173630694_li878017433916"></a><strong id="mrs_01_24117__en-us_topic_0000001173630694_b076108183911">set hive.tez.container.size=8192;</strong></p>
|
|
</li></ul>
|
|
</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li542571511399">After a column name is changed to a new one using the Hive SQL <strong id="mrs_01_24117__en-us_topic_0000001173630694_b2071184945018">as</strong> statement, the error message "Invalid table alias or column reference 'xxx'." is displayed when the original column name is used for compilation.<p id="mrs_01_24117__en-us_topic_0000001173630694_p1598367115115">Solution: Run the <strong id="mrs_01_24117__en-us_topic_0000001173630694_b3848147204011">set hive.cbo.enable=true;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li542517150394">The error message "Unsupported SubQuery Expression 'xxx': Only SubQuery expressions that are top level conjuncts are allowed." is displayed during Hive SQL subquery compilation.<p id="mrs_01_24117__en-us_topic_0000001173630694_p1998314755117"><a name="mrs_01_24117__en-us_topic_0000001173630694_li542517150394"></a><a name="en-us_topic_0000001173630694_li542517150394"></a>Solution: Run the <strong id="mrs_01_24117__en-us_topic_0000001173630694_b89197114112">set hive.cbo.enable=true</strong><strong id="mrs_01_24117__en-us_topic_0000001173630694_b5884292416">;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li114256155395">The error message "CalciteSubquerySemanticException [Error 10249]: Unsupported SubQuery Expression Currently SubQuery expressions are only allowed as Where and Having Clause predicates." is displayed during Hive SQL subquery compilation.<p id="mrs_01_24117__en-us_topic_0000001173630694_p1498319715510"><a name="mrs_01_24117__en-us_topic_0000001173630694_li114256155395"></a><a name="en-us_topic_0000001173630694_li114256155395"></a>Solution: Run the <strong id="mrs_01_24117__en-us_topic_0000001173630694_b66495485610">set hive.cbo.enable=true;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li1542531519391">The error message "Error running query: java.lang.AssertionError: Cannot add expression of different type to set." is displayed during Hive SQL compilation.<p id="mrs_01_24117__en-us_topic_0000001173630694_p19832713516"><a name="mrs_01_24117__en-us_topic_0000001173630694_li1542531519391"></a><a name="en-us_topic_0000001173630694_li1542531519391"></a>Solution: Run the <strong id="mrs_01_24117__en-us_topic_0000001173630694_b0912144220569">set hive.cbo.enable=false;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li12425115153916">The error message "java.lang.NullPointerException at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeStats$GenericUDAFNumericStatsEvaluator.init." is displayed during Hive SQL execution.<p id="mrs_01_24117__en-us_topic_0000001173630694_p18984173514"><a name="mrs_01_24117__en-us_topic_0000001173630694_li12425115153916"></a><a name="en-us_topic_0000001173630694_li12425115153916"></a>Solution: Run the <strong id="mrs_01_24117__en-us_topic_0000001173630694_b194101485578">set hive.map.aggr=false;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li1042517156392">When <strong id="mrs_01_24117__en-us_topic_0000001173630694_b390832513585">hive.auto.convert.join</strong> is set to <strong id="mrs_01_24117__en-us_topic_0000001173630694_b83481428135814">true</strong> (enabled by default) and <strong id="mrs_01_24117__en-us_topic_0000001173630694_b43561042115812">hive.optimize.skewjoin</strong> is set to <strong id="mrs_01_24117__en-us_topic_0000001173630694_b1538434718587">true</strong>, the error message "ClassCastException org.apache.hadoop.hive.ql.plan.ConditionalWork cannot be cast to org.apache.hadoop.hive.ql.plan.MapredWork" is displayed.<p id="mrs_01_24117__en-us_topic_0000001173630694_p99846713516">Solution: Run the <strong id="mrs_01_24117__en-us_topic_0000001173630694_b10656858165910">set hive.optimize.skewjoin=false;</strong> statement.</p>
|
|
</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li154251515163911">When <strong id="mrs_01_24117__en-us_topic_0000001173630694_b45115261904">hive.auto.convert.join</strong> is set to <strong id="mrs_01_24117__en-us_topic_0000001173630694_b02123291907">true</strong> (enabled by default), <strong id="mrs_01_24117__en-us_topic_0000001173630694_b1937311359016">hive.optimize.skewjoin</strong> is set to <strong id="mrs_01_24117__en-us_topic_0000001173630694_b1091110360012">true</strong>, and <strong id="mrs_01_24117__en-us_topic_0000001173630694_b15381105011013">hive.exec.parallel</strong> is set to <strong id="mrs_01_24117__en-us_topic_0000001173630694_b36217132117">true</strong>, the error message "java.io.FileNotFoundException: File does not exist:xxx/reduce.xml" is displayed.<p id="mrs_01_24117__en-us_topic_0000001173630694_p121221524184216">Solution:</p>
|
|
<ul id="mrs_01_24117__en-us_topic_0000001173630694_ul106001291423"><li id="mrs_01_24117__en-us_topic_0000001173630694_li1985053411421">Method 1: Switch the execution engine to Tez. For details, see <a href="mrs_01_1750.html">Switching the Hive Execution Engine to Tez</a>.</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li13425165612423">Method 2: Run the <strong id="mrs_01_24117__en-us_topic_0000001173630694_b1961610306517">set hive.exec.parallel=false;</strong> statement.</li><li id="mrs_01_24117__en-us_topic_0000001173630694_li169581526194216">Method 3: Run the <strong id="mrs_01_24117__en-us_topic_0000001173630694_b437983717219">set hive.auto.convert.join=false;</strong> statement.</li></ul>
|
|
</li></ul>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1752.html">Common Issues About Hive</a></div>
|
|
</div>
|
|
</div>
|
|
|