forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
20 lines
2.8 KiB
HTML
20 lines
2.8 KiB
HTML
<a name="mrs_01_2055"></a><a name="mrs_01_2055"></a>
|
|
|
|
<h1 class="topictitle1">Why the Job Information Obtained from the restful Interface of an Ended Spark Application Is Incorrect?</h1>
|
|
<div id="body1595920225316"><div class="section" id="mrs_01_2055__sa85f95303ed84023a630b97647ce9375"><h4 class="sectiontitle">Question</h4><p id="mrs_01_2055__ace986a6756d04610a8a73c8072bf2323">The job information obtained from the restful interface of an ended Spark application is incorrect: the value of <span class="parmname" id="mrs_01_2055__p91eed999cdac4788bb12ad0304e3f8d0"><b>numActiveTasks</b></span> is negative, as shown in <a href="#mrs_01_2055__f25d2e3813ec3476b9a49627d17776421">Figure 1</a>:</p>
|
|
<div class="fignone" id="mrs_01_2055__f25d2e3813ec3476b9a49627d17776421"><a name="mrs_01_2055__f25d2e3813ec3476b9a49627d17776421"></a><a name="f25d2e3813ec3476b9a49627d17776421"></a><span class="figcap"><b>Figure 1 </b>job information</span><br><span><img id="mrs_01_2055__i13ca13252e4d4983954e112f3f537e23" src="en-us_image_0000001349290529.png"></span></div>
|
|
<div class="note" id="mrs_01_2055__na2897dd9524244fd8d25cdf8b6336eed"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_2055__a5274c175aa9f4a2391d8025a49c77289">numActiveTasks indicates the number of active tasks.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_2055__sf411ab093c4c4296bd4853815d1b4d0c"><h4 class="sectiontitle">Answer</h4><div class="p" id="mrs_01_2055__a399d992b826a4ae3a5b0518fec6e10b8">The job information can be obtained in either of the following methods:<ul id="mrs_01_2055__u8a5f2ebc9b004aa4a586d7d937e51f98"><li id="mrs_01_2055__l725881fb4656435783773fa759125258">Set <strong id="mrs_01_2055__b2362192518141">spark.history.briefInfo.gather=true</strong> and then view the brief JobHistory information.</li><li id="mrs_01_2055__l3c0a929b1f754d029fa370f14120c74a">Visit the JobHistory2x page of Spark (URL: https://IP:port/api/v1/<appid>/jobs/).</li></ul>
|
|
</div>
|
|
<p id="mrs_01_2055__aff254036acd7481eb4735feb61ba22bc">The value of <span class="parmname" id="mrs_01_2055__pb6a7003826194c758cfe308c7885e947"><b>numActiveTasks</b></span> in the job information is calculated from the difference between the number of SparkListenerTaskStart events and the number of SparkListenerTaskEnd events in the <span class="filepath" id="mrs_01_2055__fe2fc2da00af842f5a8df4c1d9d828a76"><b>eventLog</b></span> file. If some events are not recorded in the <span class="filepath" id="mrs_01_2055__f5425702cc2f54fecb9d405b667368a25"><b>eventLog</b></span> file, the job information obtained from the restful interface is incorrect.</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_2002.html">Common Issues About Spark2x</a></div>
|
|
</div>
|
|
</div>
|
|
|