forked from docs/doc-exports
Su, Xiaomeng
76a5b1ee83
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com> Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
114 lines
11 KiB
HTML
114 lines
11 KiB
HTML
<a name="dli_spark_lead"></a><a name="dli_spark_lead"></a>
|
|
|
|
<h1 class="topictitle1">lead</h1>
|
|
<div id="body8662426"><p id="dli_spark_lead__en-us_topic_0000001703779925_p18471163815215">This function is used to return the value of the <em id="dli_spark_lead__en-us_topic_0000001703779925_i15330123864410">n</em>th row downwards within a specified window.</p>
|
|
<div class="section" id="dli_spark_lead__en-us_topic_0000001703779925_section6750104885712"><h4 class="sectiontitle">Restrictions</h4><p id="dli_spark_lead__en-us_topic_0000001703779925_p1465134511259">The restrictions on using window functions are as follows:</p>
|
|
<ul id="dli_spark_lead__en-us_topic_0000001703779925_ul246534592519"><li id="dli_spark_lead__en-us_topic_0000001703779925_li1446514452259">Window functions can be used only in select statements.</li><li id="dli_spark_lead__en-us_topic_0000001703779925_li346519455256">Window functions and aggregate functions cannot be nested in window functions.</li><li id="dli_spark_lead__en-us_topic_0000001703779925_li546514532510">Window functions cannot be used together with aggregate functions of the same level.</li></ul>
|
|
</div>
|
|
<div class="section" id="dli_spark_lead__en-us_topic_0000001703779925_section45291954203217"><h4 class="sectiontitle">Syntax</h4><pre class="screen" id="dli_spark_lead__en-us_topic_0000001703779925_screen946684512254">lead(<expr>[, bigint <offset>[, <default>]]) over([partition_clause] orderby_clause)</pre>
|
|
</div>
|
|
<div class="section" id="dli_spark_lead__en-us_topic_0000001703779925_section992014913317"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_spark_lead__en-us_topic_0000001703779925_table837615402204" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="dli_spark_lead__en-us_topic_0000001703779925_row153777401208"><th align="left" class="cellrowborder" valign="top" width="22.759999999999998%" id="mcps1.3.4.2.2.4.1.1"><p id="dli_spark_lead__en-us_topic_0000001703779925_p133778403209">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="15.920000000000002%" id="mcps1.3.4.2.2.4.1.2"><p id="dli_spark_lead__en-us_topic_0000001703779925_p23771240202011">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="61.31999999999999%" id="mcps1.3.4.2.2.4.1.3"><p id="dli_spark_lead__en-us_topic_0000001703779925_p12377134082019">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dli_spark_lead__en-us_topic_0000001703779925_row9377164012206"><td class="cellrowborder" valign="top" width="22.759999999999998%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p13771040152017">expr</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p537712407206">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="61.31999999999999%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p103773403209">Expression whose return result is to be calculated</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_spark_lead__en-us_topic_0000001703779925_row1578218624013"><td class="cellrowborder" valign="top" width="22.759999999999998%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p1437714013203">offset</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p1037744013209">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="61.31999999999999%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p183777400203">Offset. It is a constant of the BIGINT type and its value is greater than or equal to 0. The value <strong id="dli_spark_lead__en-us_topic_0000001703779925_b1493847654114353">0</strong> indicates the current row, the value <strong id="dli_spark_lead__en-us_topic_0000001703779925_b1352747664114353">1</strong> indicates the previous row, and so on. The default value is <strong id="dli_spark_lead__en-us_topic_0000001703779925_b1365689406114356">1</strong>. If the input value is of the STRING or DOUBLE type, it is implicitly converted to the BIGINT type before calculation.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_spark_lead__en-us_topic_0000001703779925_row4273191014405"><td class="cellrowborder" valign="top" width="22.759999999999998%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p123778405203">default</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p8377164011209">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="61.31999999999999%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p8168136112011">Constant. The default value is <strong id="dli_spark_lead__en-us_topic_0000001703779925_b1085013072114424">NULL</strong>.</p>
|
|
<p id="dli_spark_lead__en-us_topic_0000001703779925_p103771740162014">Default value when the range specified by <strong id="dli_spark_lead__en-us_topic_0000001703779925_b1595815531114512">offset</strong> is out of range. The value must be the same as the data type corresponding to <strong id="dli_spark_lead__en-us_topic_0000001703779925_b1730392883114512">expr</strong>. If <strong id="dli_spark_lead__en-us_topic_0000001703779925_b2102850685114529">expr</strong> is non-constant, the evaluation is performed based on the current row.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_spark_lead__en-us_topic_0000001703779925_row1160161417206"><td class="cellrowborder" valign="top" width="22.759999999999998%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p181421114175510">partition_clause</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p1214218144555">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="61.31999999999999%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p1114218146554">Partition. Rows with the same value in partition columns are considered to be in the same window.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_spark_lead__en-us_topic_0000001703779925_row1879151412218"><td class="cellrowborder" valign="top" width="22.759999999999998%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p12181324182913">orderby_clause</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p91821324192918">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="61.31999999999999%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_spark_lead__en-us_topic_0000001703779925_p418272442914">It is used to specify how data is sorted in a window.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_spark_lead__en-us_topic_0000001703779925_section210162513312"><h4 class="sectiontitle">Return Values</h4><p id="dli_spark_lead__en-us_topic_0000001703779925_p184124317231">The return value is of the data type of the parameter.</p>
|
|
</div>
|
|
<div class="section" id="dli_spark_lead__en-us_topic_0000001703779925_section13277192233920"><h4 class="sectiontitle">Example Code</h4><p id="dli_spark_lead__en-us_topic_0000001703779925_p54661045172519"><strong id="dli_spark_lead__en-us_topic_0000001703779925_b55699091931540">Example data</strong></p>
|
|
<div class="p" id="dli_spark_lead__en-us_topic_0000001703779925_p9466545122513">To help you understand how to use functions, this example provides source data and function examples based on the source data. Run the following command to create the <strong id="dli_spark_lead__en-us_topic_0000001703779925_b7311454083165">logs</strong> table and add data:<pre class="screen" id="dli_spark_lead__en-us_topic_0000001703779925_screen4466164515255">create table logs(
|
|
cookieid string,
|
|
createtime string,
|
|
url string
|
|
)
|
|
STORED AS parquet;</pre>
|
|
</div>
|
|
<p id="dli_spark_lead__en-us_topic_0000001703779925_p246654522515">Adds the following data:</p>
|
|
<pre class="screen" id="dli_spark_lead__en-us_topic_0000001703779925_screen1746614517256">cookie1 2015-04-10 10:00:02 url2
|
|
cookie1 2015-04-10 10:00:00 url1
|
|
cookie1 2015-04-10 10:03:04 url3
|
|
cookie1 2015-04-10 10:50:05 url6
|
|
cookie1 2015-04-10 11:00:00 url7
|
|
cookie1 2015-04-10 10:10:00 url4
|
|
cookie1 2015-04-10 10:50:01 url5
|
|
cookie2 2015-04-10 10:00:02 url22
|
|
cookie2 2015-04-10 10:00:00 url11
|
|
cookie2 2015-04-10 10:03:04 url33
|
|
cookie2 2015-04-10 10:50:05 url66
|
|
cookie2 2015-04-10 11:00:00 url77
|
|
cookie2 2015-04-10 10:10:00 url44
|
|
cookie2 2015-04-10 10:50:01 url55</pre>
|
|
<p id="dli_spark_lead__en-us_topic_0000001703779925_p19467204572516">Groups all records by <strong id="dli_spark_lead__en-us_topic_0000001703779925_b184020160405">cookieid</strong>, sorts them by <strong id="dli_spark_lead__en-us_topic_0000001703779925_b74131674016">createtime</strong> in ascending order, and returns the values of the second and first rows downwards within the specified window. An example command is as follows:</p>
|
|
<pre class="screen" id="dli_spark_lead__en-us_topic_0000001703779925_screen5467114511256">SELECT cookieid, createtime, url,
|
|
LEAD(createtime, 2) OVER(PARTITION BY cookieid ORDER BY createtime) AS next_2_time,
|
|
LEAD(createtime, 1, '1970-01-01 00:00:00') OVER(PARTITION BY cookieid ORDER BY createtime) AS next_1_time
|
|
FROM logs;
|
|
|
|
-- Returned result:
|
|
cookieid createtime url next_2_time next_1_time
|
|
cookie1 2015-04-10 10:00:00 url1 2015-04-10 10:03:04 2015-04-10 10:00:02
|
|
cookie1 2015-04-10 10:00:02 url2 2015-04-10 10:10:00 2015-04-10 10:03:04
|
|
cookie1 2015-04-10 10:03:04 url3 2015-04-10 10:50:01 2015-04-10 10:10:00
|
|
cookie1 2015-04-10 10:10:00 url4 2015-04-10 10:50:05 2015-04-10 10:50:01
|
|
cookie1 2015-04-10 10:50:01 url5 2015-04-10 11:00:00 2015-04-10 10:50:05
|
|
cookie1 2015-04-10 10:50:05 url6 NULL 2015-04-10 11:00:00
|
|
cookie1 2015-04-10 11:00:00 url7 NULL 1970-01-01 00:00:00
|
|
cookie2 2015-04-10 10:00:00 url11 2015-04-10 10:03:04 2015-04-10 10:00:02
|
|
cookie2 2015-04-10 10:00:02 url22 2015-04-10 10:10:00 2015-04-10 10:03:04
|
|
cookie2 2015-04-10 10:03:04 url33 2015-04-10 10:50:01 2015-04-10 10:10:00
|
|
cookie2 2015-04-10 10:10:00 url44 2015-04-10 10:50:05 2015-04-10 10:50:01
|
|
cookie2 2015-04-10 10:50:01 url55 2015-04-10 11:00:00 2015-04-10 10:50:05
|
|
cookie2 2015-04-10 10:50:05 url66 NULL 2015-04-10 11:00:00
|
|
cookie2 2015-04-10 11:00:00 url77 NULL 1970-01-01 00:00:00</pre>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0475.html">Window Functions</a></div>
|
|
</div>
|
|
</div>
|
|
|