Window

GROUP WINDOW

Description

Group Window is defined in GROUP BY. One record is generated from each group. Group Window involves the following functions:

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Calculate the SUM every day (event time).
insert into temp SELECT name,
    TUMBLE_START(ts, INTERVAL '1' DAY) as wStart,
    SUM(amount)
    FROM Orders
    GROUP BY TUMBLE(ts, INTERVAL '1' DAY), name;

//Calculate the SUM every day (processing time). 
insert into temp SELECT name, 
    SUM(amount) 
    FROM Orders 
    GROUP BY TUMBLE(proctime, INTERVAL '1' DAY), name;

//Calculate the SUM over the recent 24 hours every hour (event time).
insert into temp SELECT product, 
    SUM(amount) 
    FROM Orders 
    GROUP BY HOP(ts, INTERVAL '1' HOUR, INTERVAL '1' DAY), product;

//Calculate the SUM of each session and an inactive interval every 12 hours (event time).
insert into temp SELECT name, 
    SESSION_START(ts, INTERVAL '12' HOUR) AS sStart,
    SESSION_END(ts, INTERVAL '12' HOUR) AS sEnd,
    SUM(amount)
    FROM Orders
    GROUP BY SESSION(ts, INTERVAL '12' HOUR), name;

TUMBLE WINDOW Extension

Function

The extension functions of the DLI tumbling window are as follows:
  • Periodical tumbling windows for lower latency

    Before the tumbling window ends, the window can be periodically triggered based on the configured frequency. The compute result from the start to the current time is output, which does not affect the final output. The latest result can be viewed in each period before the window ends.

  • Custom latency for higher data accuracy

    You can set a latency for the end of the window. The output of the window is updated according to the configured latency each time a piece of late data reaches.

Precautions

If you use insert to write results into the sink, the sink must support the upsert mode.

Syntax

TUMBLE(time_attr, window_interval, period_interval, lateness_interval)

Example

If the current time_attr attribute column is testtime and the window interval is 10 seconds, the statement is as follows:
TUMBLE(testtime, INTERVAL '10' SECOND, INTERVAL '10' SECOND, INTERVAL '10' SECOND)

Description

Table 3 Parameter description

Parameter

Description

Format

time_attr

Event time or processing time attribute column

-

window_interval

Duration of the window
  • Format 1: INTERVAL '10' SECOND

    The window interval is 10 seconds. You can change the value as needed.

  • Format 2: INTERVAL '10' MINUTE

    The window interval is 10 minutes. You can change the value as needed.

  • Format 3: INTERVAL '10' DAY

    The window interval is 10 days. You can change the value as needed.

period_interval

Frequency of periodic triggering within the window range. That is, before the window ends, the output result is updated at an interval specified by period_interval from the time when the window starts. If this parameter is not set, the periodic triggering policy is not used by default.

lateness_interval

Time to postpone the end of the window. The system continues to collect the data that reaches the window within lateness_interval after the window ends. The output is updated for each data that reaches the window within lateness_interval.

NOTE:

If the time window is for processing time, lateness_interval does not take effect.
Values of period_interval and lateness_interval cannot be negative numbers.
  • If period_interval is set to 0, periodic triggering is disabled for the window.
  • If lateness_interval is set to 0, the latency after the window ends is disabled.
  • If neither of the two parameters is set, both periodic triggering and latency are disabled and only the regular tumbling window functions are available .
  • If only the latency function needs to be used, set period_interval INTERVAL '0' SECOND.

OVER WINDOW

The difference between Over Window and Group Window is that one record is generated from one row in Over Window.

Syntax

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
SELECT agg1(attr1) OVER (
  [PARTITION BY partition_name]
  ORDER BY proctime|rowtime 
  ROWS  
 BETWEEN (UNBOUNDED|rowCOUNT) PRECEDING AND CURRENT ROW FROM TABLENAME

SELECT agg1(attr1) OVER (
  [PARTITION BY partition_name]
  ORDER BY proctime|rowtime 
  RANGE  
  BETWEEN (UNBOUNDED|timeInterval) PRECEDING AND CURRENT ROW FROM TABLENAME

Description

Table 4 Parameter description

Parameter

Parameter Description

PARTITION BY

Indicates the primary key of the specified group. Each group separately performs calculation.

ORDER BY

Indicates the processing time or event time as the timestamp for data.

ROWS

Indicates the count window.

RANGE

Indicates the time window.

Precautions

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
// Calculate the count and total number from syntax rules enabled to now (in proctime).
insert into temp SELECT name,
    count(amount) OVER (PARTITION BY name ORDER BY proctime RANGE UNBOUNDED preceding) as cnt1,
    sum(amount) OVER (PARTITION BY name ORDER BY proctime RANGE UNBOUNDED preceding) as cnt2
    FROM Orders;
  
//Calculate the count and total number of the recent four records (in proctime).
insert into temp SELECT name,
    count(amount) OVER (PARTITION BY name ORDER BY proctime ROWS BETWEEN 4 PRECEDING AND CURRENT ROW) as cnt1,
    sum(amount) OVER (PARTITION BY name ORDER BY proctime ROWS BETWEEN 4 PRECEDING AND CURRENT ROW) as cnt2
    FROM Orders;

//Calculate the count and total number last 60s (in eventtime). Process the events based on event time, which is the timeattr field in Orders.
insert into temp SELECT name,
    count(amount) OVER (PARTITION BY name ORDER BY timeattr RANGE BETWEEN INTERVAL '60' SECOND PRECEDING AND CURRENT ROW) as cnt1,
    sum(amount) OVER (PARTITION BY name ORDER BY timeattr RANGE BETWEEN INTERVAL '60' SECOND PRECEDING AND CURRENT ROW) as cnt2
    FROM Orders;
Parent topic: Data Manipulation Language (DML)