全部產品
Search
文件中心

Realtime Compute for Apache Flink:Window Top-N

更新時間:Jul 18, 2024

Window TOP-N需要同時遵循Window TVF和Top-N兩者的修改要求,支援的相容性修改較少。本文為您介紹Window Top-N變更的可相容性和不可相容性詳情。

可相容的變更

  • 新增、刪除window屬性欄位,該修改屬於完全相容變更。

    -- 原始SQL。
    select a, b, c, window_start from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, window_start, window_end)
      ) where rk < 3;
    
    
    -- 新增window end,該修改屬於完全相容變更。
    select a, b, c, window_start, window_end from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, window_start, window_end)
      ) where rk < 3;
  • 是否輸出rank number的值,不影響相容性,該修改屬於完全相容變更。

    -- 原始SQL。
    select a, b, c, window_start from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, window_start, window_end)
      ) where rk < 3;
    
    
    -- 輸入rank number值:rk,該修改屬於完全相容變更。
    select a, b, c, window_start, rk from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, window_start, window_end)
      ) where rk < 3;
  • Partition Key順序變化,該修改屬於完全相容變更。

    -- 原始SQL。
    select a, b, c, window_start from (
      select *,
          row_number() over (partition by a, b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, b, window_start, window_end)
      ) where rk < 3;
    
    -- 修改Partition Key順序,屬於完全相容變更。
    select a, b, c, window_start from (
      select *,
          row_number() over (partition by b, a, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, b, window_start, window_end)
      ) where rk < 3;

不相容的變更

  • 修改window相關屬性(window 的類型,window 的大小,時間相關屬性),該修改屬於不相容變更。

    樣本可參見Window TVF修改window相關屬性樣本,詳情請參見不相容的變更

  • 新增、刪除、修改統計維度(group key)或者統計維度涉及欄位的計算邏輯發生變化,該修改屬於不相容變更。

    樣本可參見Window TVF修改group key樣本,詳情請參見不相容的變更

  • 新增、刪除、修改統計指標,該修改屬於不相容變更。TOP-N的輸入發生變化,該修改屬於不相容變更。

    -- 原始SQL。
    select a, b, c, window_start from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, window_start, window_end)
      ) where rk < 3;
    
    -- 新增統計指標:min(d) as d,屬於不相容修改。
    -- 該修改會導致TOP-N的輸入發生變化。
    select a, b, c, d, window_start from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
          min(d) as d,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, window_start, window_end)
      ) where rk < 3;
  • 新增、刪除、修改partition by key或者partition by key涉及欄位的計算邏輯發生變化,該修改屬於不相容變更。

    樣本可參見Top N修改partition by key樣本,詳情請參見不相容的變更

  • 修改order by相關屬性(排序欄位和方向),該修改屬於不相容修改。

    樣本可參見Top N修改order by key樣本,詳情請參見不相容的變更

  • 修改rank range的值(Top N中N 的值),則該修改屬於不相容修改。

    樣本可參見Top N修改rank range樣本,詳情請參見不相容的變更

  • 修改group key順序,視窗相關的group key順序發生變化,但是普通group key相對順序保持不變。或者普通group key順序發生變化,但視窗group key相對順序不變,都屬於不相容變更。

    -- 原始SQL。
    select a, b, c, window_start from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, b, window_start, window_end)
      ) where rk < 3;
    
    -- 視窗相關的group key順序變化,但是普通group key相對順序不變,屬於不相容變更。
    -- 改變Window Rank的Schema
    select a, b, c, window_start from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by a, b, window_end, window_start)
      ) where rk < 3;
    
    -- 普通group key順序發生變化,但視窗group key相對順序不變,屬於不相容變更。
    select a, b, c, window_start from (
      select *,
          row_number() over (partition by b, window_start, window_end order by c) as rk
      from (
        select a,
          sum(b) as b,
          max(c) as c,
            window_start,
            window_end
        from table (tumble(table MyTable, descriptor(ts), interval '1' minute))
          group by b, a, window_start, window_end)
      ) where rk < 3;