How to decompose "from-to term" record by each date record in BigQuery

How to decompose "from-to term" record by each date record in BigQuery

How can I decompose "from-to term" record by each date record ?

BigQuery(BQ) is very useful for data analysis or processing.
It is good to handle huge data.
It returns summary result in short time.

BQ can handle date data.
Then sometimes we handle "from-to" record like below.

date_fromdate_to
2021-11-012021-11-05

Once we get this record, we would like to convert it to each date records like below.

date
2021-11-01
2021-11-02
2021-11-03
2021-11-04
2021-11-05

So how can we decompose "from-to term" record by each date record ?

So today I introduce about "How to decompose "from-to term" record by each date record in BigQuery".

目次

How to decompose "from-to term" record by each date record in BigQuery

In order to decompose "from-to term" to each date record in BigQuery, we should change our mind.

Instead of "decompose", we do this.

Generate date records and filter it by "from-to".

Let's try it.

Generate date records and filter it by "from-to"

First, generate each date records.

SQL is below.

WITH
  t1 AS (
  SELECT
    date_seq
  FROM
    UNNEST(GENERATE_DATE_ARRAY('2021-01-01', '2021-01-31')) AS date_seq
  )
SELECT
  *
FROM
  t1

Then we get this result.

Result

Rowdate_seq
12021-01-01
22021-01-02
32021-01-03
42021-01-04
52021-01-05
62021-01-06
72021-01-07
82021-01-08
92021-01-09
102021-01-10
112021-01-11
122021-01-12
132021-01-13
142021-01-14
152021-01-15
162021-01-16
172021-01-17
182021-01-18
192021-01-19
202021-01-20
212021-01-21
222021-01-22
232021-01-23
242021-01-24
252021-01-25
262021-01-26
272021-01-27
282021-01-28
292021-01-29
302021-01-30
312021-01-31

In order to make continuous date records, we can use GENERATE_DATE_ARRAY() function.

Then use UNNEST to decompose array to each record.

Array functions  |  BigQuery  |  Google Cloud

Next, filter it with using "from-to" term.

WITH
  t1 AS (
  SELECT
    date_seq
  FROM
    UNNEST(GENERATE_DATE_ARRAY('2021-01-01', '2021-01-31')) AS date_seq
  ),
  t2 AS (
  SELECT
    DATE("2021-01-10") AS date_from,
    DATE("2021-01-20") AS date_to
  )
SELECT
  *
FROM
  t1,
  t2
WHERE
  t1.date_seq BETWEEN t2.date_from
  AND t2.date_to

Result

Rowdate_seqdate_fromdate_to
12021-01-102021-01-102021-01-20
22021-01-112021-01-102021-01-20
32021-01-122021-01-102021-01-20
42021-01-132021-01-102021-01-20
52021-01-142021-01-102021-01-20
62021-01-152021-01-102021-01-20
72021-01-162021-01-102021-01-20
82021-01-172021-01-102021-01-20
92021-01-182021-01-102021-01-20
102021-01-192021-01-102021-01-20
112021-01-202021-01-102021-01-20

Now we decomposed "from-to term" record by each date record.

Conclusion

Today I explained about "How to decompose "from-to term" record by each date record in BigQuery".

In order to each date record, we need to change "decompose" mind and think like below.

Generate date records and filter it by "from-to".

With using GENERATE_DATE_ARRAY() and UNNEST, make date records.

Then filter it by between clause.

Instead of decomposing, we filter it. It is simple.

Yes. It is easy trick.

How to decompose "from-to term" record by each date record in BigQuery

この記事が気に入ったら
いいね または フォローしてね!

If you like this article, please share !
  • URLをコピーしました!
  • URLをコピーしました!

Author

karasanのアバター karasan System engineer

Mid-career engineer (AI, Data science, Salesforce, etc.).
Good at Python and SQL.

目次