Extract dates from week numbers on BigQuery - google-bigquery

I have a large data file containing a string type column 'YearMonthWeek'
It contains values such as '20160101' for the first week of January 2016, or '20161040' for the 40th week of the year 2016 apparently falling in October.
Now, I want to convert these strings to actual dates so that every YearMonthWeek value is converted to, say the first day of that week. (Whether that ends up being Monday or Sunday I don't really care).
I tried the following query:
PARSE_TIMESTAMP('%Y%m%W', CAST(YearMonthWeek AS STRING)) AS datefield
(See this documentation for details)
This runs without errors, but returns me the first day of the month for every single entry...
So for example '20160101' and '20160102' both get parsed as 2016-01-01 00:00:00 UTC.
Is this an issue with the PARSE_TIMESTAMP function, or am I missing something?

Try doing something like
DATE_ADD(date_expression, INTERVAL %W WEEK)
Static example:
SELECT
DATE_ADD(
DATE(PARSE_TIMESTAMP('%Y', SUBSTR(CAST('20161252' AS STRING),0,4))),
INTERVAL (CAST(SUBSTR(CAST('20160102' AS STRING),7) AS INT64)) week)
AS datefield
-
Row datefield
1 2016-01-15
You may add something as a margin to it, according to ISO 8601, the first week of the year is the one that contains January 4th. So you could have something like: 4 + 7*($week - 1)

Related

Date Functions Trunc (SysDate)

I am running the below query to get data recorded in the past 24 hours. I need the same data recorded starting midnight (DATE > 12:00 AM) and also data recorded starting beginning of the month. Not sure if using between will work or if there is better option. Any suggestions.
SELECT COUNT(NUM)
FROM TABLE
WHERE
STATUS = 'CNLD'
AND
TRUNC(TO_DATE('1970-01-01','YYYY-MM-DD') + OPEN_DATE/86400) = trunc(sysdate)
Output (Just need Count). OPEN_DATE Data Type is NUMBER. the output below displays count in last 24 hours. I need the count beginning midnight and another count starting beginning of the month.
The query you've shown will get the count of rows where OPEN_DATE is an 'epoch date' number representing time after midnight this morning*. The condition:
TRUNC(TO_DATE('1970-01-01','YYYY-MM-DD') + OPEN_DATE/86400) = trunc(sysdate)
requires every OPEN_DATE value in your table (or at least all those for CNLD rows) to be converted from a number to an actual date, which is going to be doing a lot more work than necessary, and would stop a standard index against that column being used. It could be rewritten as:
OPEN_DATE >= (trunc(sysdate) - date '1970-01-01') * 86400
which converts midnight this morning to its epoch equivalent, once, and compares all the numbers against that value; using an index if there is one and the optimiser thinks it's appropriate.
To get everything since the start of the month you could just change the default behaviour of trunc(), which is to truncate to the 'DD' element, to truncate to the start of the month instead:
OPEN_DATE >= (trunc(sysdate, 'MM') - date '1970-01-01') * 86400
And the the last 24 hours, subtract a day from the current time instead of truncating it:
OPEN_DATE >= ((sysdate - 1) - date '1970-01-01') * 86400
db<>fiddle with some made-up data to get 72 back for today, more for the last 24 hours, and more still for the whole month.
Based on your current query I'm assuming there won't be any future-dated values, so you don't need to worry about an upper bound for any of these.
*Ignoring leap seconds...
It sounds like you have a column that is of data type TIMESTAMP and you only want to select rows where that TIMESTAMP indicates that it is today's date? And as a related problem, you want to find those that are the current month, based on some system values like CURRENT TIMESTAMP and CURRENT DATE? If so, let's call your column TRANSACTION_TIMESTAMP instead of (reserved word) DATE. Your first query could be:
SELECT COUNT(NUM)
FROM TABLE
WHERE
STATUS = 'CLND'
AND
DATE(TRANSACTION_TIMESTAMP)=CURRENT DATE
The second example of finding all for the current month up to today's date could be:
SELECT COUNT(NUM)
FROM TABLE
WHERE
STATUS = 'CLND'
AND
YEAR(DATE(TRANSACTION_TIMESTAMP)=YEAR(CURRENT DATE) AND
MONTH(DATE(TRANSACTION_TIMESTAMP)=MONTH(CURRENT DATE) AND
DAY(DATE(TRANSACTION_TIMESTAMP)<=DAY(CURRENT DATE)

SQL ORACLE Get week numbers from multiple datetime rows

I have 70.000 rows of data, including a date time column (YYYY-MM-DD HH24-MM-SS.).
I want to split this data into 3 separate columns; Hour, day and Week number.
The date time column name is 'REGISTRATIONDATE' from the table 'CONTRACTS'.
This is what I have so far for the day and hour columns:
SELECT substr(REGISTRATIONDATE, 0, 10) AS "Date",
substr(REGISTRATIONDATE, 11, 9) AS "Hour"
FROM CONTRACTS;
I have seen the options to get a week number for specific dates, this assignment concerns 70.000 dates so this is not an option.
You (the OP) still have to explain what week number to assign to the first few days in a year, until the first Monday of the year. Do you assign a week number for the prior calendar year? In a Comment I asked about January 1, 2017, as an example; that was a Sunday. The week from January 2 to January 8 of 2017 is "week 1" according to your definition; what week number do you assign to Sunday, January 1, 2017?
The straightforward calculation below assigns to it week number 0. Other than that, the computation is trivial.
Notes: To find the Monday of the week for any given date dt, we can use trunc(dt, 'iw'). iw stands for ISO Week, standard week which starts on Monday and ends on Sunday.
Then: To find the first Monday of the year, we can start with the date January 7 and ask for the Monday of the week in which January 7 falls. (I won't explain that one - it's easy logic and it has nothing to do with programming.)
To input a fixed date, the best way is with the date literal syntax: date '2017-01-07' for January 7. Please check the Oracle documentation for "date literals" if you are not familiar with it.
So: to find the week number for any date dt, compute
1 + ( trunc(dt, 'iw') - trunc(date '2017-01-07', 'iw') ) / 7
This formula finds the Monday of the ISO Week of dt and subtracts the first Monday of the year - using Oracle date arithmetic, where the difference between two dates is the number of days between them. So to find the number of weeks we divide by 7; and to have the first Monday be assigned the number 1, instead of 0, we need to add 1 to the result of dividing by 7.
The other issue you will have to address is to convert your strings into dates. The best solution would be to fix the data model itself (change the data type of the column so that it is DATE instead of VARCHAR2); then all the bits of data you need could be extracted more easily, you would make sure you don't have dates like '2017-02-29 12:30:00' in your data (currently, if you do, you will have a very hard time making any date calculations work), queries will be a lot faster, etc. Anyway, that's an entirely different issue so I'll leave it out of this discussion.
Assuming your REGISTRATIONDATE if formatted as 'MM/DD/YYYY'
the simples (and the faster ) query is based ond to to_char(to_date(REGISTRATIONDATE,'MM/DD/YYYY'),'WW')
(otherwise convert you column in a proper date and perform the conversio to week number)
SELECT substr(REGISTRATIONDATE, 0, 10) AS "Date",
substr(REGISTRATIONDATE, 11, 9) AS "Hour",
to_char(to_date(REGISTRATIONDATE,'MM/DD/YYYY'),'WW') as "Week"
FROM CONTRACTS;
This is messy, but it looks like it works:
to_char(
to_date(RegistrationDate,'YYYY-MM-DD HH24-MI-SS') +
to_number(to_char(trunc(to_date(RegistrationDate,'YYYY-MM-DD HH24-MI-SS'),'YEAR'),'D'))
- 2,
'WW')
On the outside you have the solution previous given by others but using the correct date format. In the middle there is an adjustment of a certain number of days to adjust for where the 1st Jan falls. The trunc part gets the first of Jan from the date, the 'D' gets the weekday of 1st Jan. Since 1 represents Sunday, we have to use -2 to get what we need.
EDIT: I may delete this answer later, but it looks to me that the one from #mathguy is the best. See also the comments on that answer for how to extend to a general solution.
But first you need to:
Decide what to do dates in Jan before the first Monday, and
Resolve the underlying problems in the date which prevent it being converted to dates.
On point 1, if assigning week 0 is not acceptable (you want week 52/53) it gets a bit more complicated, but we'll still be able to help.
As I see it, on point 2, either there is something systematically wrong (perhaps they are timestamps and include fractions of a second) or there are isolated cases of invalid data.
Either the length, or the format, or the specific values don't compute. The error message you got suggests that at least some of the data is "too long", and the code in my comment should help you locate that.

Best way to break down by weeks in BigQuery

So what I'm looking to do is create a report that shows how many sales a company had on a weekly basis.
So we have a time field called created that looks like this:
2016-04-06 20:58:06 UTC
This field represents when the sale takes place.
Now lets say I wanted to create a report that gives you how many sales you had on a weekly basis. So the above example will fall into something like Week of 2016-04-03 (it doesn't have to exactly say that, I'm just going for the simplest way to do this)
Anyone have any advice? I imagine it involves using the UTEC_TO_xxxxxx functions.
The documentation advises to use standard SQL functions, like DATE_TRUNC():
SELECT DATE_TRUNC(DATE '2019-12-25', WEEK) as week;
you can use WEEK() function - it gives you week number
SELECT WEEK('2016-04-06 20:58:06 UTC')
if you need first day of the week - you can try something like
STRFTIME_UTC_USEC((UTC_USEC_TO_WEEK(TIMESTAMP_TO_USEC(TIMESTAMP('2016-05-02 20:58:06 UTC')), 0)),'%Y-%m-%d')
I had to add parentheses:
SELECT DATE_TRUNC(DATE('2016-04-06 20:58:06 UTC'), WEEK) as week;
This is quite an old question and things have moved on since.
In my case, I found that the old WEEK function is no longer recognised, so I had to instead use the EXTRACT function. The doc for it can be found here.
For me it was enough to just extract the ISOWEEK from the timestamp, which results in the week of the year (the ISOYEAR) as a number.
ISOWEEK: Returns the ISO 8601 week number of the datetime_expression. ISOWEEKs begin on Monday. Return values are in the range [1, 53]. The first ISOWEEK of each ISO year begins on the Monday before the first Thursday of the Gregorian calendar year.
So I did this:
SELECT EXTRACT(ISOWEEK FROM created) as week
And if you want to see the week's last day, rather than the week's number in a year, then:
SELECT last_day(datetime(created), isoweek) as week

Teradata Change format of Week Number

I'm pretty new to SQL so I hope this isn't a dumb question, tried to google but couldn't find anything.
I'm summing sales of departments per week in SQL and am using TD_SYSFNLIB.WEEKNUMBER_OF_YEAR (trans_dt) to get the week number.
I think everything is working except I'd like to change the format of the weeks to the start date of the week, e.g. week 1 = 1/4/15
Also, i'm not sure how to handle the very first of the year week 0 since I think that should be grouped up with week 52 of last year.
The following date math trick should get you Beginning of Week as an actual date without having to join to the SYS_CALENDAR view or using a function:
SELECT CURRENT_DATE - ((CURRENT_DATE - DATE '0001-01-07) MOD 7) AS BOW;
Starting with TD14 there's NEXT_DAY which returns the following weekday, if you subtract 7 days you get the previous day:
next_day(trans_dt - 7, 'sunday')

How to ADDDATE() in Access using 'DAY' vs 'd'

I am trying to create a 'Due Date' column where it takes the date from the 'DATE_OF' column and adds the dynamic variable, "DAYS".
SELECT DATEADD('DAY',TABLE1.DAYS,TABLE1.DATE_OF) AS DUE_DATE
FROM TABLE1
This is the format of the date from 'DATE_OF'
04/16/14 12:00 AM
06/24/14 12:00 AM
04/01/14 12:00 AM
DAYS uses only integers (Values between 10 and 90).
But I keep getting this for every result
12/30/99 12:00 AM
What am I doing wrong?
UPDATE
I referenced mssqltips.com and found that the abbreviation for 'DAY' is 'd'. When I changed this, it worked. I do not know why.
As you've come to figure out, there are dissimilarities between many other RDMS' SQL/functions and Access SQL/functions.
Unfortunately, 'DAY' is not a proper interval when using the DateAdd function in MS Access. It is 'd'.
For further note: If you were going to hardcode in a date, the # on either side of the date are required. More details can be found here
Access does not accept not accept the value 'DAY' in the ADDDATE() function.
A complete list of accepted values can be found here
Here are the details of the accepted values.
Value Explanation
yyyy Year
q Quarter
m Month
y Day of the year
d Day
w Weekday
ww Week
h Hour
n Minute
s Second