How to group years in decades in sqlite3 in jupyter notebook? - sql

Question

I'm suppose to Find the decade D with the largest number of films and the total number of films in D. A decade is a sequence of 10 consecutive years. For example, say in your database you have movie information starting from 1965. Then the first decade is 1965, 1966, ..., 1974; the second one is 1967, 1968, ..., 1976 and so on.
I'm suppose to implement this in jupyter note book where I imorpted sqlite3
I wrote the following code for it.
Select count(*) as total_films,concat(decade,'-',decade+9)
FROM (Select floor(YEAR('year')/10)*10 as decade FROM movie) t
GROUP BY decade
Order BY total_films desc;
However, the notebook threw error like "no such function: floor" and "no such function: Year" and no such function: concat"
Therefore, after going through sqlite documentation I changed code to
Select count(*) as total_films,decade||'-'||decade+9
FROM (Select cast(strftime('%Y',year)/10 as int)*10 as decade FROM movie) t
GROUP BY decade
Order BY total_films desc;
However, I got an incorrect output :
count(*) decade||'-'||decade+9
0 117 NaN
1 3358 -461.0
Would appreciate insights on why this is happening.
Updating question after going through comments by c.Perkins
1) I began, checking the type of year column
using the query PRAGMA table_info(movie)
Got the following result
cid name type notnull dflt_value pk
0 0 index INTEGER 0 None 0
1 1 MID TEXT 0 None 0
2 2 title TEXT 0 None 0
3 3 year TEXT 0 None 0
4 4 rating REAL 0 None 0
5 5 num_votes INTEGER 0 None 0
Since the year column is of the type text I changed to int using the cast function and check for nulls or NaN SELECT CAST(year as int) as yr FROM MOVIE WHERE yr is null
I didn't get any results, therefore it appears there are no nulls. However, on using the query SELECT CAST(year as int) as yr FROM MOVIE order by yr asc I see a lot of zeros in the year column
yr
0 0
1 0
2 0
3 0
4 0
-
-
-
-
3445 2018
3446 2018
3447 2018
3448 2018
3449 2018
3450 2018
From the above we see that the year is given as it is and in another stamp, therefore using strftime('%Y', year) did not yield result as mentioned in the comment.
Therefore, keeping all the above in mind, I changed the inner query to
SELECT (CAST( (year/10) as int) *10) as decade FROM MOVIE WHERE decade!=0 order by decade asc
Output for the above query :
decade
0 1930
1 1930
2 1930
3 1930
4 1930
5 1930
6 1940
7 1940
8 1940
-
-
-
3353 2010
3354 2010
3355 2010
3356 2010
3357 2010
Finally, placing this inner query, in the first query I wrote above
Select count(*) as total_films,decade||'-'||decade+9 as period
FROM (SELECT (CAST( (year/10) as int) *10) as decade FROM MOVIE WHERE decade!=0 order by decade asc)
GROUP BY decade
Output :
total_films period
0 6 1939
1 12 1949
2 71 1959
3 145 1969
4 254 1979
5 342 1989
6 551 1999
7 959 2009
8 1018 2019
As far as I can see the only issue is with period column where instead of showing 1930-1939 it is only showing 1939 and so on, if is using || is not right, is there anythother function that could be used ? because concat is not working.
Thanks in advance.

Answer 1

Pending updates to the question as requested in comments, here are a few immediate points that might help solve the problem without having all details:
Does the movie.year column contain null values? Likewise non-numeric or non-date values? the NaN (Not A Number) result likely indicates a null/invalid data in the source. (Technically there is no such NaN value in SQLite, so I'm assuming that the question data is copied form some other data grid or processed output.)
What type of data is in the column movie.year? Does it contain full ISO-8601 date strings or a Julian-date numeric value? Or does it only contain the year (as the column name implies)? If it only contains the year (as a string or integer), then the function call like strftime('%Y', year) will NOT return what you expect and is unnecessary. Just refer to the column directly.
I suspect this is where the -461.0 is coming from.
The operator / is an "integer division" operator if both operands are integers. A valid isolated year value will be an integer and the literal 10 is of course an integer, so integer division will automatically drop any decimal part and returns only the integer part of the division without having to explicitly cast to integer.
According to sqlite docs, the concatenation operator || has highest precedence. That means in the expression decade||'-'||decade+9, the concatenation is applied first so that one possible intermediate would be '1930-1930'+9. (Technically I would consider this result undefined since the string value does not contain a basic data type. In practices on my system, the string is apparently interpreted as 1930 and the overall result is the integer value 1939. Either way you will get unexpected bogus results rather than the desired string.)