TABLE_DATE_RANGE for xxxx_yyyymm format tables - sql

I'm having a problem trying to query for 15 months worth of data.
I know about bigquery's wildcard functions, but I can't seem to get them to work with my tables.
For example, if my tables are called:
xxxx_201501,
xxxx_201502,
xxxx_201503,
...
xxxx_201606
How can I select everything from 201501 until today (current_timestamp)?
It seems that it's necessary to have the tables per day, am I wrong?
I've also read that you can use regex but can't find the way.


With Standard SQL, you can use a WHERE clause on a _TABLE_SUFFIX pseudo column as described here:
Is there an equivalent of table wildcard functions in BigQuery with standard SQL?
In this particular case, it would be:
SELECT ... from `mydataset.xxx_*` WHERE _TABLE_SUFFIX >= '201501';


This is a bit long for a comment.
If you are using the standard SQL dialect, then I don't think the functionality is yet implemented.
If you are using the legacy SQL dialect, then you can use a function such as TABLE_DATE_RANGE(). This and other table wildcard functions are well documented.
EDIT:
Oh, I see. The simplest way would be to store the tables as YYYYMM01 so you can use the range query.
But, you can also use table_query():
from table_query(t, 'right(table_id, 6) >= ''201501'' ')

Related

Request last hour data from Big Query with Standard SQL

This is my problem.
I would like to request only the data of the last hour from Big Query.
I would like to use Standard Sql.
I would like to pay only for read the data in this interval of time.
Example :
My partition of the day take 200 Go. I request data of the last hour (40Go). Is it possible to pay only for 40Go in Standard SQL ?
Thanks !

You can use table decorators (specifically range decorators) but they are supported in BigQuery Legacy SQL ONLY
To get data from the last hour you can use below:
SELECT <list_of_fields>
FROM [yourproject:yourdataset.yourtable#-3600000-]
Of course, the preferred query syntax for BigQuery is standard SQL - so you can either have your query logic built with Legacy SQL syntax and thus have whole logic in one query or you can use split logic to first get last hour data into temp table using legacy's sql decorators and then use standard sql to apply needed logic
Meantime see below opened issue on Google's Issue Tracker:
Support an equivalent to table decorators in standard SQL
From that thread - looks like the closest feature to meet your case could be hourly partitioning - whenever it will be available

How compare only date with nhibernate and criteria

im trying to compare two dates like this
ICriteria criteria = base.DataStore.TransactionScope.NHibernateSession.CreateCriteria<CcCorte>();
criteria.Add(Restrictions.Ge("Start", init.Date));
But im just need to compare the dates without the time, i can take only date for server variable but dont know how to do it with the other.
Any ideas?

The following functions are available in HQL, maybe you could use them:
second(...), minute(...), hour(...), day(...), month(...), year(...)
So yea write your query using hql.
and if you think thats not helpful you can always write sql query and use sql servers DateTime functions like DateDiff etc and achieve the same...
Hope that helps.

Expression Too Complex In Access 2007

When I try to run this query in Access through the ODBC interface into a MySQL database I get an "Expression too complex in query expression" error. The essential thing I'm trying to do is translate abbreviated names of languages into their full body English counterparts. I was curious if there was some way to "trick" access into thinking the expression is smaller with sub queries, or if someone else had a better idea of how to solve this problem. I thought about making a temporary table and doing a join on it, but that's not supported in Access SQL.
Just as an FYI, the query worked fine until I added the big long IFF chain. I tested the query on a smaller IFF chain for three languages, and that wasn't an issue, so the problem definitely stems from the huge IFF chain (It's 26 deep). Also, I might be able to drop some of the options (like combining the different forms of Chinese or Portuguese)
As a test, I was able to get the SQL query to work after paring it down to 14 IFF() statements, but that's a far cry from the 26 languages I'd like to represent.
SELECT TOP 5 Count( * ) AS [Number of visits by language], IIf(login.lang="ar","Arabic",IIf(login.lang="bg","Bulgarian",IIf(login.lang="zh_CN","Chinese (Simplified Han)",IIf(login.lang="zh_TW","Chinese (Traditional Han)",IIf(login.lang="cs","Czech",IIf(login.lang="da","Danish",IIf(login.lang="de","German",IIf(login.lang="en_US","United States English",IIf(login.lang="en_GB","British English",IIf(login.lang="es","Spanish",IIf(login.lang="fr","French",IIf(login.lang="el","Greek",IIf(login.lang="it","Italian",IIf(login.lang="ko","Korean",IIf(login.lang="hu","Hungarian",IIf(login.lang="nl","Dutch",IIf(login.lang="pl","Polish",IIf(login.lang="pt_PT","European Portuguese",IIf(login.lang="pt_BR","Brazilian Portuguese",IIf(login.lang="ru","Russian",IIf(login.lang="sk","Slovak",IIf(login.lang="sl","Slovenian","IIf(login.lang="fi","Finnish",IIf(login.lang="sv","Swedish",IIf(login.lang="tr","Turkish","Unknown")))))))))))))))))))))))))) AS [Language]
FROM login, reservations, reservation_users, schedules
WHERE (reservations.start_date Between DATEDIFF('s','1970-01-01 00:00:00',[Starting Date in the Following Format YYYY/MM/DD]) And DATEDIFF('s','1970-01-01 00:00:00',[Ending Date in the Following Format YYYY/MM/DD])) And reservations.is_blackout=0 And reservation_users.memberid=login.memberid And reservation_users.resid=reservations.resid And reservation_users.invited=0 And reservations.scheduleid=schedules.scheduleid And scheduletitle=[Schedule Title]
GROUP BY login.lang
ORDER BY Count( * ) DESC;
# Michael Todd
I completely agree. The list of languages should have been a table in the database and the login.lang should have been a FK into that table. Unfortunately this isn't how the database was written, and it's not really mine to modify. The languages are placed into the login.lang field by the PHP running on top of the database.

I thought about making a temporary table and doing a join on it, but that's not supported in Access SQL.
Did you try making a table of languages within Access, and joining it to the MySQL tables?

You may try the below expression. what I did is, your expression is cut down to two parts, then a final 'IIf' check will do the trick. You will have additional 2 fields and you may ignore those. I had the same situation and this worked well for me. PS: You may need to double check the closing brackets in the below expression. I did it quickly.
Thanks,
Shibin
IIf(login.lang="ar","Arabic",IIf(login.lang="bg","Bulgarian",IIf(login.lang="zh_CN","Chinese (Simplified Han)",IIf(login.lang="zh_TW","Chinese (Traditional Han)",IIf(login.lang="cs","Czech",IIf(login.lang="da","Danish",IIf(login.lang="de","German",IIf(login.lang="en_US","United States English",IIf(login.lang="en_GB","British English",IIf(login.lang="es","Spanish",IIf(login.lang="fr","French",IIf(login.lang="el","Greek",IIf(login.lang="it","Italian",""))))))))))))) as l1,
IIf(login.lang="ko","Korean",IIf(login.lang="hu","Hungarian",IIf(login.lang="nl","Dutch",IIf(login.lang="pl","Polish",IIf(login.lang="pt_PT","European Portuguese",IIf(login.lang="pt_BR","Brazilian Portuguese",IIf(login.lang="ru","Russian",IIf(login.lang="sk","Slovak",IIf(login.lang="sl","Slovenian","IIf(login.lang="fi","Finnish",IIf(login.lang="sv","Swedish",IIf(login.lang="tr","Turkish","Unknown")))))))))))) as l2,
IIf(l1="",l2,l1) AS [Language]

If you can't use a lookup table, create a custom VB function, so that instead of 26 IIf statements, you have one function call.

SQL: Ordering by how much greater than something is?

I have two datetime fields here: actual_delivery and scheduled_delivery
What I want to do an ORDER BY on is how much great actual_delivery is than scheduled_delivery.
I'm using MySQL locally and PostgreSQL in production, so it needs to work for both.

If I were doing it in SQL Server I'd calculate DATEDIFF(actual_delivery, scheduled_deliver) AS [DeliveryDifference] then order by that computed column.
A quick search indicates there's a datediff function in MySql but the syntax may be slightly different in PostgreSQL so you may have to create your own function there.

Try this:
SELECT actual_delivery, scheduled_delivery, actual_delivery - scheduled_delivery as difference FROM tablename ORDER BY difference

django: datediff sql queries?

I'm trying to do the equivalent of the following SQL in Django:
SELECT * FROM applicant WHERE date_out - date_in >= 1 AND date_out - date_in <= 6
I can do this as a RAW sql query, but this is becoming frustrating in dealing with a RawQuerySet instead of a regular QuerySet object as I would like to be able to filter it later in the code.

I came across the issue of Django not natively supporting Datediff (and other database equivalents), and needed to use such a function many times for a particular project.
Upon further reading, it became clear that the implementation of calculating an interval from two dates differs widely between major database flavours. This is probably why it's not got a native abstraction function in Django yet. So I wrote my own Django ORM function for datediff:
See: mike-db-tools Github repository
You'll see the varying syntax between the database backends written in the docstrings for the respective databases. Datediff supports sqlite, MySQL / MariaDB, PostgreSQL and Oracle.
Usage (Django 1.8+):
from db_tools import Datediff
# Define a new dynamic fields to contain the calculated date difference
applicants = Applicant.objects.annotate(
days_range=Datediff('date_out','date_in', interval='days'),
)
# Now you can use this dynamic field in your standard filter query
applicants = applicants.filter(days_range__gte=1, days_range__lte=6)
I'm really quite derpy when it comes to my code, so I encourage you to fork and improve.

You can use the extra() method and pass in a where keyword argument. The value of where should be a list that contains the SQL WHERE clause of the query above. I tested this with Postgresql 8.4 and this is what it looked like in my case:
q = Applicant.objects.extra(where = ["""date_part('day', age(date_out, date_in)) >= 1 and
date_part('day', age(date_out, date_in)) <= 6"""])
This will return you a valid QuerySet instance.