GCP Bigquery - query empty values from a record type value - sql

I'm trying to query all resources that has empty records on a specific column but I'm unable to make it work. Here's the query that I'm using:
SELECT
service.description,
project.labels,
cost AS cost
FROM
`xxxxxx.xxxxx.xxxx.xxxx`
WHERE
service.description = 'BigQuery' ;
Here's the results:
As you can see, I'm getting everything with that query, but as mentioned, I'm looking to get resources with empty records only for example record 229,230 so on.
Worth to mention that schema for the column I'm trying to query is:
project.labels RECORD REPEATED
The above was mentioned because I tried using several combinations of WHERE but everything ends up in error.

To identify empty repeated record - you can use ARRAY_LENGTH in WHERE clause like in below example
WHERE ARRAY_LENGTH(project.labels) = 0

Related

Query with Totals Query results as criteria returns the expected number of results squared

Background
I am building an Access 2010 database that has a table [ControllerAdjustments] that keeps track of all adjustments made to controllers with an [AdjustmentID] autonumber field, a [ControllerID] field, an [AdjustmentDate] field, [Setpoint] field, and a [Power] field. The [Power] field represents the power level when the adjustment was made. Ultimately I need two queries to return two sets of results, one query should return the current status of all controllers (basically the most recent adjustment made on each controller) and the other should return the most recent adjustment made on each controller where power level is 100%. I plan to use each of these queries to feed a report. Note: field names changed slightly for convenience when typing, full names given in the code blocks...
Method
I focused on the Current Query first, and figured I would just copy it and make necessary changes to create a 100% Query. I started with a totals query on the [ControllerAdjustments] table, that had [ControllerID] as a Group By field and [AdjustmentDate] as a field that returned the Max value. This query returns exactly the number of records I expected, and after reviewing the sample bogus data I put in the table to check it, it seems to return exactly the records I need. I then created a Select Query that returned all the fields I want in my Current Report, namely the [ControllerAdjustments] table and the related records in upstream related tables. I then set the criteria for the [ControllerID] field in my Select Query to equal [Total_CurrentContAdjs]![ControllerID] and the [AdjustmentDate] in the Select Query to [Total_CurrentContAdjs]![MaxOfAdjustmentDate]. Running this query returns exactly what I want. The SQL for this query is below:
SELECT List_Units.UnitID, List_EDTanks.TankNameShort, List_Controllers.ControllerType, ControllerAdjustments.AdjustmentDate, ControllerAdjustments.ControllerSetpoint, ControllerAdjustments.RxPower
FROM Total_ContAdjsCurrent, ((List_Units INNER JOIN List_EDTanks ON List_Units.UnitID = List_EDTanks.UnitID) INNER JOIN List_Controllers ON List_EDTanks.EDTankID = List_Controllers.EDTankID) INNER JOIN ControllerAdjustments ON List_Controllers.ControllerID = ControllerAdjustments.ControllerID
WHERE (((ControllerAdjustments.AdjustmentDate)=[Total_ContAdjsCurrent]![MaxOfAdjustmentDate]) AND ((ControllerAdjustments.ControllerID)=[Total_ContAdjsCurrent]![ControllerID]))
ORDER BY List_Units.Unit, List_EDTanks.TankSortOrder, List_Controllers.ControllerType DESC;
I then copied the Totals query and added a column for Power, selected Where, unchecked show, and put in 100 for criteria. This works as expected. I then copied my select query, and changed the criteria fields to direct to my new 100% Totals query. This is where my problems begin.
Problem
The second 100% Query does not seem to like the criteria, as it initially throws out the familiar parameter window. This is the SQL Statement for the second query, virtually the same except for referring to the 100% Totals query:
SELECT List_Units.UnitID, List_EDTanks.TankNameShort, List_Controllers.ControllerType, ControllerAdjustments.AdjustmentDate, ControllerAdjustments.ControllerSetpoint, ControllerAdjustments.RxPower
FROM Total_ContAdjsCurrent, Total_ContAdjsStdyState, ((List_Units INNER JOIN List_EDTanks ON List_Units.UnitID = List_EDTanks.UnitID) INNER JOIN List_Controllers ON List_EDTanks.EDTankID = List_Controllers.EDTankID) INNER JOIN ControllerAdjustments ON List_Controllers.ControllerID = ControllerAdjustments.ControllerID
WHERE (((ControllerAdjustments.AdjustmentDate)=[Total_ContAdjsStdyState]![MaxOfAdjustmentDate]) AND ((ControllerAdjustments.ControllerID)=[Total_ContAdjsStdyState]![ControllerID]))
ORDER BY List_Units.Unit, List_EDTanks.TankSortOrder, List_Controllers.ControllerType DESC;
Initially, Access did not add my Totals query into the show table box in design view, because its results were not directly used in the Select Query. So, I added the Totals query to the top, and that allowed my query to run without asking for parameters, but now it returns the number of results I was expecting squared. Basically if I am expecting 3 records: 1, 2, and 3, it is giving me: 1, 1, 1, 2, 2, 2, 3, 3, and 3. For the life of me I cannot figure out why it is doing this, especially because the exact same setup for my Current Query returns exactly what is expected... I thought maybe the where clause in my totals query had something to do with it, so I created a Select Query for the [ControllerAdjustments] table that returned all records with 100 for power. I then used this query for my totals query instead of the totals query itself, but this did not do anything different. I am at a loss, and not sure what else I can do to get the results I want. Any suggestions welcome, thank you!
I solved this by simply starting over from scratch and rebuilding my 100% query. Reviewing the SQL statements, they look identical, however for some reason my query now returns the right number of records. I have no idea why this worked, and am still curious what went wrong in the first place if anyone with time available cares to dig into it, but the original problem statement has been corrected--even if I have no idea how I did it haha...

SQL Query stopped working and cant figure out why

Basically I am using MS Access 2013 to import all active work items that are assigned to a specific group from an API and select the data into 2 new tables (Requests & Request_Tasks).
I then have a form sourced from a query to select specific fields from the 2 tables.
Until yesterday it was working with no problems and nothing has changed.
All of the data appears in the 2 tables so the import from the API works fine.
When it comes to the query selecting the data from the 2 tables (Which are already populated with the correct data) the query returns only data from Requests table with blank fields instead of data from Request_Tasks.
The strange part is that out of 28 active work items it returns 24 correctly and the last 4 are having the problem.
Every new task added to the group has the problem also.
Query is below.
SELECT
Request_Tasks.RQTASK_Number,
Request_Tasks.Request_Number,
Requests.Task, Requests.Entity,
Request_Tasks.Description,
Request_Tasks.Request_Status,
Requests.Requested_for_date,
Request_Tasks.Work_On_Date,
Request_Tasks.Estimated_Time,
Request_Tasks.Actual_Time_Analysis,
Request_Tasks.Offers_Built,
Request_Tasks.Number_of_links_Opened,
Request_Tasks.Number_of_Links_Extended,
Request_Tasks.Number_Of_links_closed,
Request_Tasks.Build_Allocated_to,
Request_Tasks.Buld_Review_Allocated_to,
Request_Tasks.Keying_Allocated_to,
Request_Tasks.Keying_Approval_allocated_to,
Request_Tasks.Actual_Build_Time,
Request_Tasks.Actual_Stakeholder_Support,
Request_Tasks.Task_Completed_Date
FROM Request_Tasks
RIGHT JOIN Requests
ON Request_Tasks.Request_Number = Requests.Request_Number
WHERE (((Request_Tasks.Task_Completed_Date)>=Date()
Or (Request_Tasks.Task_Completed_Date) Is Null)
AND ((Requests.Task)<>"7"
And (Requests.Task)<>"8" And (Requests.Task)<>"9"))
OR (((Request_Tasks.Task_Completed_Date)>=Date()
Or (Request_Tasks.Task_Completed_Date) Is Null)
AND ((Requests.Task)<>"7"
And (Requests.Task)<>"8"
And (Requests.Task)<>"9"))
ORDER BY Request_Tasks.Work_On_Date Is Null DESC , Request_Tasks.Work_On_Date, Requests.Entity Is Null DESC , Requests.Task;
Any help would be great.
Thanks.
The query is using RIGHT JOIN, which means rows from Requests table is always reported even if there is no corresponding entry in Request_tasks table.
A full example is here http://www.w3schools.com/Sql/sql_join_right.asp
In your case, most likely somechange might have happened during data load/API and Request_tasks table is not being populated. That is the reason you see blank data for fields from that table.
Solution
Manually check data for 4 faulty records in Request_tasks table.
Ensure keys in both table request_number are matching including data type and any leading space/non printable characters (if they are string type of data) for faulty records.
Query seems fine, its more of issue with data based on problem statement.

SQL BETWEEN return empty rows even if value exist

what am I doing wrong with my sql query? It always return an empty rows even if there is a value exist.
Here is my query:
SELECT *
FROM users
WHERE user_theme_id IN ( 9735, 9325, 4128 )
AND ( user_date_created BETWEEN '2013-06-04' AND '2013-06-10' );
I tried to cut my original query one by one, I got a result. Here is the first one:
SELECT * FROM users WHERE user_theme_id IN (9735, 9325, 4128 );
I got 3 rows for this result. See attached snapshot:
Now, the next query that I run is this:
SELECT *
FROM users
WHERE user_date_created BETWEEN '2013-06-04' AND '2013-06-10';
I do get 3 results on this. See attached snapshot:
By the way, this sql that uses BETWEEN should suppose return 4 rows but it only return 3. It doesn't return the data which has the created date of 2013-06-10 08:27:43
What am I doing wrong with my original query Why does it always return an empty rows?
If you are getting results by separately running different where clauses doesn't guarantee that AND 2 where clauses will return an answer.
There has to be intersection of rows to get result while AND.
You should validate your data and see if overlapping exists.
I have able to make it work by not using the SQL BETWEEN operators but instead COMPARISON OPERATORS like: >= || <=
I have read it from W3schools.com, the SQL between can produce different results in different databases.
This is the content:
Notice that the BETWEEN operator can produce different result in different databases!
In some databases, BETWEEN selects fields that are between and excluding the test values.
In other databases, BETWEEN selects fields that are between and including the test values.
And in other databases, BETWEEN selects fields between the test values, including the first test value and excluding the last test value.
Therefore: Check how your database treats the BETWEEN operator!
That is what happened in the issue that I am facing. The first field was being treated as part of the test values and the 2nd field was being excluded. Using the comparison operators give accurate result.

SQL - How to insert a subquery into a query in order to retrieve unique values

I am writing reports using Report Builder 3, and I need some help with an sql query to get unique values.
Using the following sample data:
I need to be able to get one single value for feeBudRec returned for each feeRef. The value of each feeBudRec is always the same for each individual feeRef (eg for every data row for feeRef LR01 will have a feeBudRec of 1177).
The reason why I need to get a single feeBudRec value for each feeRef is that I need to be able to total the feeBudRec value for each feeRef in a feePin (eg for feePin LEE, I need to total the feeBudRec values for LR01 and PS01, which should be 1177 + 1957 to get a total of 3134; but if I don't have unique values for feeBudRec, it will add the values for each row, which would bring back a total of 11756 for the 8 LEE rows).
My experience with writing SQL queries is very limited, but from searching the internet, it looks like I'll need to put in a subquery into my SQL query in order to get a single unique feeBudRec figure for each feeRef, and that a subquery that gets a minimum feeBudRec value for each feeRef should work for me.
Based on examples I've found, I think the following subquery should work:
SELECT a.feeRef, a.feeBudRec
FROM (
SELECT uvw_EarnerInfo.feeRef, Min(uvw_EarnerInfo.feeBudRec) as AvailableTime
FROM uvw_EarnerInfo
GROUP BY
uvw_EarnerInfo.feeRef
) as x INNER JOIN uvw_EarnerInfo as a ON a.feeRef = x.feeRef AND a.feeBudRec = x.AvailableTime;
The problem is that I have no idea how to insert that subquery into the query I'm using to produce the report (as follows):
SELECT
uvw_EarnerInfo.feeRef
,uvw_EarnerInfo.PersonName
,uvw_EarnerInfo.PersonSurname
,uvw_EarnerInfo.feePin
,uvw_RB_TimeLedger.TimeDate
,uvw_RB_TimeLedger.matRef
,uvw_RB_TimeLedger.TimeTypeCode
,uvw_RB_TimeLedger.TimeCharge
,uvw_RB_TimeLedger.TimeElapsed
,uvw_WoffTimeByTime.WoffMins
,uvw_WoffTimeByTime.WoffCharge
,uvw_EarnerInfo.feeBudRec
,uvw_EarnerInfo.personOccupation
FROM
uvw_RB_TimeLedger
LEFT OUTER JOIN uvw_WoffTimeByTime
ON uvw_RB_TimeLedger.TimeId = uvw_WoffTimeByTime.TimeId
RIGHT OUTER JOIN uvw_EarnerInfo
ON uvw_EarnerInfo.feeRef = uvw_RB_TimeLedger.feeRef
WHERE
uvw_RB_TimeLedger.TimeDate >= #TimeDate
AND uvw_RB_TimeLedger.TimeDate <= #TimeDate2
If that subquery will get the correct results, can anyone please help me with inserting it into my report query. Otherwise, can anyone let me know what I will need to do to get a unique feeBudRec value for each feeRef?
Depends on the exact schema, but assuming the uvw_EarnerInfo lists the Pin, Ref, and Rec without duplicates, try adding an extra column (after personOccupation) on the end of your query such as :
feeBudRecSum = (Select SUM(FeeBudRec) From uvw_EarnerInfo x
where x.feePin = uvw_EarnerInfo.feePin
Group By x.FeePin)
Note that you would not Sum these values in your report. This column should have the total you are looking for.
The key to Report Builder is to get your query correct from the offset and let the wizard then structure your report for you. It takes all the hard work out of structuring your report manually.
I haven't used Report Builder for a while now but in the query builder of the report displaying the graphical representation of your query you should be able to drag and drop columns in and out of the query set. Dragging a column upwards and out of the box (showing your columns) would have the effect of causing your report to break on this column.
If you restructure this way you will probably have to run the report generator again to regenerate the report and restructure it.
Once you are happy with the structure you can then begin to add the summary columns.

SQL statement HAVING MAX(some+thing)=some+thing

I'm having trouble with Microsoft Access 2003, it's complaining about this statement:
select cardnr
from change
where year(date)<2009
group by cardnr
having max(time+date) = (time+date) and cardto='VIP'
What I want to do is, for every distinct cardnr in the table change, to find the row with the latest (time+date) that is before year 2009, and then just select the rows with cardto='VIP'.
This validator says it's OK, Access says it's not OK.
This is the message I get: "you tried to execute a query that does not include the specified expression 'max(time+date)=time+date and cardto='VIP' and cardnr=' as part of an aggregate function."
Could someone please explain what I'm doing wrong and the right way to do it? Thanks
Note: The field and table names are translated and do not collide with any reserved words, I have no trouble with the names.
Try to think of it like this - HAVING is applied after the aggregation is done.
Therefore it can not compare to unaggregated expressions (neither for time+date, nor for cardto).
However, to get the last (principle is the same for getting rows related to other aggregated functions as weel) time and date you can do something like:
SELECT cardnr
FROM change main
WHERE time+date IN (SELECT MAX(time+date)
FROM change sub
WHERE sub.cardnr = main.cardnr AND
year(date)<2009
AND cardto='VIP')
(assuming that date part on your time field is the same for all the records; having two fields for date/time is not in your best interest and also using reserved words for field names can backfire in certain cases)
It works because the subquery is filtered only on the records that you are interested in from the outer query.
Applying the same year(date)<200 and cardto='VIP' to the outer query can improve performance further.