Below is simplified example of my data. As you can see – there are just two rows here
So I run below and suddenly getting unexpected result
What I expected was something like:
Why am I getting wrong result?
Moreover, when I run below – I am getting only one row. Why second row with id=1 is not showing??
Is there BigQuery bug or what?
Disclaimer: I was asked exactly this type of question few times offline (outside of StackOverflow) and recently saw very same question on SO (I can't understand this BigQuery magic. find string with LIKE) but unfortunately it was deleted so I decided to Post this on my own
The reason for GROUP BY not grouping those two rows is that str field in those rows are actually different. Unfortunately, BigQuery Web UI collapses spaces in result panel when it is in Table mode. To see real/original values you can switch to JSON mode, as below
Same reason is for unexpected result for use of LIKE
As of how to deal with this? It depends!
For example you can kind of normalize your strings by suppressing spaces by yourself as it is shown below
P.S. In our internal tools – we just fixed the issue with suppressed spaces and just simply show all spaces:
Related
I wish to analyze the queries executed on certain redshift warehouse (not mine).
In order to do so I'm using a query with a join on stl_querytext and stl_query.
My question is how come I'm also getting illegal queries (I.E queries with wrong sql syntax)?
When I've tried it in my local redshift I haven't seen those. Also, couldn't find relevant documentation.
Is this a configuration issue? And in case I'm supposed to those queries is there a way to know those are illegal ones?
Thanks,
Nir.
So stl_querytext breaks long queries into parts identified by sequence number. I hope you are reconstructing the parts into the original query as a first step. This can be done with listagg() function as long as the resulting query doesn't over the max tex field (about 320 parts).
Now this is not enough to get valid SQL back in all cases because you need to treat combining the parts differently depending if the section of the query is inside or outside a text string in the query. (Is white space needed between parts or not)
I've done this exact process a bunch so this is doable. I don't have a perfect process on the whitespace question, I get close. Maybe others know the expression to get an exact recreation of the query from stl_querytext.
I need case insensitivity in my queries so I found IGNORE CASE which works superbly when used in queries that target the browser (I am talking about BQ web UI). If I choose a destination table (an absolute must for me) and select Allow Large Results (with unchecked Flatten Results) then I get a cryptic error like this:
Error: unexpected LIMIT clause at: 2.200 - 2.206
Even though this Official Google BigQuery issue and feature request tracker post seems to speak of the same issue and even though the problem seems to have been acknowledged back in Jan 2015 the solution isn't apparent.
I could potentially use a bunch of temp tables with lowercased search columns as a workaround but that sounds awfully difficult with the number of tables and columns that I have and the complex queries that I intend to run.
Any other possible workarounds? Why isn't this working yet on BQ?
Yes, it is a known problem, and it has not been neglected. The code changes to fix it are (surprisingly) not trivial, but they are mostly done. Not team is carefully looking how to enable and deploy them. I cannot give you a timeline, but the fix to this problem is coming.
The only workarounds in the meantime, are to wrap all the string comparisons, string GROUP BYs and string ORDER BYs with conversion to LOWER() (or UPPER()) of operands.
I'm querying for test results which are associated with any test set that has a particular tag.
However, this query does not work:
(TestSet.Tags.Name = "foo")
What does work is:
(TestSet.Tags.Name contains "foo")
I would think the first query should work if the second one returns matches with the tag "foo". I presume this is a bug?
I can get around this problem by using the second query, but of course the problem is that this can match a tag named "foo2" as well, so my query can have extra results (potentially many more) and I have to filter them out. Additionally, now I need to have my query fetch the "Tags" as well, so every result I get back is larger because of it.
Yes, as user1195996 suggested this feels like a bug. Your same queries work as expected against defect or user stories. Please work with Rally Support on this issue so we can work to correct it.
We have recently had to do some work with an OpenEdge database that a third party product uses, and today (after much hair-pulling), we finally identified why a view was returning no results.
This view in question combines about 100 separate tables, and is then queried against (we have limited rights to this database). One of the fields returned by this view is a hard-coded string literal, along the lines of
'John Smith' AS TheName
We were having difficulty running queries that included this string, which we were trying to RTrim (the view returned a lot of trailing spaces) and then concatenate with another field.
However, if we used RTrim on this field then, instead of returning an error message, or a null or something like that, the row simply wasn't returned. We weren't trying to use it in a WHERE clause or JOIN, this was simply part of the SELECT ... FROM VIEWNAME. After reviewing the view, it seemed that the view had erroneously detected the length of the string as 9 characters (no length was specified in the definition), and RTrim just didn't work.
Now, I could understand why this might lead to an error message, or a NULL value in the SELECT, but why would the row simply not be returned at all? This doesn't seem like good SQL behaviour and I've never seen it happen with any other RDBMS.
Other info : we're test querying via ODBC and WinSQL, with a view to this being included in an existing ASP.NET app. We don't have access to the backend except via this, although we do have rights to create views.
Update : As a freaky follow-up, we have now discovered that if we attempt to query this view without any WHERE clause, no records are returned. This may have the same cause.
This sounds like it could be related to the SQL-WIDTH within the progress database. One problem with Progress is that if the content of the field exceeds the SQL-WIDTH then you will get strange SQL behaviour (sometimes the driver might fail, other times you get no results).
To identify this you need to use the dbtool command to check for SQL-WIDTH's that may be exceeded.
Make sure you don't have blanks. Trimming doesn't remove blanks only spaces. Blanks are also not nulls. There is a difference in the character set while it is not visibly different in your editor.
I have run into this with a few databases, DBII, Oracle, PostGreSQL. Check the character set of your editor and try viewing the tables, you might see nothing or you might see big rectangles.
That sounds like very strange behavior. Just code around it, do the trim and/or string manipulation in the application and go on your way.
I am using application builder and setting up select boxes to populate a query that spits out into a report below, I'm having a problem with one populating though, and I can't immediatly see why;
Here's how I'm referencing;
AND CINEMA.LOCATION = :pticketstown
this works fine and pulls in the choice from the select box which is popluated from a query on the cinema table, however when I try and do the same thing for another table, try and pull in a film title from a select box choice - it doesnt seem to work, the rport says no output.
Im setting up the select box the same way, and its pulling in the films, but when as soon as I reference it in the sql report, the ouptut is empty.
FILM.FILM_TITLE = :pfilm
Has nayone any thoughts on why it would fail on something that uses the same method ? is it a case that the value isn't bieng passed across? I'm a little stumped if that's the case as I set it up the same way as the working one.
Thanks for looking
I assume from the sound of it you are using application express? Have you turned on debugging to see exactly what is being passed? It might be encoding the spaces or something.
It seems strange that the two, seemingly similar queries would behave differently. Do you have access to the actual sql produced? If so, i would try running that against your db direct through the command line or sql developer if you have it.