I am trying to set values on BigQuery table using java big query api but its throwing NullPointerException (java.lang.NullPointerException: null value in entry: popup=null ) every time I send the null value.
The null value should be completely acceptable since my Mode is NULLABLE on schema itself. I have bunch of other fields that have null value on them.
Any suggestion on this issue would be really helpful for me, I am stuck no where due to this.
Note : I may ignore and not set those fields having null values on them but that is not the solution I am looking for. My piece of code is below:
TableRow row = new TableRow();
row.set("ip", "test");
row.set("popup", null);
Don't explicitly set the value to null. Simply ignore it. If it's not present in the payload to BigQuery, it will be set to null. You will not be able to set it to null anyway, because the API is checking for null parameters, and you can't change that behaviour.
So, do this instead:
TableRow row = new TableRow();
row.set("ip", "test");
Related
I'm using npgsql as a nuget package in visual studio 2017 with visual basic.
Various commands do work very well but an ExecuteScalar allways returns 'nothing' although it should give a result.
The command looks like this:
Dim ser As Integer
Dim find = New NpgsqlCommand("SELECT serial from dbo.foreigncode WHERE code = '#code';", conn)
Dim fcode = New NpgsqlParameter("code", NpgsqlTypes.NpgsqlDbType.Varchar)
find.Parameters.Add(fcode)
find.Prepare()
fcode.Value = "XYZ"
ser = find.ExecuteScalar() ==> nothing
When the command string is copied as a value during debugging and pasted into the query tool of PGADMIN it delivers the correct result. The row is definitely there.
Different Commands executed with ExecuteNonQuery() work well, including ones performing UPDATE statements on the row in question.
When I look into the properties of the parameter fcode immediately before the ExecuteScalar it shows 'fcode.DataTypeName' caused an exception 'System.NotImplementedException'.
If I change my prepared statement to "SELECT #code" and set the value of the parameter to an arbitrary value just this value is returned. There is no access to the table taking place because the table name is not part of the SELECT in this case. If I remove the WHERE CLAUSE in the SELECT and just select one column, I would also expect that something has to be returned. But again it is nothing.
Yes there is a column named serial. It is of type bigint and can not contain NULL.
A Query shows that there is no single row that contains NULL in any column.
Latest findings:
I queried a different table where the search column and the result column happen to have the same datatype. It works, so syntax, passing of parameter, prepare etc. seems to work in principal.
The System.NotImplementedException in the DataTypeName property of the parameter occurs as well but it works anyway.
I rebuilt the index of the table in question. No change.
Still: when I copy/paste the CommandText and execute it in PGAdmin it shows the correct result.
Modifying the Command and using plain text there without parameter and without prepare still does yield nothing. The plain text CommandText was copy/pasted from PGAdmin where it was successfully executed before.
Very strange.
Reverting search column and result column also gives nothing as a result.
Please try these two alternatives and post back your results:
' Alternative 1: fetch the entire row, see what's returned
Dim dr = find.ExecuteReader()
While (dr.Read())
Console.Write("{0}\t{1} \n", dr[0], dr[1])
End While
' Alternative 2: Check if "ExecuteScalar()" returns something other than an int
Dim result = find.ExecuteScalar()
... and (I just noticed Honeyboy Wilson's response!) ...
Fix your syntax:
' Try this first: remove the single quotes around "#code"!
Dim find = New NpgsqlCommand("SELECT serial from dbo.foreigncode WHERE code = #code;", conn)
Update 1
Please try this:
Dim find = New NpgsqlCommand("SELECT * from dbo.foreigncode;", conn)
Q: Does this return anything?
Dim dr = find.ExecuteReader()
While (dr.Read())
Console.Write("{0}\t{1} \n", dr[0], dr[1])
End While
Q: Does this?
Dim result = find.ExecuteScalar()
Q: Do you happen to have a column named "serial"? What is it's data type? Is it non-null for the row(s) with 'XYZ'?
Please update your original post with this information.
Update 2
You seem to be doing ":everything right":
You've confirmed that you can connect,
You've confirmed that non-query updates to the same table work (with npgsql),
You've confirmed that the SQL queries themselves are valid (by copying/pasting the same SQL into PGAdmin and getting valid results).
As Shay Rojansky said, "System.NotImplementedException in the DataTypeName property" is a known issue stepping through the debugger. It has nothing to do with your problem: https://github.com/npgsql/npgsql/issues/2520
SUGGESTIONS (I'm grasping at straws)::
Double-check "permissions" on your database and your table.
Consider installing a different version of npgsql.
Be sure your code is detecting any/all error returns and exceptions (it sounds like you're probably already doing this, but it never hurts to ask)
... and ...
Enable verbose logging, both client- and server-side:
https://www.npgsql.org/doc/logging.html
https://www.postgresql.org/docs/9.0/runtime-config-logging.html
... Finally ...
Q: Can you make ANY query, from ANY table, using ANY query method (ExecuteReader(), ExecuteScalar(), ... ANYTHING) from your npgsql/.Net client AT ALL?
I finally found it. It's often the small things that can have a big impact.
When the value was assigned to the parameter a substring index was incorect.
Now it works perfectly.
Thanks to everybody who spent his time on this.
I'm new to Dataflow. I've got a predefined-schema containing a non-repeated RECORD field called "device":
device.configId: STRING
device.version: STRING
Using a ParDo transform, I tried inserting a TableRow with this kind of field, as follows:
TableRow row = new TableRow();
row.put("field1", "val1");
TableRow device = new TableRow();
device.put("configId", "conf1");
device.put("version", "1.2.3");
row.put("device", device);
out.output(row);
I logged the table row, it looks like this:
{field1=val1, device={configId=conf1, version=1.2.3}}
I output it to a standard transform: BigQueryIO.write()
But the latter issues an error:
java.lang.RuntimeException: java.io.IOException:
Insert failed: [{"errors":[{
"debugInfo":"",
"location":"device.configid",
"message":"This field is not a record.",
"reason":"invalid"
}],"index":0}]
Not sure why, but note the location spells "configid" in lowecase - not in camel case as in the original log.
Any ideas on how to insert such an object to BigQuery?
Found out the problem. Apparently, this error message was caused only when the "configId" field was set to null rather than "conf1". To be exact, it was implicitly set to JSONObject.NULL coming from some input object.
I have a custom field being added on user story (HierarchicalRequirement) level.
The WSAPI documentation shows the following details for the field:
c_CustomFieldName
Required false
Type string
Max Length 32,768
Sortable true
Explicit Fetch false
Query Expression Operators contains, !contains, =, !=
When trying to create a report using Custom List to identify user stories where this field is empty, I add (c_CustomFieldName = "") to the query.
And yet, the result shows rows where this field is not empty.
How can that be?
I tried querying on null, but it didn't work.
thx in advance
What you're doing should work- are you getting errors, or just incorrect data? It almost seems like it's ignoring your query altogether.
I tried to repro both with the custom list app and against wsapi directly and the following all worked as expected:
(c_CustomText = "") //empty
(c_CustomText = null) //empty
(c_CustomText != "") //non-empty
(c_CustomText != null) //non-empty
It's possible you're running into some weird data-specific edge case in your data. It may be worth following up with support.
I am trying to get unique value of a field with the code:
query.set("q","*:*" );
query.setGetFieldStatistics(true);
query.setGetFieldStatistics("popu_s");
QueryResponse rsp = solr.query(query);
FieldStatsInfo stats = rsp.getFieldStatsInfo().get("popu_s");
System.out.println(stats.getCount());
System.out.println(stats.getCountDistinct());
stats.getCount() gives the correct count. However, stats.getCountDistinct() always returns null.
Any idea?
getCountDistinct() in FieldStatsInfo is null because it was not provided when creating the FieldStatsInfo object (what was returned from solr.query(query)).
There is no guarantee that any of the fields in this pojo are populated.
As an alternative try using stats.getCardinality() although again there is no guarantee this is poulated.
Found the solution!
To make getCountDistinct() return a value, one needs to add this
query.addStatsFieldCalcDistinct("popu_s", true);
There were three similar questions in StackOverFlow but none gave an answer..
If have found why this error in occurring but don't know the fix.
I am using Strongly Typed Dataset for my project which is created as a dll for DAL.
I have added the Sql Server Table into this dataset using the designer and has created a DataAdapter
It works fine when i insert using DataTableAdapter
daLabTest.Insert(txtLabTestId.Text, cmbLabTestType.Text, cmbTestName.Text, txtLabFees.Text, dtpLabEffDate.Value)
but when i want to show the data from the table in a combobox or gridview i get this error.
i told that i found out what the problem is, I just previewed the data using DataSet designer and found out that the Function returns data like this...
The query i wrote to view this in dataset is
Select distinct(TestType) from LabTestTypes
so this should return only one column but the dataset is returning 5 columns but others as null, and the TestName column is a primary which should not be null when returned, so the problem exists..
To resolve this i tried to change the NullValue & AllowDBNull property to [Empty] and true respectively but that didn't worked for me.
Please help me in this...
That overly general constraint exception is nasty, where's the InnerException after so many complaints?!
This template may help identify the problem row and column but a "Fill" version of the query function is needed. E.g. GetDistinct*() --> Fill*(). Then a table can be created and interrogated for the row's error text.
SomeTable tTable = new SomeTable()
try {
// sorry, if you have a GetData, change to the fill version
someTable.FillByActiveLogin(tTable, loginName);
} catch (System.Data.ConstraintException constrExc) {
System.Data.DataRow[] rowsErr = tTable.GetErrors();
for (int i = 0; i < rowsErr.Count(); i++)
if (rowsErr[i].HasErrors)
Dbg.WriteLine(rowsErr[i].RowError);
}
(Thanks Michael S for this hint whoever/wherever you are!)
I got this error in a function from a DLL that uses a stored procedure. The procedure did not return all the fields in the table. One of the fields excluded was one that cannot be null. That apparently caused the constraint exception. When I changed the procedure and the DLL to include that field, the exception went away.
After spending ages on this problem myself, I have resolved it modifying the query in the dataset to return a dummy value (that can be ignored) for each key field that is not required in the output.
So your query would become...
Select distinct TestType, 1 as ID, "Dummy" as TestName, "Dummy" as TestFees, "Dummy" as TestDate
from LabTestTypes