Google BigQuery: Error: Invalid schema update. Field has changed mode from REQUIRED to NULLABLE - google-bigquery

I'm trying to append the results of a query to another table.
It doesn't work and sends out the following error:
Error: Invalid schema update. Field X has changed mode from REQUIRED to NULLABLE.
The field X is indeed REQUIRED, but I don't try to insert any NULL-values into that specific column (the whole table doesn't have a single NULL value).
This looks like a bug to me. Anyone knows a way to work around this issue?

The issue is fixed after switching from Legacy SQL to Standard SQL.

Related

How to fix this error: Microsoft.Data.SqlClient.SqlException (0x80131904): Invalid column name 'NormalizedEmail' and etc

My application runs off my database that I created. I added migration, update-database and all that jazz so that it works perfectly fine.
Now that I have to convert my project to use the LIVE database, I'm getting this error message:
Microsoft.Data.SqlClient.SqlException (0x80131904): Invalid column name 'NormalizedEmail'.
Invalid column name 'ConcurrencyStamp'.
Invalid column name 'LockoutEnd'.
Invalid column name 'NormalizedEmail'.
Invalid column name 'NormalizedUserName'.
Invalid column name 'UserType'.
I'm not sure how to go about fixing this but when I use my old database it works, with this new one it just keeps giving me this error when I try to log in a user or anything to do with my database.
Help please!
ThANK YOU!
Remove this columns from code
I think it's mismatch with database
look at this
The problem is very simple to state and very difficult to solve. But there IS a solution.
Manual solution
If all automatic approaches fail and you do not have any extra information, then you can at least ensure that your schema is technically compatible with the application's expectations.
Depending on the RDBMS that you use and which was not specified in the question, you can get all the table names and column names. In MySQL and PostgreSQL you could query information_schema.columns, in SQL Server you can join sys.tables and sys.columns for that purpose. Make sure that you order the results by tablename, columnname and export it. Do it both for your old db and prod db. Find out what the differences are and implement alter statements to add the missing columns.
Automatic solution
If you have some scripts versioned somewhere that were doing the alters and possibly filling the new columns with data, then run those either by hand or a migration tool. Make sure that if such files exist, then you find them.
Removal
You can also remove the column references from code, as Cemil suggested in his/her answer, but you should avoid doing this, unless you are absolutely sure that it is feasible for your situation. The basic assumption is that the code references these for a reason and you are missing the columns from the database where they need to be created. Do not remove the column references from code until this basic assumption is proven wrong.

exporting the data gives error when the type is 'smalldatetime' and value is null

In SQL server, I have a table with 'smalldatetime' data type for one of the column. When I am trying to export the data using Microsoft SQL server import/export wizard, it is giving me conversion error because the field contains NULL value in Date field. Can you please help me in resolving this error.
What steps need to be followed in order to resolve the conversion error.
Are you trying to copy "table to table" or "query to table"?
Cause if your problem is only with null values you can write a query and use ISNULL() function.
Can you post the error here? I´m asking cause maybe your problem isn´t it. Maybe you´re trying to convert a type into another that is not compatible.

Deleting rows in BigQuery fails with "Invalid schema update"

I'm trying to delete some rows from a BigQuery table (using standard SQL dialect):
DELETE FROM ocds.releases
WHERE
ocid LIKE 'ocds-b5fd17-%'
However, I get the following error:
Query Failed
Error: Invalid schema update. Field packageInfo has changed mode from REQUIRED to NULLABLE
Job ID: ocds-172716:bquijob_2f60927_15d13c97149
It seems as though BigQuery doesn't like deleting rows with a REQUIRED column. Is there any way around this?
It has been a known limitation that BigQuery DML doesn't work with tables with required fields (see https://cloud.google.com/bigquery/docs/reference/standard-sql/data-manipulation-language#known_issues).
We are in the process of removing this limitation. We whitelisted your project today. Please try running your query again in the same project. Let us know if the problem is still there, or if you want to have more projects whitelisted.

DB2 LOAD Modifier - GeneratedOverride or IdentityOverride

I am performing a DB2 load, and I am struggling to understand the impact of using GeneratedOverride over IdentityOverride. When I run the following command:
db2 load from tab123.ixf of ixf replace into application.table_abc
All rows are rejected, with the following error being the culprit:
SQL3550W The field value in row row-number and column column-number is not NULL, but the target column has been defined as GENERATED ALWAYS.
So to try and step around this, I executed
:
db2 load from tab123.ixf of ixf modified by identityoverride replace into application.table_abc
But this immediately returned this error:
SQL3526N The modifier clause "IDENTITY OVERRIDE" is inconsistent with the current load command. Reason code: "3".
From checking the reason code I see that the issue is "Generated or identity related file type modifiers have been specified but the target table contains no such columns." .. but the SQL3550W error seems to infer that the columns are generated always!
The only way I can get these rows to commit to the table is to run..
db2 load from tab123.ixf of ixf modified by generatedoverride replace into application.table_abc
Can anyone enlighten me to why I am recieving the SQL3526N error, or what the implications of running generatedoverride are?
Thanks for sticking with me..
Generated columns are not necessarily identity columns, apparently that's the case in your situation. Check the CREATE TABLE syntax to see what are other ways to generate column values.
By using the GENERATEDOVERRIDE option during the load you are obviously replacing (overriding) the generated values with those from the input file.

SQL Server reports 'Invalid column name', but the column is present and the query works through management studio

I've hit a bit of an impasse. I have a query that is generated by some C# code. The query works fine in Microsoft SQL Server Management Studio when run against the same database.
However when my code tries to run the same query I get the same error about an invalid column and an exception is thrown. All queries that reference this column are failing.
The column in question was recently added to the database. It is a date column called Incident_Begin_Time_ts .
An example that fails is:
select * from PerfDiag
where Incident_Begin_Time_ts > '2010-01-01 00:00:00';
Other queries like Select MAX(Incident_Being_Time_ts); also fail when run in code because it thinks the column is missing.
Any ideas?
Just press Ctrl + Shift + R and see...
In SQL Server Management Studio, Ctrl+Shift+R refreshes the local cache.
I suspect that you have two tables with the same name. One is owned by the schema 'dbo' (dbo.PerfDiag), and the other is owned by the default schema of the account used to connect to SQL Server (something like userid.PerfDiag).
When you have an unqualified reference to a schema object (such as a table) — one not qualified by schema name — the object reference must be resolved. Name resolution occurs by searching in the following sequence for an object of the appropriate type (table) with the specified name. The name resolves to the first match:
Under the default schema of the user.
Under the schema 'dbo'.
The unqualified reference is bound to the first match in the above sequence.
As a general recommended practice, one should always qualify references to schema objects, for performance reasons:
An unqualified reference may invalidate a cached execution plan for the stored procedure or query, since the schema to which the reference was bound may change depending on the credentials executing the stored procedure or query. This results in recompilation of the query/stored procedure, a performance hit. Recompilations cause compile locks to be taken out, blocking others from accessing the needed resource(s).
Name resolution slows down query execution as two probes must be made to resolve to the likely version of the object (that owned by 'dbo'). This is the usual case. The only time a single probe will resolve the name is if the current user owns an object of the specified name and type.
[Edited to further note]
The other possibilities are (in no particular order):
You aren't connected to the database you think you are.
You aren't connected to the SQL Server instance you think you are.
Double check your connect strings and ensure that they explicitly specify the SQL Server instance name and the database name.
In my case I restart Microsoft SQL Sever Management Studio and this works well for me.
If you are running this inside a transaction and a SQL statement before this drops/alters the table you can also get this message.
I eventually shut-down and restarted Microsoft SQL Server Management Studio; and that fixed it for me. But at other times, just starting a new query window was enough.
If you are using variables with the same name as your column, it could be that you forgot the '#' variable marker. In an INSERT statement it will be detected as a column.
Just had the exact same problem. I renamed some aliased columns in a temporary table which is further used by another part of the same code. For some reason, this was not captured by SQL Server Management Studio and it complained about invalid column names.
What I simply did is create a new query, copy paste the SQL code from the old query to this new query and run it again. This seemed to refresh the environment correctly.
In my case I was trying to get the value from wrong ResultSet when querying multiple SQL statements.
In my case it seems the problem was a weird caching problem. The solutions above didn't work.
If your code was working fine and you added a column to one of your tables and it gives the 'invalid column name' error, and the solutions above doesn't work, try this: First run only the section of code for creating that modified table and then run the whole code.
Including this answer because this was the top result for "invalid column name sql" on google and I didn't see this answer here. In my case, I was getting Invalid Column Name, Id1 because I had used the wrong id in my .HasForeignKey statement in my Entity Framework C# code. Once I changed it to match the .HasOne() object's id, the error was gone.
I've gotten this error when running a scalar function using a table value, but the Select statement in my scalar function RETURN clause was missing the "FROM table" portion. :facepalms:
Also happens when you forget to change the ConnectionString and ask a table that has no idea about the changes you're making locally.
I had this problem with a View, but the exact same SQL code worked perfectly as a query. In fact SSMS actually threw up a couple of other problems with the View, that it did not have with the query. I tried refreshing, closing the connection to the server and going back in, and renaming columns - nothing worked. Instead I created the query as a stored procedure, and connected Excel to that rather than the View, and this solved the problem.