I am trying to import data from Oracle to SQL Server through SSIS package data flow task.
One column has special characters and that column is also part of composite key in that table.
So after loading the data on SQL when I am trying to enforce the uniqueness it fails as while loading the data the special characters are getting converted to something else.
Is there any property or alternative so that the special characters are imported the way they are in Oracle.
Thanks in advance.
This seems to be a common issue from my experience.
I will try to describe some steps that might help you.
First, you need to disable the primary key column and run the dtsx again.
Second, you must locate the identical values which cause the primary key violation.
This means that two different chars are mapped to the same char probably and produces the error.
Take in mind that SSIS cannot read with UTF-8 from Oracle, you can see it for your self if you go to Advanced View in Oracle Source Object and inspect the input/output columns, usually, the code page is 1251.
The only solution is to use a Derived Column and replace the problematic character manually before the insert. Also, it will help more if you do this solution.
OracleDB->File->Derived Column->SQLServer
Related
My application runs off my database that I created. I added migration, update-database and all that jazz so that it works perfectly fine.
Now that I have to convert my project to use the LIVE database, I'm getting this error message:
Microsoft.Data.SqlClient.SqlException (0x80131904): Invalid column name 'NormalizedEmail'.
Invalid column name 'ConcurrencyStamp'.
Invalid column name 'LockoutEnd'.
Invalid column name 'NormalizedEmail'.
Invalid column name 'NormalizedUserName'.
Invalid column name 'UserType'.
I'm not sure how to go about fixing this but when I use my old database it works, with this new one it just keeps giving me this error when I try to log in a user or anything to do with my database.
Help please!
ThANK YOU!
Remove this columns from code
I think it's mismatch with database
look at this
The problem is very simple to state and very difficult to solve. But there IS a solution.
Manual solution
If all automatic approaches fail and you do not have any extra information, then you can at least ensure that your schema is technically compatible with the application's expectations.
Depending on the RDBMS that you use and which was not specified in the question, you can get all the table names and column names. In MySQL and PostgreSQL you could query information_schema.columns, in SQL Server you can join sys.tables and sys.columns for that purpose. Make sure that you order the results by tablename, columnname and export it. Do it both for your old db and prod db. Find out what the differences are and implement alter statements to add the missing columns.
Automatic solution
If you have some scripts versioned somewhere that were doing the alters and possibly filling the new columns with data, then run those either by hand or a migration tool. Make sure that if such files exist, then you find them.
Removal
You can also remove the column references from code, as Cemil suggested in his/her answer, but you should avoid doing this, unless you are absolutely sure that it is feasible for your situation. The basic assumption is that the code references these for a reason and you are missing the columns from the database where they need to be created. Do not remove the column references from code until this basic assumption is proven wrong.
I am developing one automation tool for data migration from one table to other table, here I am looking for one function or SP for which I will pass source column and destination column as input parameter. I want output parameter to return true when source column data is compatible to copy to destination column. If not then it should return false.
For example, if a source column is varchar and a destination column is integer, the script should check all the data in a source column in good enough to move to an integer column or not and return the output flag. I want a script to work like this for all types of data types. Any suggestions will be helpful.
If you're on SQL Server 2012 you have TRY_CAST(), TRY_CONVERT() and TRY_PARSE() at your disposal (see this post by Biz Nigatu of blog.dbandbi for comparison).
That said, you still need to check for truncation errors, e.g. by converting to target datatype and back, then comparing the original value with the one after conversions.
I've seen similar tools in the past, might be a good idea to see if one isn't already available online for free. Even a purchase might be less expensive than the time you put into developing and troubleshooting your own tool.
I have an SSIS solution for this that I put together using EzAPI. I've got it posted to GitHub, so feel free to look:
https://github.com/thevinnie/SyncDatabases
Now, the part of that which is relevant to you would be where I use the information schema to make sure that the source and destination are schema matches. If not, C# script task will generate the statement to create, alter or drop the necessary column(s).
The EzAPI part is cool with SSIS because it will allow you to programatically generate an SSIS package. For the project requirements, I needed to be able to load data every time and not let schema changes in the source break the process.
Comments and recommendations are welcome. Hopefully it'll help, but I think you'll be looking at the INFORMATION_SCHEMA.COLUMNS either way.
I'm attempting to run a simple statement against an Access DB to find records.
Data validation in the records was horrible, and I cannot sanitize it. Meaning, it must be preserved as is.
I need to be able to search against a string with white space and hyphen characters removed. The following statement will work in Access 2010 direct:
select * from dummy where Replace(Replace([data1],' ',''),'-','') = 'ABCD1234';
Running it from an ODBC connection via PHP will not. It produces the following error:
SQL error: [Microsoft][ODBC Microsoft Access Driver] Undefined function 'Replace' in expression., SQL state 37000 in SQLExecDirect
Creating a query in the database that runs the function and attempting to search its values indirectly causes the same error:
select * from dummy_indirect where Expr1 = 'ABCD1234';
I've attempted to use both ODBC drivers present. ODBCJR32.dll (03/22/2010) and ACEODBC.dll (02/18/2007). To my knowledge these should be current as it was installed with the full Access 2010 and Access 2010 Database Engine.
Any ideas on how to work around this error and achieve the same effect are welcome. Please note, that I cannot alter the database in way, shape, or form. That indirect query was created in another mdb file that has the original tables linked from the original DB.
* Update *
OleDB did not really affect anything.
$dsn= "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=c:\dummy.mdb;";
I'm not attempting to use it as a web backend either. I'm not a sadomasochist.
There is a legacy system that I must support that does use Access as a backend. Data gets populated there from other old systems that I must integrate into more modern systems. Hence, the creation of an API with Apache/PHP that is running on the server supporting the legacy system.
I need to be able to search a table that has an alphanumeric case identifier to get a numeric identifier that is unique and tied to a generator (Autonumber in access). Users have been using it a trash box for years (inconsistent data entry with sporadic notations) so the only solution I have is to strip everything except alphanumeric out of both the field value and the search value and attempt to perform a LIKE comparison against it.
If not replace() which is access supported, what ODBC compatible functions exist that I can use do the same kind of comparison?
Just to recap, the Access db engine will not recognize the Replace() function unless your query is run from within an Access application session. Any attempt from outside Access will trigger that "Undefined function" error message. You can't avoid the error by switching from ODBC to OleDb as the connection method. And you also can't trick the engine into using Replace() by hiding it in separate query (in the same or another Access db) and using that query as the data source for your main query.
This behavior is determined by Access' sandbox mode. That linked page includes a list of functions which are available in the default sandbox mode. That page also describes how you can alter the sandbox mode. If you absolutely must have Replace() available for your query, perhaps the lowest setting (0) would allow it. However, I'm not recommending you do that. I've never done it myself, so don't know anything about the consequences.
As for alternatives for Replace(), it would help to know about the variability in the values you're searching. If the space or dash characters appear in only one or a few consistent positions, you could do a pattern match with a Like expression. For example, if the search field values consist of 4 letters, an optional space or dash, followed by 4 digits, a WHERE clause like this should work for the variations of "ABCD1234":
SELECT * FROM dummy
WHERE
data1 = 'ABCD1234'
OR data1 Like 'ABCD[- ]1234';
Another possibility is to compare against a list of values:
SELECT * FROM dummy
WHERE
data1 IN ('ABCD1234','ABCD 1234','ABCD-1234');
However if your search field values can include any number of spaces or dashes at any position within the string, that approach is no good. And I would look real hard for some way to make the query task easier:
You can't clean the stored values because you're prohibited from altering the original Access db in any way. Perhaps you could create a new Access db, import the data, and clean that instead.
Set up the original Access db as a linked server in SQL Server and build your query to take advantage of SQL Server features.
Surrender. :-( Pull in a larger data set to your PHP client code, and evaluate which rows to use vs. which to ignore.
I'm not sure you can do this with ODBC and your constraints. The MS Access driver is limited (by design; MS wants you to use SQL Server for back ends).
Can you use OLEDB? that might be an option.
I recently updated an SSIS package that had been working fine and now I receive the following error:
Text was truncated or one or more characters had no match in the target code page.
The package effectively transferred data from tables in one database to a table in another database on another server. The update I made was to add another column to the transfer. The column is Char(10) in length and it is the same length on both the source and destination server. Before the data is transferred it Char(10) there as well. I've seen people reporting this error in blog posts as well as on Stack, none of what I have read has helped. One solution I read about involved using a data conversion to explicitly change the offending column, this did not help (or I misapplied the fix).
whihc version of SQl Server and SSIS are you usign?
I would say to take a look at the output and imput fields of your components. CHAR always ocupies all it's length (I mean, char(10) will always use 10 bytes) and since you are having a truncation error, it may be a start. try to increase the size of the field or cast as varchar on the query that loads the data (not as a permanet solution, just to try to isolate the problem)
Which connection you are using ADO.Net or OLEDB connection ??
Try deleting the source and destination if there are not much of changes you have to make ..Sometime the metadata cuases this problems. If this doesn't solve your problem post the screen shot of error.
Does anyone have an idea what this error means or how to solve it? I am using Access 2003 and SQL2005. It comes up when trying to add a record on a particular subform.
[Microsoft][SQL Native Client] Invalid character value for cast specification (#0)
This MS bug report describes the same message, but it is a bug in SQL Server 6.5 that has already been solved.
Solved: Apparently having no PK on the destination table was causing this, it didn't have anything to do with the subform or the query from Access. I wasn't even aware there were tables in this database without PK. Adding PK to the destination table solved it. The strange thing is the same query string that errored when executed via SQL native client, executed through SSMS with no errors. Hope this helps anyone else who has come across that strange message.
Hum, I would check the text box default on the access side. I would also bring up the linked table in design mode, and you want to check the data type that ms-access assumes here. For non supported data types ms-access will generally use a string, and sql server might be wanting something else.
So, check both the Primary key (PK) in main table, and then check the data type used (assumed) in the child table for the foreign key (FK) column. While we are at this, check your expressions used for the child/master link settings in the sub-form control (not the form, not the sub-form, but the sub-form control used in your form that links up these two tables).
Sub forms in access are sensitive if you don’t have a timestamp column in the sql server table. As mentioned check the PK and the FK data types and make sure they match up (just bring up the tables in design mode in ms-access -- you get an error message about the design mode being read only, but just continue on so you can check/view to ensure the data types match up).
So for the child table, you need a PK, a FK, and also a timestamp column (you don’t have to display the TS column in the sub-form, but you need it in the table).
Sub-forms in ms-access are sensitive and often fail if you don’t include a timestamp column in the sql table. (access uses these row version columns to determine if the data been changed).
Is one of your fields in the view calculated/built with the CAST function? In this case, you might not have the right to update/add a value for that field.
Can you execute your view in the MS SQL Studio interface and try to insert a record?
Another cause to this issue is that if you change a table name without alterting the view then the "Dependencies" of that view still remians with the table old name.
Let say I have a table 'A' and a view 'Av' which derives from 'A', and I created a new Table which will be named 'A' and I changed 'A's name to 'A_old' but I didn't executed an ALTER VIEW, so the dependencies of 'Av' still remain on 'A_old' but the view is derives from 'A' and it cuasing this Error in Access when trying to open the view as a linked table
I just spent a day battling this with an Access ADP project that was imported into a new Access 2016 ACCDB file. Initially I figured it was an issue with the application code, but I was getting this keying records directly into the table. Interestingly, the records always got written - it seemed to be the read-back that was triggering the error. Profiling the insert sql and running that from SQL Management Studio worked without any issues.
The table that was causing the problems had a GUID Primary Key. Switching that to an int column resolved the issue.
The SQL database was also littered with a few thousand extended properties which I removed before switching the PK. There was a strong suggestion from the web that these cause problems. The source of that process is documented here: Remove All SQL Extended Properties
I had this problem with Access 2016 trying to update an ODBC linked sQL Server database. Problem was a null value in field used to join the two tables. Eliminating the null value solved the problem
OK I just had this bad experience and it had nothing to do with PK or any of this stuff in my situation. The view that reported this problem in Access was created in SQL Server originally and used a CAST of DATETIME to plain old DATE to get rid of the unneeded time part. Up until today this view had caused 0 issues in Access, but started to generate heartburn just as described above.
So, I generated a Drop/Create script for the MSS view, ran it, relinked the views in Access, and the Access database was happy with the result. All my so-called tables in Access are basically views through links to MSS for reporting. I only have 1 table that actually does changes. Other than that, I do not edit through views in Access.
The message is of course useless as usual but this was my solution in my situation.
Based solely in the message you provided above, it appears that you are trying to set an invalid value to some field or parameter, etc... The message is telling you that it is trying to convert a value into an specific data type but the value is invalid for that data type... makes sense?
Please add more details so we can help you better.