I created a SQL Server database first (2 tables) and then tried to load data through SSIS data flow task. At the last step an error has occurred.
When I remove a relationship between two tables in the database, the SSIS task is completed successfully and the data is loaded! But, after I load data into the tables, I can't create relationship between them.
Based on this you can conclude that a relationship can be created when there is no data in a table. Just to mention, data types are the same in both tables.
How could I work out a solution?
Thank you!
It seems the error in SSIS is due to a foreign key violation. The purpose of the foreign key relationship is to prevent you for loading bad data. When you loaded without the FK, you inserted bad data and cannot create a (trusted) foreign key constraint afterward.
The solution is to either fix the source data or modify your package to avoid inserted data that doesn't exist in the referenced table. The latter can be done with a lookup task, sending found rows down the happy path to the target table. You could either ignore not found rows or write those to an error table or file.
Related
For a project i need to to delete all data from all tables in a database,
I tried
DELETE FROM table1,table2,table3,...;
But it doesnt work. Any advice ? Thank
I would like to refer You to this related post
How do I use cascade delete with SQL Server?
as You will find there several possible solutions.
When using SQL it means that Your data is relations, which means most of the records are somehow related in the different tables and this relation is expressed with foreign keys. However when attempting to delete data which is id is related with data in another table a cascade deletion should be implemented, the other way around it is add additional boolean column named isDeleted(as example ofcourse) and just alter specific rows to true in this specific column and then filter by preferences. Hopefully I have managed somehow to provide with alternative and/or possible solution to Your problem.
Leaving also this link which gives some examples on cascade deletion and guide on how to implement it. ->
https://www.sqlshack.com/delete-cascade-and-update-cascade-in-sql-server-foreign-key/
P.S. also if You want just to DELETE all the data You can either use TRUNCATE TABLE or DROP DATABASE query. With the latest option You will have to recreate the database once more.
Because you want to delete all databases from all tables, you are essentially deleting the database.
The sqlite way of doing this is to just delete the database file.
In contrast to postgres or mongodb or most databases, sqlite doesn't have a server so no need to call DROP DATABASE.
As soon as the file is gone and you open the database from your application you will be in a blank state again.
I am new to SSIS packages and just require assistance on how to transfer data from one data source onto my own database.
Below is my data flow:
Now I have a ODBC Source (Http_Requests Source) where I take data from a PostgreSQL database table (see screenshot below for table columns and data):
Below is the OLE DB destination where it has the table I want to transfer the data to (this table is currently blank):
Now I tried to start debugging to extract the data but I get a few errors (displayed below):
I am a complete novice so I would like some guidance on what I need to include in order to get this SSIS package to transfer data across. Would I need to include a merge statement and how do I apply it. I heard you can write a merge as a proc and call on the proc as a sql command. Does that mean I will need to write a proc in SSMS and then call on it within the OLE DB Destination?
If somebody can provide an example and screenshot then that would be very helpful as I am really new to SSIS.
Thank you,
Check constraint on destination table or disable them before running it.
Below are query you can use.
-- Disable all table constraints
ALTER TABLE YourTableName NOCHECK CONSTRAINT ALL
-- Enable all table constraints
ALTER TABLE YourTableName CHECK CONSTRAINT ALL
Tick keep identity
box or drop primary key on the table. After you apply the changes do not forget to refresh metadata by opening the mappings in sis.
the error means that PerformanceId is an IDENTITY column on your destination table. IDENTITY columns are read only unless you tell it otherwise. So if we were in tSQL to be able to insert IDENTITY we would turn on IDENTITY_INSERT. Because you are in SSIS you can accomplish the same thing by checking the "keep identity" box.
HOWEVER when ever you get an error like this it is usually a sign that you should NOT be mapping ID to Performance ID. The question you have to ask is the Identity from your source supposed to be the identity of the destination table? Usually not, most of the time it would be another column as a surrogate key. Then you have to understand if it is even possible. because if there is a unique constraint or primary key then the identity cannot repeat which means you have to know that your source's id column will not cause a duplicate primary key violation.
More than likely the actual fix if for you to uncheck ID from the source and ignore the value.
The column PerformanceID (in the target) is almost certainly an identity column and that is why it is not working. You may not want to transfer it (and have SQL Server generate values for PerformanceID or you can check 'Keep Identity.'
I am creating this SSIS import package for about 10 tables , I am still new to this so I really appreciate any help I can get.
I need to compare my Excel source to this ~10 tables to see if any record exists ,if it exists then update it or else insert it. I am struggling on how to check on various tables where they all have auto-incremented PK. If one record doesn't exist how can I insert it and make sure the other tables have the foreign keys(auto-incremented primary key of tables) updated as well.Meaning the relationship of each record that have been divided into so many tables are tact.
My plan for the package:
Excel source
Look up transformer
Data conversion transformer
derived column transformer
multicast
OLE DB destination
Please advise on how I should go about, and the order I should follow for my transformers.
hmm ok firstly get the excel source into an sql staging table first (truncate before loading) then you could consider using the sql merge statement via an execute sql task to merge the data into the end tables. This will allow you to insert if the record dosent exist. You may need to lookup the foreign keys before running the merge. Are you able to post details of the 10 tables and import csv?
I have an issue related to data in sql server. In my database some of the constraint were not enabled i.e. they were not checked , After some time working on it we found this issue that a parent rows can be deleted without deleting child, which was an issue. I enabled all the constraint in the database using query
ALTER TABLE tbl_name CHECK CONSTRAINT ALL
above query was executed on all the tables of that database without any error . But my concern is whether it will work or not , if it will work on the existing data then what will happen to that data whose parent table data has been deleted.
I want to know is there any way such that I can validate such data data whose parent record doesn't exist in the entire database. There are about 270 constraint containing FOREIGN KEY AND UNIQUE KEY . I don't want to go for manual option.
Please help me out.
ALTER TABLE tbl_name CHECK CONSTRAINT ALL
only re-enables the constraints. Importantly, the constraints are not checked against the existing data in the database (nor are they trusted by the optimizer). If you want that to occur, you need to specify WITH CHECK as well:
ALTER TABLE tbl_name WITH CHECK CHECK CONSTRAINT ALL
(And yes, the word CHECK appears twice)
If you execute this, and there are orphaned child rows (or other invalid constraints), then the ALTER TABLE will fail with an error message. There's nothing SQL Server can do to fix this issue - it's for you to decide whether to a) remove the orphaned rows, or b) to re-create, in some manner, a suitable parent row for them.
You can also add the 'ON DELETE CASCADE' code to the end of foreign keys to prevent orphaned child rows from persisting.
This is more of a 'better practice' going forward than a solution, but I believe Damien_The_Unbeliever has answered your main question.
We have developed a large data migration from one DB schema to the other. We had built it based on the idea that the destination DB would be empty, however months ago we started putting clients on the new application which means their data is being housed in the new schema (the destination DB).
Now we're in a situation where the primary keys could overlap from the source to the destination DB and we're struggling to come up with a solution. The only solution I can think of is to check if the ID exists in the destination, updated the ID in the source to be 1 more than the greatest ID in the destination, and then migrate the record. This seems really cumbersome to have to do for hundreds of tables. Any ideas?
Sorry I don't know anything about SSIS but the following are a few ways to solve the problem using SQL.
When inserting into the destination tables, do not insert identities. As rows are inserted, capture the newly inserted identities and the old identities in a mapping table, see MERGE + OUTPUT INTO. Use the mapping table to update the tables that haven't been inserted, substituting the old identities with the new identities.
Of course for this to work, insertion into tables has to be done in an order that won't cause foreign key or constraint violations.
If you're not into doing all that, and you can lock users out of tables for short periods of time, DBCC CHECK INDENT could be used to 'reserve' identities. These new identities can then be used to update the old data and then insert with SET IDENTITY_INSERT ON.