I have FK and PK all over my db and table data needs to be specified in a certain order or else I get FK/PK insertion errors. I'm tired of executing the wizard again and again to transfer data one table at a time.
In the SQL Server export data wizard there is an option to "Write a query to specify the data to transfer". I'd like to write the query myself and specify the correct order.
Will this solve my problem?
How do I do this? Can you provide a sample query (or link to one)
The databases are on two different servers - SQL Server 2008 on each ; The database names & permissions are the same ; each table name & col is the same ; I need Identity Insert for each table.
Disable foreign keys before importing, enable them after the import:
ALTER TABLE tablename NOCHECK CONSTRAINT ALL
ALTER TABLE tablename WITH CHECK CHECK CONSTRAINT ALL
Update: Thanks for the comments, I fixed the syntax.
You could always save the package and then open and edit the package to put things in the right order (you might have to copy the data flow several times and put dependencies between them)
You can use 3rd party tools to transfer a data; these tools disable/enable constraints automatically.
Related
I am new to SSIS packages and just require assistance on how to transfer data from one data source onto my own database.
Below is my data flow:
Now I have a ODBC Source (Http_Requests Source) where I take data from a PostgreSQL database table (see screenshot below for table columns and data):
Below is the OLE DB destination where it has the table I want to transfer the data to (this table is currently blank):
Now I tried to start debugging to extract the data but I get a few errors (displayed below):
I am a complete novice so I would like some guidance on what I need to include in order to get this SSIS package to transfer data across. Would I need to include a merge statement and how do I apply it. I heard you can write a merge as a proc and call on the proc as a sql command. Does that mean I will need to write a proc in SSMS and then call on it within the OLE DB Destination?
If somebody can provide an example and screenshot then that would be very helpful as I am really new to SSIS.
Thank you,
Check constraint on destination table or disable them before running it.
Below are query you can use.
-- Disable all table constraints
ALTER TABLE YourTableName NOCHECK CONSTRAINT ALL
-- Enable all table constraints
ALTER TABLE YourTableName CHECK CONSTRAINT ALL
Tick keep identity
box or drop primary key on the table. After you apply the changes do not forget to refresh metadata by opening the mappings in sis.
the error means that PerformanceId is an IDENTITY column on your destination table. IDENTITY columns are read only unless you tell it otherwise. So if we were in tSQL to be able to insert IDENTITY we would turn on IDENTITY_INSERT. Because you are in SSIS you can accomplish the same thing by checking the "keep identity" box.
HOWEVER when ever you get an error like this it is usually a sign that you should NOT be mapping ID to Performance ID. The question you have to ask is the Identity from your source supposed to be the identity of the destination table? Usually not, most of the time it would be another column as a surrogate key. Then you have to understand if it is even possible. because if there is a unique constraint or primary key then the identity cannot repeat which means you have to know that your source's id column will not cause a duplicate primary key violation.
More than likely the actual fix if for you to uncheck ID from the source and ignore the value.
The column PerformanceID (in the target) is almost certainly an identity column and that is why it is not working. You may not want to transfer it (and have SQL Server generate values for PerformanceID or you can check 'Keep Identity.'
Using MS SQL Server Management Studio 2008.
I have a db (say ip 10.16.17.10 and called db1) and a second one (say ip 10.16.17.25 called db2).
I am trying to copy one table (and its contents) from db1 into db2.
I have the database on both (but empty in db2).
The problem is no matter how I copy/export/import, no matter what options I set in MS SQL Server Management Studio 2008 when I click 'table'->'Design' (on db2) it ALWAYS says 'Identity Spefication: NO' even tho the db1 table has it on.
From db1 I go to 'Tasks'->'export'->'source/db' and 'destination/db'->'Edit Mapping'->'Enable identity Insert' and click it on.
But no joy. ALWAYS exports without it.
I try similar thing from IMPORT on db2. Similar thing if I use COPY.
I have read MANY of the STACKOVERFLOW articles on this, they all suggest setting IDENTITY_INSERT setting to ON but when I do run below:
SET IDENTITY_INSERT [dbo].[mytable] ON
The table either doesn't exist yet or has already copied WITHOUT the identity setting on so see the error:
does not have the identity property. Cannot perform SET operation.
I have tried setting it as a property (under database properties) for db2 but when I copy/import/export never works.
Would appreciate any help here as lots of StackOverflow articles so far all seem to be having an easier time than me.
I am planning on doing this for another 50 or so tables in this database so am hoping to find a way which doesnt involve running scripts for each table.
thanks
The process of using the Export Data Wizard to copy the data from one table to another will NOT replicate all aspects of the schema (like identity and auto-increment). If you want to replicate the schema, script out your table into a create statement, change the name to db2, and create it. Then you should be able to run the export/import wizard with the identity insert option on and insert into your new table that replicates the schema of your old table.
Ended up sorting this out using MS SQL Management Studio.
Thanks to #kevin for the help regarding Import Data and Export Data. Schemas are NOT transferred across however they are the best means to transport the data once schema is up.
Found best way to MASS import/export db table schemas using below (Saved SQL create scripts to file):
Tasks->Generate Scripts->All Tables To File->with Identity on
Ran 200kb SQL file on db2 for schema.
Then ran Import Data from db1 to db2.
Done, all Identity_Inserts maintained.
thanks for help
According to the Error message I think your table does not have an IDENTITY column. Make sure that [dbo].[mytable] does have an IDENTITY column before you executing SET IDENTITY_INSERT.
SET IDENTITY_INSERT [dbo].[mytable] ON
DEMO1 (Trying to set identity ON when there is NO identity column)
--Error
'Table 'T' does not have the identity property. Cannot perform SET operation.: SET IDENTITY_INSERT T ON'
DEMO2 (Trying to set identity ON when there is identity column)
--No Errors
Follow following Steps :
From db1 I go to 'Tasks'->'export'->'source/db' and 'destination/db'->'Edit Mapping'->'Enable identity Insert' and Edit SQL - > You will able to see query structure of Table.
IN the query for eg. ID int NOT NULL, do the next step ID int NOT NULL IDENTITY(1,1)
Then proceed.
I bet it will work.
I know you can go into the design view of a table in SQL Server Management Studios and reorder columns as they appear in the design view, however this isn't possible with SQL Azure as the option is disabled. Is there a way to modify SQL Azure tables so that you can reorder their columns as they appear in the design view?
I have been running a number of database upgrades over the last few months to support new requirements and would like to reorder the way the columns appear in design view so they're easier to read, i.e. so they start with a primary key, followed by foreign keys, then normal columns and end with the added by, modified by fields. Its purely to make the tables more readable as I manage them over time.
Just run a script against the table. Its a bit of pseudocode but you should get the idea.
CREATE TABLE TableWithDesiredOrder(PK,FK1,FK2,COL1,COL2)
INSERT INTO TableWithDesiredOrder(PK,FK1,FK2,COL1,COL2....)
SELECT PK,FK1,FK2,COL1,COL2.... FROM OriginalTable
DROP TABLE OriginalTable
Finally Rename the table
sp_Rename TableWithDesiredOrder, OriginalTable
Just another option: I use SQL Delta to propagate my db changes from dev db up to Azure db. So in this case, I just change the col order locally using SSMS GUI, and SQL Delta will do the createnew>copytonew>dropold for me, along with my other local changes. (In Project Options, I set Preserve Column Order=Yes.)
I experienced the same with Azure SQL Database, basically my view changes with ALTER were not taken when did a SELECT * from the view, or the column headers were mixed with the column values.
In order to fix it I dropped the view and re-created it again. That worked.
On our original design we screwed up a foreign key constraint in our table. Now that the table is full of data we cannot change it without dropping all of the records in the table. The only solution I could think of is to create a backup table and put all of the records in there, then delete all the records, alter the table and start adding them back. Any other (BETTER) ideas? Thanks!
Using MS SQL Server
I'm a bit late, just for reference.
If You are using SQL Server Management Studio, You could generate a DROP and RECREATE script with "Keep schema and data" option.
Right click on the desired DB in object explorer
Tasks > Generate scripts
Select the table you want to script
Then clicking on Advanced button
"Script DROP and CREATE" ->"Script DROP and CREATE"
"Types of data to script" -> "Schema and data"
Hope this helps
Here's some pseudo-code. No need to make a backup table, just make a new table with the right constraint, insert your records into it, and rename.
CREATE TABLE MyTable_2
(...field definitions)
<add the constraint to MyTable_2>
INSERT INTO MyTable_2 (fields)
SELECT fields
FROM MyTable
DROP TABLE MyTable
exec sp_rename 'MyTable2', 'Mytable'
Using SQL Server Management Studio (SSMS), you can use it's table designer to specify the final condition of the table. Before saving the changes, have it generate the change script and save that script. Cancel out of the design window, open the script and review it. SSMS may already have generated a script that does everything you need, fixing the primary-foreign key relationship while preserving all existing data. If not, you will have a script, already started, that performs most of what you need to do and should be able to modify it for your needs.
This is your only solution.
Create the backup table, empty the original one, modify the table and then insert step-by-step until you find a violation.
Update All Schema Database Old by new Schema Database .
Create script (Right click on the desired DB in object explorer Tasks > Generate scripts -> select option select specific database objects and tables ->next -> advanced-> option Type of data to script Data only -> ok ->next ->next.) to data only and backup Database to old database
Drop database old and create new database and make new DB is empty .
Excute script of Old Data only on new database .
Im trying to copy data only between two SQL server 2008 databases. I need to keep the existing stored procs and functions intact and copy data only. The DB schemas are identical but im running into issues with PK's.
I first tried:
EXEC sp_MSForEachTable 'ALTER TABLE ? NOCHECK CONSTRAINT ALL'
EXEC sp_MSForEachTable 'DELETE FROM ?'
To remove all data. But get
Failure inserting into the read-only column
So i then tried to set IDENTITY_INSERT ON across all tables with:
EXEC sp_MSForEachTable 'ALTER TABLE ? NOCHECK CONSTRAINT ALL'
EXEC sp_MSForEachTable 'DELETE FROM ?'
EXEC sp_MSForEachTable 'ALTER TABLE ? SET IDENTITY_INSERT ON'
with no luck.
What is the best way to export data only between two databases, leaving the original procs and functions intact?
Thanks.
Edit: Im using SQL Export to copy the data from source to destination. I need to keep the destinations DBs procs and functions, just copy the data only.
Just remove the identity specification from all the table pkeys in the second db.
What is likely happening here is that you have pkey as an identity column in both dbs, and it makes sense to do so in the first, but you cant copy its value into another identity column.
You wouldn't want the pkey as an identity pkey in the second db anyway, then, all your foreign keys wouldn't work.
I would probably approach it from a different angle: by scripting all objects via SQL Enterprise Manager into a file and running this file on a blank database. This way, you'll have all metadata but no actual data in the second database, and you can use it for additional copies in the future.
The error you are getting doesn't seems like a PK violation or an Identity issue. I see two possible causes:
If you are getting the error when trying to insert the data, I would check if the tables have any computed columns. Many programs fail to take them into account when exporting data, and include the computed columns in the insert column list.
If you are getting that error in the delete step, probably you have a trigger that fires on delete, and it try to insert data and fails for some reason (the idea of these triggers is maintain a copy of the deleted data in another location). If that is the case, fix the insert or just disable the trigger.
I went with a varation of both answers this in the end. I used a 3rd database as a temp database.
1)I did a full back up of the database i needed the data from (live)
2)I restored this backup to my temp database.
3)I scripted the database i needed the procs and functions from, only scripting procs and funcs and using DROP and IF INCLUDES.
4)I ran the script from #3 against my temp database giving the data from DB1 and the procs and funcs from DB2
5)I restored DB2, using OVERWRITE from a backup of my temp database.
Thanks guys id mark all as correct if I could.
Hi in order to get around your issues with your constraints, please read this blog post I wrote on the subject.
http://tenbulls.co.uk/2009/07/22/checking-your-constraints-to-check-your-integrity/