Related
What is an appropriate way to structure the database table for the following scenario?
I'm trying to model an incident report, part of it will be recording whether (as part of the incident) an employee or customer had to receive medical attention. The same details are to be recorded for both. In the paper version of the form and in the interface, it is shown as a single table. The table headings would be:
Person (either employee or customer)
Injured
First aid administered
Hospitalised
Details
We already have separate employee and customer tables which old their personal details. The new table would include the ID, details on whether the person was injured, received first aid, hospitalised and any other additional details.
So I've been pondering on a few possibilities of how this could be structured:
Two separate tables for employee and customer incidents. The two tables will hold mostly the same fields with the exception of the foreign keys for EmployeeID or CustomerID.
A single table that includes all the incident related fields and additional fields for EmployeeID and CustomerID where these are nullable foreign keys to the primary keys of the Employee and Customer table respectively. A constraint to ensure that EmployeeID XOR CustomerID is not null.
Create 3 new tables: one to just hold the incidents, one to link customer ID with an incident and another to link employee ID with an incident - is this too normalised? (assuming I will never have more than customers or employees to record incident details for)
A single table that includes all the incident related fields with a single field to store either the employee ID or customer ID and a type field to specify whether it's employee or customer.
I'm leaning towards option 1, it seems to be the cleanest and should different fields need to be recorded for customers and employees, it's easy to update a single table without affecting the other. I've discounted option 4, I think this is bad design since I'm not able to add a FK constraint to the ID column.
Are there any reasons why I shouldn't go for option 1 or are there better alternatives I should also consider?
I would create "link" tables. And not repeat the DDL for the Incident.
then you have the ability to create a view which would bring together customers and employees with the incident data.
I agree with the previous answer to separate the incident details.
-- START TSQL
SET NOCOUNT ON
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'vwIncidents' and TABLE_TYPE = N'VIEW' )
BEGIN
DROP VIEW [dbo].[vwIncidents]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'CustomerToIncidentLink' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[CustomerToIncidentLink]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'EmployeeToIncidentLink' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[EmployeeToIncidentLink]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'Incident' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[Incident]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'Employee' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[Employee]
END
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'Customer' and TABLE_TYPE = N'BASE TABLE' )
BEGIN DROP TABLE [dbo].[Customer]
END
CREATE TABLE [dbo].[Employee] (
[EmployeeUUID] [uniqueidentifier] NOT NULL,
[TheVersionProperty] [timestamp] NOT NULL,
[SSN] [nvarchar](11) NOT NULL,
[LastName] [varchar](64) NOT NULL,
[FirstName] [varchar](64) NOT NULL,
[CreateDate] [datetime] NOT NULL,
[HireDate] [datetime] NOT NULL
)
GO
ALTER TABLE dbo.Employee ADD CONSTRAINT PK_Employee PRIMARY KEY NONCLUSTERED (EmployeeUUID)
GO
ALTER TABLE [dbo].[Employee] ADD CONSTRAINT CK_SSN_Unique UNIQUE (SSN)
GO
CREATE TABLE [dbo].[Customer] (
[CustomerUUID] [uniqueidentifier] NOT NULL,
[TheVersionProperty] [timestamp] NOT NULL,
[CustomerNumber] [nvarchar](11) NOT NULL,
[LastName] [varchar](64) NOT NULL,
[FirstName] [varchar](64) NOT NULL,
[CreateDate] [datetime] NOT NULL
)
GO
ALTER TABLE dbo.Customer ADD CONSTRAINT PK_Customer PRIMARY KEY NONCLUSTERED (CustomerUUID)
GO
ALTER TABLE [dbo].[Customer] ADD CONSTRAINT CK_CustomerNumber_Unique UNIQUE (CustomerNumber)
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = object_id(N'[dbo].[Incident]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
BEGIN DROP TABLE [dbo].[Incident]
END
GO
CREATE TABLE [dbo].[Incident]
(
IncidentUUID [UNIQUEIDENTIFIER] NOT NULL DEFAULT NEWSEQUENTIALID()
, IncidentName varchar(24) not null
, CreateDate smalldatetime not null
)
GO
ALTER TABLE dbo.Incident ADD CONSTRAINT PK_Incident PRIMARY KEY NONCLUSTERED (IncidentUUID)
GO
ALTER TABLE [dbo].[Incident] ADD CONSTRAINT CK_IncidentName_Unique UNIQUE (IncidentName)
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = object_id(N'[dbo].[EmployeeToIncidentLink]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
BEGIN DROP TABLE [dbo].[EmployeeToIncidentLink]
END
GO
CREATE TABLE [dbo].[EmployeeToIncidentLink] (
[LinkSurrogateUUID] [uniqueidentifier] NOT NULL,
[TheEmployeeUUID] [uniqueidentifier] NOT NULL,
[TheIncidentUUID] [uniqueidentifier] NOT NULL
)
GO
ALTER TABLE dbo.EmployeeToIncidentLink ADD CONSTRAINT PK_EmployeeToIncidentLink PRIMARY KEY NONCLUSTERED (LinkSurrogateUUID)
GO
ALTER TABLE [dbo].[EmployeeToIncidentLink] ADD CONSTRAINT FK_EmployeeToIncidentLinkToEmployee FOREIGN KEY (TheEmployeeUUID) REFERENCES dbo.Employee (EmployeeUUID)
GO
ALTER TABLE [dbo].[EmployeeToIncidentLink] ADD CONSTRAINT FK_EmployeeToIncidentLinkToIncident FOREIGN KEY (TheIncidentUUID) REFERENCES dbo.Incident (IncidentUUID)
GO
ALTER TABLE [dbo].[EmployeeToIncidentLink] ADD CONSTRAINT CONST_UNIQUE_EmpUUID_InciUUID UNIQUE (TheEmployeeUUID , TheIncidentUUID)
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = object_id(N'[dbo].[CustomerToIncidentLink]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
BEGIN DROP TABLE [dbo].[CustomerToIncidentLink]
END
GO
CREATE TABLE [dbo].[CustomerToIncidentLink] (
[LinkSurrogateUUID] [uniqueidentifier] NOT NULL,
[TheCustomerUUID] [uniqueidentifier] NOT NULL,
[TheIncidentUUID] [uniqueidentifier] NOT NULL
)
GO
ALTER TABLE dbo.CustomerToIncidentLink ADD CONSTRAINT PK_CustomerToIncidentLink PRIMARY KEY NONCLUSTERED (LinkSurrogateUUID)
GO
ALTER TABLE [dbo].[CustomerToIncidentLink] ADD CONSTRAINT FK_CustomerToIncidentLinkToCustomer FOREIGN KEY (TheCustomerUUID) REFERENCES dbo.Customer (CustomerUUID)
GO
ALTER TABLE [dbo].[CustomerToIncidentLink] ADD CONSTRAINT FK_CustomerToIncidentLinkToIncident FOREIGN KEY (TheIncidentUUID) REFERENCES dbo.Incident (IncidentUUID)
GO
ALTER TABLE [dbo].[CustomerToIncidentLink] ADD CONSTRAINT CONST_UNIQUE_CustomerUUID_InciUUID UNIQUE (TheCustomerUUID , TheIncidentUUID)
GO
CREATE VIEW dbo.vwIncidents
AS
/* combine what is in common with employees and customers for this view */
select e.LastName, e.FirstName, inc.IncidentName
from dbo.Employee e
join
dbo.EmployeeToIncidentLink link on e.EmployeeUUID = link.TheEmployeeUUID
join
dbo.Incident inc on inc.IncidentUUID = link.TheIncidentUUID
UNION ALL
select c.LastName, c.FirstName, inc.IncidentName
from dbo.Customer c
join
dbo.CustomerToIncidentLink link on c.CustomerUUID = link.TheCustomerUUID
join
dbo.Incident inc on inc.IncidentUUID = link.TheIncidentUUID
GO
There is another option (I think?)
Create a Person table, (this has info in common with customers and employees) and then create "sub class" tables for customer and employee that has unique to that entity data.
That's a topic unto itself.
See
How do we implement an IS-A Relationship?
"IS A"
or "how to subclass in a database" would be search tips.
I have this example that is close to yours, that does a "subclassing"
-- START TSQL
SET NOCOUNT ON
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'vwPersonEmail' and TABLE_TYPE = N'VIEW' )
BEGIN
DROP VIEW [dbo].[vwPersonEmail]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'PersonEmail' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[PersonEmail]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'Customer' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[Customer]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'Employee' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[Employee]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'PersonSuperType' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[PersonSuperType]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'Department' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[Department]
END
GO
CREATE TABLE [dbo].[Department](
[DepartmentUUID] [uniqueidentifier] NOT NULL,
[TheVersionProperty] [timestamp] NOT NULL,
[DepartmentName] [nvarchar](80) NULL,
[CreateDate] [datetime] NOT NULL,
[MyTinyInt] tinyint not null
)
ALTER TABLE dbo.[Department] ADD CONSTRAINT PK_Department PRIMARY KEY NONCLUSTERED ([DepartmentUUID])
GO
ALTER TABLE [dbo].[Department] ADD CONSTRAINT CK_DepartmentName_Unique UNIQUE ([DepartmentName])
GO
CREATE TABLE [dbo].[PersonSuperType] (
[PersonSuperTypeUUID] [uniqueidentifier] not null default NEWSEQUENTIALID(),
[LastName] [varchar](64) NOT NULL,
[FirstName] [varchar](64) NOT NULL
)
GO
ALTER TABLE dbo.PersonSuperType ADD CONSTRAINT PK_PersonSuperTypeUUID PRIMARY KEY CLUSTERED (PersonSuperTypeUUID)
GO
CREATE TABLE [dbo].[Employee] (
[EmployeeUUID] [uniqueidentifier] NOT NULL,
[ParentDepartmentUUID] [uniqueidentifier] NOT NULL,
[TheVersionProperty] [timestamp] NOT NULL,
[SSN] [nvarchar](11) NOT NULL,
[CreateDate] [datetime] NOT NULL,
[HireDate] [datetime] NOT NULL
)
GO
ALTER TABLE dbo.Employee ADD CONSTRAINT PK_Employee PRIMARY KEY NONCLUSTERED (EmployeeUUID)
GO
ALTER TABLE [dbo].[Employee] ADD CONSTRAINT CK_SSN_Unique UNIQUE (SSN)
GO
ALTER TABLE [dbo].[Employee] ADD CONSTRAINT FK_EmployeeToDepartment FOREIGN KEY (ParentDepartmentUUID) REFERENCES dbo.Department (DepartmentUUID)
GO
ALTER TABLE [dbo].[Employee] ADD CONSTRAINT FK_EmployeeToPersonSuperType FOREIGN KEY (EmployeeUUID) REFERENCES dbo.PersonSuperType (PersonSuperTypeUUID)
GO
CREATE TABLE [dbo].[Customer] (
[CustomerUUID] [uniqueidentifier] NOT NULL,
[TheVersionProperty] [timestamp] NOT NULL,
[CustomerNumber] [nvarchar](11) NOT NULL,
[CreateDate] [datetime] NOT NULL,
)
GO
ALTER TABLE [dbo].[Customer] ADD CONSTRAINT PK_Customer PRIMARY KEY NONCLUSTERED (CustomerUUID)
GO
ALTER TABLE [dbo].[Customer] ADD CONSTRAINT CK_Customer_CustomerNumber_Unique UNIQUE (CustomerNumber)
GO
ALTER TABLE [dbo].[Customer] ADD CONSTRAINT FK_CustomerToPersonSuperType FOREIGN KEY (CustomerUUID) REFERENCES dbo.PersonSuperType (PersonSuperTypeUUID)
GO
CREATE TABLE [dbo].[PersonEmail] (
[PersonEmailUUID] [uniqueidentifier] NOT NULL,
[PersonSuperTypeUUID] [uniqueidentifier] NOT NULL,
[EmailAddress] [varchar](256) NOT NULL
)
GO
ALTER TABLE [dbo].[PersonEmail] ADD CONSTRAINT PK_PersonEmail PRIMARY KEY NONCLUSTERED (PersonEmailUUID)
GO
ALTER TABLE [dbo].[PersonEmail] ADD CONSTRAINT CK_PersonEmail_EmailAddress_Unique UNIQUE (EmailAddress)
GO
ALTER TABLE [dbo].[PersonEmail] ADD CONSTRAINT FK_PersonEmailToPersonSuperType FOREIGN KEY (PersonSuperTypeUUID) REFERENCES dbo.PersonSuperType (PersonSuperTypeUUID)
GO
CREATE VIEW dbo.vwPersonEmail
AS
/* combine what is in common with employees and customers for this view */
select per.LastName, per.FirstName, [UniqueIdentifier] = emp.SSN, pemail.EmailAddress
from dbo.PersonSuperType per
join
dbo.Employee emp on per.PersonSuperTypeUUID = emp.EmployeeUUID
join
dbo.PersonEmail pemail on per.PersonSuperTypeUUID = pemail.PersonSuperTypeUUID
UNION ALL
select per.LastName, per.FirstName, [UniqueIdentifier] = cust.CustomerNumber, pemail.EmailAddress
from dbo.PersonSuperType per
join
dbo.Customer cust on per.PersonSuperTypeUUID = cust.CustomerUUID
join
dbo.PersonEmail pemail on per.PersonSuperTypeUUID = pemail.PersonSuperTypeUUID
GO
So now that I think about it, I might lean toward the "subclassing" way..since your two types of people might have alot of common attributes, and a few distinct ones.
If you want only to show information about 2 diferent tables in only one, you could use a View. Here you have an example check it out.
https://www.w3schools.com/sql/sql_view.asp => tutorial
https://www.ibm.com/support/knowledgecenter/en/ssw_i5_54/rzatc/rzatcviewmultsql.htm => example
I'd formalize the distinction between the incident and the details.
INCIDENT_HEADER
Customer ID <nullable, fk into the customer table>
Employee ID <nullable, fk into the employee table, which already exsits>
Summary
Date
Status (?)
INCIDENT_DETAIL
DETAIL_TYPE_ID <fk into a type table, it's up next...>
DESCRIPTION
DETAIL_TYPE
ID
DESCRIPTION
DETAIL_TYPE will end up with things like "Injury", "Hospitalized", and "First Aid". You can add more to that table later without touching the details table.
This design lets you add more kinds of details without altering the header table.
I scripted our development database locally. The Stored procedures in our app has code all over the place that checks to see if a table exists and if so, drops the table and recreates it. I noticed that when I run the code in my local database that the check to see if the table exists does not work the same as when running in development. In particular, I noticed that the OBJECTPROPERTY(ID, N'IsUserTable') returns NULL when running locally but returns 1 in DEV.
if exists (select * from [CLASS].DBO.sysobjects where ID = object_ID(N'[CLASS].DBO.[CDB_SCRB_CLIENT_ACCOUNT]') and OBJECTPROPERTY(ID, N'IsUserTable') = 1)
BEGIN
PRINT 'EXISTS!'
drop table [CLASS].DBO.[CDB_SCRB_CLIENT_ACCOUNT]
END
ELSE PRINT 'NO!'
GO
CREATE TABLE [CLASS].DBO.[CDB_SCRB_CLIENT_ACCOUNT] (
[CLIENT_ID] [bigint] NULL ,
[CLIENT_Name] [char] (70) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,
[GOC] [char] (3) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,
[AC_DATE] [datetime] NULL ,
[EFF_Date] [datetime] NULL ,
[AC_STATUS] [char] (8) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
[REPORTING_DATE] [datetime] NULL,
[V2V_IND] [char] (1) NULL, --2017.10
[LAST_OPER_ID] [char] (8) NULL,
[LAST_TIMESTAMP] [datetime] NULL
) ON [PRIMARY]
GO
NO!
Msg 2714, Level 16, State 6, Line 2
There is already an object named 'CDB_SCRB_CLIENT_ACCOUNT' in the database.
.
I know that if I change the check as follows
if exists (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[CDB_SCRB_CLIENT_ACCOUNT]') AND type in (N'U'))
drop table [CLASS].DBO.[CDB_SCRB_CLIENT_ACCOUNT]
that it will work, but I do not want to change many occurences of this code just to run the app on my local database.
SELECT ##version --on dev server
Microsoft SQL Server 2012 (SP3-CU7) (KB3205051) - 11.0.6579.0 (X64)
Dec 22 2016 11:18:09
Copyright (c) Microsoft Corporation
Enterprise Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1) (Hypervisor)
SELECT ##version --on local workstation
Microsoft SQL Server 2012 (SP1) - 11.0.3156.0 (X64)
May 4 2015 18:48:09
Copyright (c) Microsoft Corporation
Developer Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)
When I script the tables in each table, they appear to be same same. What could explain the difference?
The problem you are experiencing is that OBJECTPROPERTY is scoped to the current database. Rather than use that you could use the object_type option in your OBJECT_ID call (so OBJECT_ID(N'Database.Schema.ObjectName', 'U') where U represents a user table (you can check the relevant object_type value from the type column in sys.objects).
You can also simplify your query by just checking for the existence of the object_id, so no need to query again sys.objects in that database.
This would make your query look like the following:
IF (SELECT object_ID(N'[CLASS].DBO.[CDB_SCRB_CLIENT_ACCOUNT]', 'U')) IS NOT NULL
BEGIN
PRINT 'EXISTS!'
drop table [CLASS].DBO.[CDB_SCRB_CLIENT_ACCOUNT]
END
ELSE PRINT 'NO!'
GO
CREATE TABLE [CLASS].DBO.[CDB_SCRB_CLIENT_ACCOUNT] (
[CLIENT_ID] [bigint] NULL ,
[CLIENT_Name] [char] (70) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,
[GOC] [char] (3) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,
[AC_DATE] [datetime] NULL ,
[EFF_Date] [datetime] NULL ,
[AC_STATUS] [char] (8) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
[REPORTING_DATE] [datetime] NULL,
[V2V_IND] [char] (1) NULL, --2017.10
[LAST_OPER_ID] [char] (8) NULL,
[LAST_TIMESTAMP] [datetime] NULL
) ON [PRIMARY]
GO
I have a problem, I use SQL Server 2014. I need to copy data from a local table to the identical table on a remote server. Right now, I can only insert static data to the remote table, select data from the remote table, but I didn't get to do what I want.
Here's my sql code
IF OBJECT_ID('tempdb..##TempTable') IS NOT NULL
DROP TABLE ##TempTable
CREATE TABLE ##TempTable
(
[Name] [nvarchar](255) NOT NULL,
[Description] [nvarchar](512) NOT NULL,
[ARXUrl] [nvarchar](1000) NOT NULL,
[IsDeleted] [bit] NULL,
[CreatedDate] [datetime] NOT NULL
);
GO
INSERT INTO ##TempTable
SELECT
[Name], [Description], [ARXUrl], [IsDeleted], [CreatedDate]
FROM [dbo].[ARXSystem]
GO
-- destination database
:SETVAR remoteDB [slic-test]
:CONNECT someserver.net\SQLEXPRESS2014 -U user -P password
--source database
USE [SLIC]
GO
SELECT *
FROM ##TempTable;
INSERT INTO $(remoteDB).[dbo].[ARXSystem]
SELECT *
FROM #TempTable
GO
and here is the message I received
(1 row(s) affected)
Connecting to someserver.net\SQLEXPRESS2014 as SLIC...
Msg 208, Level 16, State 0, Line 22
Invalid object name '##TempTable'.
Disconnecting connection from someserver.net\SQLEXPRESS2014 as SLIC...
You need to use remote server via Openquery or Openrowset, which might require a server configuration change on the source system where the query is executed. To push the data from the source to target the query would look something like this:
INSERT INTO OPENROWSET('SQLNCLI',
'Server=someserver.net\SQLEXPRESS2014;User=xxxx;Password=xxxx;',
'SELECT [Name], [Description], [ARXUrl], [IsDeleted], [CreatedDate]
FROM [slic-test].[dbo].[ARXSystem]')
SELECT [Name], [Description], [ARXUrl], [IsDeleted], [CreatedDate]
FROM [dbo].[ARXSystem]
OPENROWSET documentation can be found here: https://msdn.microsoft.com/en-us/library/ms190312.aspx
Your SQL Creates a table, but its not a temporary table. It will create it in the active database. You need to use something like
DECLARE #temptable TABLE
(
[Name] [nvarchar](255) NOT NULL,
[Description] [nvarchar](512) NOT NULL,
[ARXUrl] [nvarchar](1000) NOT NULL,
[IsDeleted] [bit] NULL,
[CreatedDate] [datetime] NOT NULL
)
Then after your switch databases you can refer to that table using
INSERT INTO $(remoteDB).[dbo].[ARXSystem]
SELECT * FROM #temptable
GO
Hopefully this will help
I want to transfer data between databases automatically. For example in every one hour, data in one table in first db will be copied into a table in second db.
What do you recommend for me to do ?
By using jobs maybe?
Thanks in advance.
You can probably look at SQL Server Database Mirroring if you are already on SQL Server 2008. Avoids the need for an additional ETL tool.
Create a store procedure and schedule it for one hours:-
Create procedure copydata
AS
INSERT INTO DB1.dbo.TempTable
SELECT * FROM DB2.dbo.TempTable
Have you considered using Integration Services?
You can set up a package doing what you want it to do and then add it to a job with a scheduler.
Here is the "poor man's method".
This logic will you eliminate the need to truncate the destination table every time.
You can schedule a job to call the stored procedure every hour.
OrganizationDB, that is my db name. Yours may be different.
If your source data is on a different database, use that db's name ....
USE [OrganizationDB]
GO
DROP TABLE [dbo].[EmployeeSource]
GO
DROP TABLE [dbo].[EmployeeDestination]
GO
CREATE TABLE [dbo].[EmployeeSource](
[EmployeeUUID] [uniqueidentifier] NOT NULL,
[SSN] [char](9) NOT NULL,
[LastName] [varchar](40) NOT NULL,
[FirstName] [varchar](40) NOT NULL,
[HireDate] [smalldatetime] NOT NULL,
CONSTRAINT [PK_EmployeeSource] PRIMARY KEY NONCLUSTERED
(
[EmployeeUUID] ASC
),
CONSTRAINT [CK_EmployeeSource_SSN_Unique] UNIQUE NONCLUSTERED
(
[SSN] ASC
)
)
GO
ALTER TABLE [dbo].[EmployeeSource] ADD DEFAULT (newsequentialid()) FOR [EmployeeUUID]
GO
ALTER TABLE [dbo].[EmployeeSource] ADD DEFAULT (getdate()) FOR [HireDate]
GO
CREATE TABLE [dbo].[EmployeeDestination](
[EmployeeUUID] [uniqueidentifier] NOT NULL,
[SSN] [char](9) NOT NULL,
[LastName] [varchar](40) NOT NULL,
[FirstName] [varchar](40) NOT NULL,
[HireDate] [smalldatetime] NOT NULL,
CONSTRAINT [PK_EmployeeDestination] PRIMARY KEY NONCLUSTERED
(
[EmployeeUUID] ASC
),
CONSTRAINT [CK_EmployeeDestination_SSN_Unique] UNIQUE NONCLUSTERED
(
[SSN] ASC
)
)
GO
ALTER TABLE [dbo].[EmployeeDestination] ADD DEFAULT (newsequentialid()) FOR [EmployeeUUID]
GO
ALTER TABLE [dbo].[EmployeeDestination] ADD DEFAULT (getdate()) FOR [HireDate]
GO
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[uspPoorMansDataCopy]') AND type in (N'P', N'PC'))
DROP PROCEDURE [dbo].[uspPoorMansDataCopy]
Go
/*
exec [dbo].[uspPoorMansDataCopy]
*/
CREATE PROCEDURE [dbo].[uspPoorMansDataCopy]
AS
SET NOCOUNT ON
/* USE SURROGATE KEY */
INSERT INTO dbo.EmployeeDestination
(
[EmployeeUUID]
, [SSN]
, [LastName]
, [FirstName]
, [HireDate]
)
Select
[EmployeeUUID]
, [SSN]
, [LastName]
, [FirstName]
, [HireDate]
From
OrganizationDB.dbo.EmployeeSource es
where
not exists (select null from dbo.EmployeeDestination innerDestination where innerDestination.EmployeeUUID = es.EmployeeUUID)
/* OR USE UNIQUE CONSTRAINT */
INSERT INTO dbo.EmployeeDestination
(
[EmployeeUUID]
, [SSN]
, [LastName]
, [FirstName]
, [HireDate]
)
Select
[EmployeeUUID]
, [SSN]
, [LastName]
, [FirstName]
, [HireDate]
From
OrganizationDB.dbo.EmployeeSource es
where
not exists (select null from dbo.EmployeeDestination innerDestination where UPPER(innerDestination.SSN) = upper(es.SSN))
SET NOCOUNT OFF
GO
GRANT EXECUTE ON [dbo].[uspPoorMansDataCopy] TO public
GO
If this is only data from one table and if both databases are on the same server you can also consider creating INSERT trigger on that table that will automatically catch all new data and copy it into secondary database.
I wouldn’t do this though if databases are not on the same server and if servers are not in the same physical location because it might cause performance issues.
I have been sniffing out forums and blogs lately, but need some help with a long running query. It's part of a system of stored procedures. This specified statement used to run in about 5 minutes, but lately has been running up to 72 hours!
Here's the setup:
SQL Server 2005 with 28GB memory. Two mountpoints to a SAN with shared disks consisting of 10 spindles. Data is on one mountpoint, Log on another, tempdb on data space. Just one userdatabase on this server.
Here's two tables, condit and condmod. Condit contains 800K records, condmod is initially empty. I issue a truncate mcmain.condmod before the process starts, for testing purposes.
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condit__con_notm__000AF8CF]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condit] DROP CONSTRAINT [DF__condit__con_notm__000AF8CF]
END
GO
/****** Object: Table [mcmain].[condit] Script Date: 02/07/2012 11:57:47 ******/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condit]') AND type in (N'U'))
DROP TABLE [mcmain].[condit]
GO
/****** Object: Table [mcmain].[condit] Script Date: 02/07/2012 11:57:49 ******/
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condit]') AND type in (N'U'))
BEGIN
CREATE TABLE [mcmain].[condit](
[con_levgln] [char](13) NULL,
[con_stat] [char](4) NULL,
[con_dscgrp] [char](35) NULL,
[con_levart] [char](20) NULL,
[con_desc] [char](50) NULL,
[con_disc1] [numeric](5, 0) NULL,
[con_disc2] [numeric](5, 0) NULL,
[con_disc3] [numeric](5, 0) NULL,
[con_ntprce] [numeric](9, 0) NULL,
[con_dtstrt] [datetime] NULL,
[con_dtend] [datetime] NULL,
[con_volc] [char](8) NULL,
[con_updnmr] [char](20) NULL,
[con_notmod] [bit] NULL,
[con_ascver] [char](5) NULL,
[con_prddat] [datetime] NULL,
[con_cusgln] [char](13) NULL,
[con_cusdeb] [char](40) NULL,
[con_rowid] [int] IDENTITY(1,1) NOT NULL
) ON [PRIMARY]
END
GO
IF NOT EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condit__con_notm__000AF8CF]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condit] ADD DEFAULT ((0)) FOR [con_notmod]
END
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condmod__com_not__7E22B05D]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condmod] DROP CONSTRAINT [DF__condmod__com_not__7E22B05D]
END
GO
/****** Object: Table [mcmain].[condmod] Script Date: 02/07/2012 11:57:56 ******/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND type in (N'U'))
DROP TABLE [mcmain].[condmod]
GO
/****** Object: Table [mcmain].[condmod] Script Date: 02/07/2012 11:57:58 ******/
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND type in (N'U'))
BEGIN
CREATE TABLE [mcmain].[condmod](
[com_levgln] [char](13) NULL,
[com_stat] [char](4) NULL,
[com_dscgrp] [char](35) NULL,
[com_levart] [char](20) NULL,
[com_desc] [char](50) NULL,
[com_disc1] [numeric](5, 0) NULL,
[com_disc2] [numeric](5, 0) NULL,
[com_disc3] [numeric](5, 0) NULL,
[com_ntprce] [numeric](9, 0) NULL,
[com_dtstrt] [datetime] NULL,
[com_dtend] [datetime] NULL,
[com_volc] [char](8) NULL,
[com_updnmr] [char](20) NULL,
[com_notmod] [bit] NULL,
[com_ascver] [char](8) NULL,
[com_prddat] [datetime] NULL,
[com_cusgln] [char](13) NULL,
[com_cusdeb] [char](40) NULL,
[com_rowid] [int] IDENTITY(1,1) NOT NULL
) ON [PRIMARY]
END
GO
IF NOT EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condmod__com_not__7E22B05D]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condmod] ADD DEFAULT ((0)) FOR [com_notmod]
END
GO
Here's the isolated code that runs a long time:
DECLARE #TempIdTable TABLE ([com_rowid] Int PRIMARY KEY)
INSERT #TempIdTable([com_rowid])
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
LEFT OUTER JOIN [mcmain].[condit] AS cdt
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
--select * from #TempIdTable
INSERT INTO mcmain.condit(con_levgln
,con_stat
,con_dscgrp
,con_levart
,con_desc
,con_disc1
,con_disc2
,con_disc3
,con_ntprce
,con_dtstrt
,con_dtend
,con_volc
,con_notmod
,con_updnmr
,con_ascver
,con_cusgln
,con_cusdeb)
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM [mcmain].[condmod] AS cmd
INNER JOIN #TempIdTable AS tit
ON tit.com_rowid = cmd.com_rowid
The insert into the #TempIdTable takes forever. What can I do to speed up this process?
TIA
Cees Cappelle
p.s. I have clustered indexes on both tables, like:
/****** Object: Index [condmodTest] Script Date: 02/07/2012 13:24:34 ******/
IF NOT EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND name = N'condmodTest')
CREATE CLUSTERED INDEX [condmodTest] ON [mcmain].[condmod]
(
[com_levgln] ASC,
[com_dscgrp] ASC,
[com_levart] ASC,
[com_volc] ASC,
[com_cusgln] ASC,
[com_cusdeb] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
If I do a straight select (just now), it takes 2 seconds.
Here's the code:
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM mcmain.condmod
LEFT OUTER JOIN mcmain.condit
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
I just did a Actial Execution plan with the following code. It took 3'16'' ???
truncate table mcmain.condit
CREATE TABLE #TempIdTable ([com_rowid] Int PRIMARY KEY)
-- DECLARE #TempIdTable TABLE
-- ([com_rowid] Int PRIMARY KEY)
INSERT #TempIdTable
([com_rowid])
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
LEFT OUTER JOIN [mcmain].[condit] AS cdt
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
-- AND com_updnmr = #plannummer
INSERT INTO mcmain.condit
(con_levgln
,con_stat
,con_dscgrp
,con_levart
,con_desc
,con_disc1
,con_disc2
,con_disc3
,con_ntprce
,con_dtstrt
,con_dtend
,con_volc
,con_notmod
,con_updnmr
,con_ascver
,con_cusgln
,con_cusdeb)
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM [mcmain].[condmod] AS cmd
INNER JOIN #TempIdTable AS tit
ON tit.com_rowid = cmd.com_rowid
Same statement but using a tablevariable took 1'39''
When I run the sp, the same statement takes hours and hours. Still don't get it.
If the SELECT itself takes a long time
You could consider using NOT EXISTS rather than OUTER JOIN ... NULL as this is often more efficient.
Also I would get rid of the non sargable ISNULL comparisons.
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
WHERE NOT EXISTS (SELECT *
FROM [mcmain].[condit] AS cdt
WHERE con_levgln = com_levgln
AND EXISTS (SELECT con_dscgrp,
con_levart,
con_volc,
con_cusgln,
con_cusdeb
INTERSECT
SELECT com_dscgrp,
com_levart,
com_volc,
com_cusgln,
com_cusdeb))
If the SELECT runs quite quickly on its own but just not when inserting to the table variable then check whether in the quick case you get a parallel plan.
Queries that insert to table variables do not get parallelised so if that is the issue you could consider changing to a #temp table.
If neither of these suggestions help then I suggest you start monitoring the wait types when this procedure runs. See the paper "SQL Server 2005 Waits and Queues"
Note: the use of #tables uses tempdb.
You should try increasing the number of temp db files... you can start with the number of tempdb files equal to the number of procs you have... but as a good starting point just jump to 10... The problem you could be having is contention for tempdb files. You should look for page latching in tempdb.
This article discusses how the generalization I wrote above is a misconception (but not necessarily wrong) but in doing so does a good job of explaining the issue. It also points you to other articles which give informatoin on how to look for page latching.
A SQL Server DBA myth a day: (12/30) tempdb should always have one data file per processor core