I have a large SaaS system database, running for 8+ years, no problems. It is an Azure SQL database, and we host the corresponding web application through Azure too.
Suddenly, in the early hours of this morning, some of the C# web app reports start failing due to duplicate table records being detected. I check the table in question and yes, there are duplicate identical records, with clashing unique keys, in the table.
I've never seen this before. How can a unique key fail to enforce itself during inserts/updates?
EDIT:
Here's the schema:
CREATE TABLE [tenant_clientnamehere].[tbl_cachedstock](
[clusteringkey] [bigint] IDENTITY(1,1) NOT NULL,
[islivecache] [bit] NOT NULL,
[id] [uniqueidentifier] NOT NULL,
[stocklocation_id] [uniqueidentifier] NOT NULL,
[stocklocation_referencecode] [nvarchar](50) NOT NULL,
[stocklocation_description] [nvarchar](max) NOT NULL,
[productreferencecode] [nvarchar](50) NOT NULL,
[productdescription] [nvarchar](max) NOT NULL,
[unitofmeasurename] [nvarchar](50) NOT NULL,
[targetstocklevel] [decimal](12, 3) NULL,
[minimumreplenishmentquantity] [decimal](12, 3) NULL,
[minimumstocklevel] [decimal](12, 3) NULL,
[packsize] [int] NOT NULL,
[isbuffermanageddynamically] [bit] NOT NULL,
[dbmcheckperioddays] [int] NULL,
[dbmcheckperiodbuffergroup_id] [uniqueidentifier] NULL,
[ignoredbmuntildate] [datetime2](7) NULL,
[notes1] [nvarchar](100) NOT NULL,
[notes2] [nvarchar](100) NOT NULL,
[notes3] [nvarchar](100) NOT NULL,
[notes4] [nvarchar](100) NOT NULL,
[notes5] [nvarchar](100) NOT NULL,
[notes6] [nvarchar](100) NOT NULL,
[notes7] [nvarchar](100) NOT NULL,
[notes8] [nvarchar](100) NOT NULL,
[notes9] [nvarchar](100) NOT NULL,
[notes10] [nvarchar](100) NOT NULL,
[seasonaleventreferencecode] [nvarchar](50) NULL,
[seasonaleventtargetstocklevel] [decimal](12, 3) NULL,
[isarchived] [bit] NOT NULL,
[isobsolete] [bit] NOT NULL,
[currentstocklevel] [decimal](12, 3) NULL,
[quantityenroute] [decimal](12, 3) NULL,
[recommendedreplenishmentquantity] [decimal](12, 3) NULL,
[bufferpenetrationpercentage] [int] NOT NULL,
[bufferzone] [nvarchar](10) NOT NULL,
[bufferpenetrationpercentagereplenishment] [int] NOT NULL,
[bufferzonereplenishment] [nvarchar](10) NOT NULL,
CONSTRAINT [PK_tbl_cachedstock] PRIMARY KEY CLUSTERED
(
[clusteringkey] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY],
CONSTRAINT [UK_tbl_cachedstock_1] UNIQUE NONCLUSTERED
(
[islivecache] ASC,
[id] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
ALTER TABLE [tenant_clientnamehere].[tbl_cachedstock] ADD CONSTRAINT [DF__tbl_cache__isarc__1A200257] DEFAULT ((0)) FOR [isarchived]
GO
ALTER TABLE [tenant_clientnamehere].[tbl_cachedstock] ADD CONSTRAINT [DF__tbl_cache__isobs__1B142690] DEFAULT ((0)) FOR [isobsolete]
GO
And the clash (two of which are still in the table) is:
islivecache = 1
id = BA7AD2FD-EFAA-485C-A200-095626C583A3
The cause of this turns out to be very simple - but troubling: every single Unique Key, in every single table, in every single schema, was simultaneously set to "Is Disabled", so whilst they exist they're not being applied.
I've manually cleared out records dupes and rebuilt all indexes to re-enforce the checks, and everything is fine again, but I have no idea how technically this can suddenly occur.
I'm now currently working with Azure support to get to the bottom of it.
Related
I have a table where I am storing different documents of different source as follows
CREATE TABLE [dbo].[Document](
[DocumentId] [int] IDENTITY(1,1) NOT NULL,
[EntityId] [int] NOT NULL,
[DocumentGuid] [uniqueidentifier] NOT NULL,
[DocumentTypeCdId] [int] NOT NULL,
[DocumentName] [nvarchar](500) NOT NULL,
[DocumentType] [nvarchar](500) NOT NULL,
[DocumentData] [nvarchar](max) NOT NULL,
[IsSuppressed] [bit] NULL,
[CreatedBy] [nvarchar](200) NULL,
[CreatedDt] [datetime] NULL,
[UpdatedBy] [nvarchar](200) NULL,
[UpdatedDt] [datetime] NULL,
CONSTRAINT [PK_Document] PRIMARY KEY CLUSTERED
(
[DocumentId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
ALTER TABLE [dbo].[Document] WITH CHECK ADD CONSTRAINT [FK_Document_DocumentTypeCd] FOREIGN KEY([DocumentTypeCdId])
REFERENCES [dbo].[DocumentTypeCd] ([DocumentTypeCdId])
GO
ALTER TABLE [dbo].[Document] CHECK CONSTRAINT [FK_Document_DocumentTypeCd]
GO
EntityId will be from different source tables, so can I add this column to be a FK of all those source table. Currently I have nearly 10 Source tables. If not what is the better approach to handle this scenario
You have a problem in the design of your database. In such a case you need to have a parent ancestor table that hold the keys of all type of documents, then multiple children table, each one speciallized for a speciic document type.
This is called inheritance and children must not share the same key value (children table with excusion ids...)
I have a design/performance question.
I have this next table.
CREATE TABLE [dbo].[DW_Visits_2016](
[VisitId] [int] NOT NULL,
[UserId] [int] NOT NULL,
[VisitReferrer] [varchar](512) NULL,
[VisitFirstRequest] [varchar](255) NOT NULL,
[VisitAppName] [varchar](255) NULL,
[VisitCountry] [varchar](50) NULL,
[VisitDate] [smalldatetime] NOT NULL,
[VisitMins] [int] NOT NULL,
[VisitHits] [int] NOT NULL,
[EntryTag] [varchar](100) NOT NULL,
[VisitCount] [int] NOT NULL,
[VisitInitialDate] [datetime] NOT NULL,
[AggregateType] [varchar](50) NULL,
[MemberId] [int] NULL,
[ServerName] [varchar](50) NULL,
[BrowserUserAgent] [varchar](255) NULL,
[LastModifiedDate] [smalldatetime] NULL,
[Guid] [uniqueidentifier] NULL,
[SessionId] [varchar](100) NULL,
[IPAddress] [varchar](40) NULL,
CONSTRAINT [PK_Visits] PRIMARY KEY NONCLUSTERED
(
[VisitId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
)
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[Visits] WITH CHECK ADD CONSTRAINT [CK_Visits_VisitDate] CHECK (([VisitDate]>='2016-01-01' AND [VisitDate]<'2017-01-01'))
GO
ALTER TABLE [dbo].[Visits] CHECK CONSTRAINT [CK_Visits_VisitDate]
And this same table for 2015 ... 2010.
Every table has around 150 million rows. So, combined we are talking about 1,050 million rows.
I received a requirement where BI people wants to have this combined on a single view (Something crazy like select * from all_visits).
Luckily they gave me some ‘where’ clauses, and some columns they don’t need, so the final result would be 6 columns and 20% of the rows (210 million rows), but nonetheless, a ‘view’ is just a stored query. Even though the box has 60GB of ram, it’s shared with many other databases.
Options I see:
Instead of a view… Creating the views as tables and move them to a dedicated box.
Create one view per year?
Switch all of this to mongodb or something like vertica?!
Any of the previous options combined with column stored indexes?
All examples of table partitioning that I have found are quite simple but I need to partition many tables by one criteria.
For example I have tables: Contractors and Products where ContractorId in Products table is a foreign key.
I created function and schema for ContractorId. It works perfectly for Contractors table but when it comes to the Products table...
I have no idea how should I use it because when I try I always got the information: "The filegroup 'PRIMARY' specified for the clustered index 'PK_dbo.Products' was used for table 'dbo.Products' even though partition scheme 'scheme_Contractors' is specified for it". My Products table looks like:
CREATE TABLE [dbo].[Products](
[ProductId] [int] IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](max) NULL,
[Amount] [int] NULL,
[Color] [nvarchar](max) NULL,
[Price] [decimal](18, 2) NULL,
[Guarantee] [nvarchar](max) NULL,
[GuaranteeType] [int] NULL,
[AdditionalFeatures] [nvarchar](max) NULL,
[Valid] [bit] NULL,
[ContractorId] [int] NOT NULL,
[ProducerId] [int] NOT NULL,
[ProductCategoryId] [int] NOT NULL,
CONSTRAINT [PK_dbo.Products] PRIMARY KEY ( [ProductId] ASC ) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] )
GO
ALTER TABLE [dbo].[Products] WITH CHECK ADD CONSTRAINT [FK_dbo.Products_dbo.Contractors_ContractorId] FOREIGN KEY([ContractorId])
REFERENCES [dbo].[Contractors] ([ContractorId])
GO
ALTER TABLE [dbo].[Products] CHECK CONSTRAINT [FK_dbo.Products_dbo.Contractors_ContractorId]
GO
Could anymone tell me please - is it possible to use my schema on ContractorId column and how? Thank you in advance!
In agreement with Dan Guzman, I'd like to point out there should be no [PRIMARY] specification in the table definition.
We use partitioning on large scale. It is very comfortable to partition all tables on the same partitioning scheme, because the SQL engine will use its multi-processor paralellisation capabilities to the full.
When a certain group of partitions is in one database file and another paration in another file you can even become flexible with disc-usage and backups.
So you first need a partition function to define the values of the partitioning scheme:
CREATE PARTITION FUNCTION [ContractorPartitionFunction](int) AS RANGE LEFT
FOR VALUES (contractor1,contractor2,...)
Then you need to create the partition scheme
CREATE PARTITION SCHEME [ContractorPartitionScheme]
AS PARTITION [ContractorPartitionFunction]
TO ([File_001],[File_002],...,[PRIMARY])
Then for all tables and indexes you now create you should remove ON [PRIMARY] from the definitions as the target filegroup, but instead you should use
ON [ContractorPartitionScheme](ContractorId)
So you table definition should now read:
CREATE TABLE [dbo].[Products](
[ProductId] [int] IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](max) NULL,
[Amount] [int] NULL,
[Color] [nvarchar](max) NULL,
[Price] [decimal](18, 2) NULL,
[Guarantee] [nvarchar](max) NULL,
[GuaranteeType] [int] NULL,
[AdditionalFeatures] [nvarchar](max) NULL,
[Valid] [bit] NULL,
[ContractorId] [int] NOT NULL,
[ProducerId] [int] NOT NULL,
[ProductCategoryId] [int] NOT NULL)
ON ContractorPartitionScheme(ContractorId)
CREATE UNIQUE NONCLUSTERED INDEX PK_dbo.Products ON Products
(
productId,
ConstructorId
) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON ContractorPartitionScheme(ContractorId)
I have a table called UserCredentials and it has a uniqueidentifier column [UserCredentialId]. When I try to create a new user, I get 00000000-0000-0000-0000-000000000000 in my first try, then when I try adding another user, it says PK cannot be duplicated. At first I had a hard time guessing what does it mean but, I think its because of my uniqueidentifier is not generating random id.
What to do?
EDIT
Here is my SQL table structure:
CREATE TABLE [dbo].[UserCredential](
[UserCredentialId] [uniqueidentifier] NOT NULL,
[UserRoleId] [int] NOT NULL,
[Username] [varchar](25) NOT NULL,
[Password] [varchar](50) NOT NULL,
[PasswordSalt] [varchar](max) NOT NULL,
[FirstName] [varchar](50) NOT NULL,
[LastName] [varchar](50) NOT NULL,
[PayorCode] [varchar](20) NOT NULL,
[ProviderCode] [varchar](50) NULL,
[CorporationCode] [varchar](50) NULL,
[Department] [varchar](50) NULL,
[Status] [varchar](1) NOT NULL,
[DateCreated] [datetime] NOT NULL,
[DateActivated] [datetime] NULL,
[Deactivated] [datetime] NULL,
[DateUpdated] [datetime] NULL,
[CreatedBy] [varchar](50) NOT NULL,
[UpdatedBy] [varchar](50) NOT NULL,
[EmailAddress] [varchar](50) NULL,
[ContactNumber] [int] NULL,
[Picture] [varbinary](max) NULL,
CONSTRAINT [PK_UserCredential_1] PRIMARY KEY CLUSTERED
(
[UserCredentialId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
I already set it to (newid()) but still not working.
Set the Id of your user instance to Guid.NewGuid();
user.Id = Guid.NewGuid();
Change your table definition to
[UserCredentialId] UNIQUEIDENTIFIER DEFAULT (NEWSEQUENTIALID()) NOT NULL
Check why prefer NEWSEQUENTIALID than newid at http://technet.microsoft.com/en-us/library/ms189786.aspx.
When a GUID column is used as a row identifier, using NEWSEQUENTIALID can be faster than using the NEWID function. This is because the NEWID function causes random activity and uses fewer cached data pages. Using NEWSEQUENTIALID also helps to completely fill the data and index pages.
You can have all values in the table default to a new value during insert, much like an IDENTITY.
Run this to set the default of all inserts to use newid().
ALTER TABLE [dbo].[UserCredential] ADD CONSTRAINT [DF_UserCredential_UserCredentialID] DEFAULT (newid()) FOR [UserCredentialId]
GO
I am using SQL Server 2008 and a primary key of a database table I am using is not an IDENTITY column (not sure why). I need to change that.
I am in SQL Server Management Studio in design view, under column properties and for some reason I can't change the identity specifications to Yes.
Is there something that I am missing.. I am new to SQL Server - any ideas on what I am missing??
Here is the create table
CREATE TABLE [dbo].[AR_Transactions](
[Trans_ID] [bigint] NOT NULL,
[DateTime] [datetime] NOT NULL,
[Cashier_ID] [nvarchar](50) NULL,
[CustNum] [nvarchar](12) NOT NULL,
[Trans_Type] [nvarchar](2) NOT NULL,
[Prev_Cust_Balance] [money] NULL,
[Prev_Inv_Balance] [money] NULL,
[Trans_Amount] [money] NOT NULL,
[Payment_Method] [nvarchar](4) NULL,
[Payment_Info] [nvarchar](20) NULL,
[Description] [nvarchar](38) NULL,
[Invoice_Number] [bigint] NOT NULL,
[Store_ID] [nvarchar](10) NOT NULL,
[Dirty] [bit] NOT NULL,
[Station_ID] [nvarchar](5) NULL,
[Payment_Type] [smallint] NULL,
CONSTRAINT [pkAR_Transactions]
PRIMARY KEY CLUSTERED([Store_ID] ASC, [Trans_ID] ASC)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
ALTER TABLE [dbo].[AR_Transactions]
ADD CONSTRAINT [DF_AR_Transactions_Trans_ID_AR_Transactions]
DEFAULT ((0)) FOR [Trans_ID]
ALTER TABLE [dbo].[AR_Transactions]
ADD CONSTRAINT [DF_AR_Transactions_Invoice_Number_AR_Transactions]
DEFAULT ((0)) FOR [Invoice_Number]
Here is the query that I need to run... its a complete hack to try to auto-increment my inserts myself
BEGIN TRANSACTION
INSERT INTO
[cresql].[dbo].[AR_Transactions](Trans_ID, DateTime , Dirty, Store_ID, Trans_Type,
Cashier_ID, CustNum, Trans_Amount, Prev_Cust_Balance)
SELECT
(SELECT MAX(Trans_ID ) + 1 FROM [cresql].[dbo].[AR_Transactions]),
DATEADD(MINUTE, -30, Getdate()), 1, 1001, 'C', 100199, CustNum,
-Acct_Balance, Acct_Balance
FROM [cresql].[dbo].[Customer]
WHERE Acct_Balance <> 0
UPDATE [cresql].[dbo].[Customer]
SET Acct_Balance = 0
WHERE Acct_Balance <> 0
COMMIT TRANSACTION
To illustrate Martin's point:
And PS: - as Mikael Eriksson rightfully mentions (and documents nicely), this Identity Specification remains grayed out as long as that column you're working on has a default constraint.
You need to expand the "Identity Specification" node to change it via the (Is Identity) property.
This will rebuild the table so you might also need to go into Tools -> Options -> Designers -> Prevent saving changes that require table re-creation.
This can be an extremely time consuming operation on large tables as well as entailing a lot of logging and locking. To perform this operation on a large table see my answer here.
Remove the default constraint of column Trans_ID first. Then you can set Is Identity to Yes in the designer.
This is properties for column Trans_ID in your table AR_Transactions. (Is Identity) is disabled:
Remove the default constraint and (Is Identity) is no longer disabled:
Set to yes and save. Default Value or Binding is disabled instead:
You can't use ALTER TABLE ... ALTER COLUMN to modify a column to have an identity property. You'll need to
drop the primary key constraint and any foreign key constraints referencing the column in question in your table.
add a new column with the identity property. It should have the same type (int, I presume) as the existing column.
update the table to seed the new column with the values of the existing column.
alter the new column to make it non-nullable.
drop the old/existing column.
rename the new column so that its name is the same as that of the old column.
Recreate the primary key and foreign key references you dropped in the 1st step.
Simple! Or something.
CREATE TABLE [dbo].[AR_Transactions](
[Trans_ID] [bigint] IDENTITY(1,1) NOT NULL,
[DateTime] [datetime] NOT NULL,
[Cashier_ID] [nvarchar](50) NULL,
[CustNum] [nvarchar](12) NOT NULL,
[Trans_Type] [nvarchar](2) NOT NULL,
[Prev_Cust_Balance] [money] NULL,
[Prev_Inv_Balance] [money] NULL,
[Trans_Amount] [money] NOT NULL,
[Payment_Method] [nvarchar](4) NULL,
[Payment_Info] [nvarchar](20) NULL,
[Description] [nvarchar](38) NULL,
[Invoice_Number] [bigint] NOT NULL,
[Store_ID] [nvarchar](10) NOT NULL,
[Dirty] [bit] NOT NULL,
[Station_ID] [nvarchar](5) NULL,
[Payment_Type] [smallint] NULL,
CONSTRAINT [pkAR_Transactions]
PRIMARY KEY CLUSTERED([Store_ID] ASC, [Trans_ID] ASC)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
ALTER TABLE [dbo].[AR_Transactions]
ADD CONSTRAINT [DF_AR_Transactions_Trans_ID_AR_Transactions]
DEFAULT ((0)) FOR [Trans_ID]
ALTER TABLE [dbo].[AR_Transactions]
ADD CONSTRAINT [DF_AR_Transactions_Invoice_Number_AR_Transactions]
DEFAULT ((0)) FOR [Invoice_Number]