SQL Server partition and index - sql

I have a requirement to design a table that is going to have around 80 million records. I created a partition for every month using persisted column (if its wrong suggest me the best way). Please find below scripts that I used to create tables and partition and the query that's going to be used often. Only Insertion and deletion will be done on this table.
-- Create the Partition Function
CREATE PARTITION FUNCTION PF_Invoice_item (int)
AS RANGE LEFT FOR VALUES (1,2,3,4,5,6,7,8,9,10,11,12);
-- Create the Partition Scheme
CREATE PARTITION SCHEME PS_Invoice_item
AS PARTITION PF_Invoice_item ALL TO ([Primary]);
CREATE TABLE [Invoice]
(
[invoice_id] [bigint] NOT NULL,
[Invoice_Number] [varchar](255) NULL,
[Invoice_Date] [date] NULL,
[Invoice_Total] [numeric](18, 2) NULL,
[Outstanding_Balance] [decimal](18, 2) NULL,
CONSTRAINT [PK_Invoice_id] PRIMARY KEY CLUSTERED([invoice_id] ASC)
)
CREATE TABLE [InvoiceItem](
[invoice_item_id] [bigint] NOT NULL,
[invoice_id] [bigint] NOT NULL,
[invoice_Date] [date] NULL,
[make] [varchar](255) NULL,
[serial_number] [varchar](255) NULL,
[asset_id] [varchar](100) NULL,
[application] [varchar](255) NULL,
[customer] [varchar](255) NULL,
[ucid] [varchar](255) NULL,
[dcn] [varchar](255) NULL,
[dcn_name] [varchar](255) NULL,
[device_serial_number] [varchar](255) NULL,
[subscription_name] [varchar](255) NULL,
[product_name] [varchar](255) NULL,
[subscription_start_date] [date] NULL,
[subscription_end_date] [date] NULL,
[duration] [varchar](50) NULL,
[promo_name] [varchar](255) NULL,
[promo_end_date] [date] NULL,
[discount] [decimal](18, 2) NULL,
[tax] [decimal](18, 2) NULL,
[line_item_total] [decimal](18, 2) NULL,
[mth] AS (datepart(month,[invoice_date])) PERSISTED NOT NULL,**
[RELATED_PRODUCT_RATEPLAN_NAME] [varchar](250) NULL,
[SUB_TOTAL] [decimal](18, 2) NULL,
[BILLING_START_DATE] [date] NULL,`enter code here`
[BILLING_END_DATE] [date] NULL,
[SUBSCRIPTION_ID] [varchar](200) NULL,
[DEVICE_TYPE] [varchar](200) NULL,
[BASE_OR_PROMO] [varchar](200) NULL,
CONSTRAINT [PK_InvoiceItem_ID] PRIMARY KEY CLUSTERED ([invoice_item_id]
ASC,[mth] ASC))
ON PS_Invoice_item(mth);
GO
ALTER TABLE [InvoiceItem] WITH CHECK ADD CONSTRAINT [FK_Invoice_ID]
FOREIGN KEY([invoice_id])
REFERENCES [Invoice] ([invoice_id])
GO
I will be using below queries
select subscription_name,duration,start_date,end_date,promotion_name,
promotion_end_date,sub_total,discount,tax,line_item_total from InvoiceItem
lt inner join Invoice on lt.invoice_id=invoice.invoice_id where
invoice.invoice_number='' and lt.customer='' and lt.ucid='' lt.make='' and
lt.SERIAL_NUMBER='' and lt.dcn='' and lt.application=''
select customer,make,application from billing.AssetApplicationTotals
lineItem inner join billing.Invoice invoice on
lineItem.invoice_id=invoice.invoice_id where invoice.invoice_number='';
SELECT [invoice_Date],[make],[serial_number],[application],[customer],
[ucid],[dcn],[dcn_name],[device_serial_number]
,[subscription_name],[product_name],[subscription_start_date],
[subscription_end_date],[duration],[promo_name],[promo_end_date]
FROM [InvoiceItem] where [application]=''
SELECT [invoice_Date],[make],[serial_number],[application],[customer],
[ucid],[dcn],[dcn_name],[device_serial_number]
,[subscription_name],[product_name],[subscription_start_date],
[subscription_end_date],[duration],[promo_name],[promo_end_date]
FROM [InvoiceItem] where [customer]=''
What is the best way to create index? Shall I create separate non clustered index for each filter, or shall I have Composite index and shall I have covering index to avoid key lookup?

Related

Query not using index in exists statement

I have the index IDX_tbl_SpeedRun_StatusTypeID_GameID_CategoryID_LevelID_PlusInclude on table dbo.tbl_SpeedRun below. The exists statement in the query below is taking a while (1m 10s) saying there is a missing index ON [dbo].[tbl_SpeedRun] ([StatusTypeID],[LevelID]).
Why is the exists statement not using the index I created? It already includes the columns [StatusTypeID],[LevelID].
Table:
CREATE TABLE [dbo].[tbl_SpeedRun]
(
[OrderValue] [int] NOT NULL IDENTITY(1,1),
[ID] [varchar] (50) NOT NULL,
[StatusTypeID] [int] NOT NULL,
[GameID] [varchar] (50) NOT NULL,
[CategoryID] [varchar] (50) NOT NULL,
[LevelID] [varchar] (50) NULL,
[SubCategoryVariableValues] [varchar] (1000) NULL,
[PlayerIDs] [varchar] (1000) NULL,
[PlatformID] [varchar] (50) NULL,
[RegionID] [varchar] (50) NULL,
[IsEmulated] [bit] NOT NULL,
[Rank] [int] NULL,
[PrimaryTime] [bigint] NULL,
[RealTime] [bigint] NULL,
[RealTimeWithoutLoads] [bigint] NULL,
[GameTime] [bigint] NULL,
[Comment] [varchar] (MAX) NULL,
[ExaminerUserID] [varchar] (50) NULL,
[RejectReason] [varchar] (MAX) NULL,
[SpeedRunComUrl] [varchar] (2000) NOT NULL,
[SplitsUrl] [varchar] (2000) NULL,
[RunDate] [datetime] NULL,
[DateSubmitted] [datetime] NULL,
[VerifyDate] [datetime] NULL,
[ImportedDate] [datetime] NOT NULL CONSTRAINT [DF_tbl_SpeedRun_ImportedDate] DEFAULT(GETDATE()),
[ModifiedDate] [datetime] NULL
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[tbl_SpeedRun]
ADD CONSTRAINT [PK_tbl_SpeedRun]
PRIMARY KEY NONCLUSTERED ([ID]) WITH (FILLFACTOR=90) ON [PRIMARY]
GO
CREATE CLUSTERED INDEX [IDX_tbl_SpeedRun_OrderValue]
ON [dbo].[tbl_SpeedRun] ([OrderValue]) WITH (FILLFACTOR=90) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [IDX_tbl_SpeedRun_StatusTypeID_GameID_CategoryID_LevelID_PlusInclude]
ON [dbo].[tbl_SpeedRun] ([StatusTypeID], [GameID], [CategoryID],[LevelID])
INCLUDE ([SubCategoryVariableValues], [PlayerIDs], [Rank],[PrimaryTime])
GO
Query:
SELECT
CASE
WHEN EXISTS (SELECT 1 FROM dbo.tbl_SpeedRun rn WITH (NOLOCK)
WHERE rn.LevelID = l.ID AND rn.StatusTypeID = 1)
THEN 1
ELSE 0
END
FROM
dbo.tbl_Level l WITH (NOLOCK)
WHERE
l.GameID = 'pd0wq901'
ORDER BY
l.OrderValue
This is the from clause of your subquery:
WHERE rn.LevelID = l.ID AND rn.StatusTypeID = 1
A helpful index for this predicate would involve the two columns, in any order.
Your existing index does not satisfy that requirement. It has columns:
[StatusTypeID], [GameID], [CategoryID], [LevelID])
INCLUDE ([SubCategoryVariableValues], [PlayerIDs], [Rank], [PrimaryTime])
Both columns are here, but buried within others - so the database cannot take advantage of it to speed up the subquery.
Bottom line: creating a large index that involves a lot of columns does not speed up queries by default. Instead, you can analyze each query individually and define the proper optimization.

Handle database Journal transaction table with sales and purchase and bank transaction

I create a journal transaction table like this:
CREATE TABLE [dbo].[Finance_Journal_header]
(
[ID] [NVARCHAR](60) NOT NULL,
[Header_number] [INT] NOT NULL,
[Header_date] [DATETIME] NOT NULL,
[nots] [NVARCHAR](4000) NULL,
[UserID] [NVARCHAR](60)NOT NULL,
[isdelete] [BIT] NOT NULL,
[begin_Date] [DATETIME] NOT NULL,
[Company_ID] [NVARCHAR](60) NOT NULL,
[Last_update] [DATETIME] NULL
)
CREATE TABLE [dbo].[Finance_Journal_details]
(
[ID] [NVARCHAR](60) NOT NULL,
[Header_ID] [NVARCHAR](60) NOT NULL,
[Account_code] [NVARCHAR](60) NOT NULL,
[sr] [INT] NOT NULL,
[nots] [NVARCHAR](4000) NULL,
[Debit_value] [NUMERIC](14, 3)NOT NULL,
[Credit_Value] [NUMERIC](14, 3) NOT NULL,
[UserID] [NVARCHAR](60) NOT NULL,
[isdelete] [BIT] NULL,
[begin_Date] [DATETIME] NOT NULL,
[Company_ID] [NVARCHAR](60) NOT NULL,
[Cost_center] [NVARCHAR](60)NOT NULL,
[related_doc] [INT] NOT NULL,
[Exchange_Rate] [DECIMAL](18, 2)NOT NULL,
[Currency_name] [NVARCHAR](60) NOT NULL,
[Last_update] [DATETIME] NULL
)
I have some other tables for Modules transaction like sales, purchase, bank and treasury transaction
How I can link other modules transaction with GL Journal items?
I have two possible solutions:
Create a general view do union select to join any transaction like journal items
Fire a trigger to journal items table details in update with any modules change in any table insert or update or delete
What is the best way to do that?

t-sql end of month insert

I have created two database tables named TSALESFACT and DimDate. TSALESFACT table contains the monthly sales target values. And DimTime contains the date values which begins from 01/01/2005.
CREATE TABLE [dbo].[TSalesFact](
[DateID] [smalldatetime] NULL,
[YearID] [int] NOT NULL,
[MonthID] [int] NOT NULL,
[SeasonID] [char](10) NOT NULL,
[DepartmentID] [char](10) NOT NULL,
[SalesAmount] [int] NOT NULL
)
CREATE TABLE [dbo].[dim_Date](
[ID] [int] NOT NULL,
[Date] [datetime] NOT NULL,
[Day] [char](2) NOT NULL,
[DaySuffix] [varchar](4) NOT NULL,
[DayOfWeek] [varchar](9) NOT NULL,
[DOWInMonth] [tinyint] NOT NULL,
[DayOfYear] [int] NOT NULL,
[WeekOfYear] [tinyint] NOT NULL,
[WeekOfMonth] [tinyint] NOT NULL,
[Month] [char](2) NOT NULL,
[MonthName] [varchar](9) NOT NULL,
[Quarter] [tinyint] NOT NULL,
[QuarterName] [varchar](6) NOT NULL,
[Year] [char](4) NOT NULL,
[StandardDate] [varchar](10) NULL,
[HolidayText] [varchar](50) NULL)
I would like to INSERT these sales target values into a new table named DailySalesTargets. But, the main purpose of this operation is writing the target sales to the each of end of month date. All of the date values for the relevant month will be zero. My desired resultset for this operation is like below:
How can I achieve this? Any idea?
I think this is what you are trying to do.
insert into [DailySalesTargets]([DateID],[YearID],[MonthID],
[SeasonID],[DepartmentID],[SalesAmount])
select d.[Date],d.[Year],d.[Month]
,dptSeason.[SeasonID],dptSeason.[DepartmentID]
,case when EOMONTH(d.[Date])=s.[DateID] then s.[SalesAmount] else 0 end as [SalesAmount]
from [dbo].[dim_Date] d
cross join (select distinct [DepartmentID],[SeasonID] from [dbo].[TSalesFact]) dptSeason
left join [dbo].[TSalesFact] s on d.[Date] = s.[DateID]
and s.[DepartmentID]=dptSeason.[DepartmentID]
and s.[SeasonID]=dptSeason.[SeasonID]

how to prefix identity column character in msSQL

How can I alter a table's primary identity column to include a letter before the number? I have a table named vendor_master with a primary key VendorID that I would like to store as v1,v2,v3 and so on
CREATE TABLE [dbo].[vendor_master](
[vendorID] [int] IDENTITY(1,1) NOT NULL,
[Vname] [nvarchar](50) NOT NULL,
[Email] [nvarchar](50) NOT NULL,
[Mobile] [bigint] NULL,
[Landline] [bigint] NULL,
[Address] [nvarchar](max) NOT NULL,
[Pincode] [int] NOT NULL)
If it's important to you to have the field in your table, try the following:
CREATE TABLE [dbo].[vendor_master]
(
[vendorID] [int] IDENTITY(1,1) NOT NULL,
[Vname] [nvarchar](50) NOT NULL,
[Email] [nvarchar](50) NOT NULL,
[Mobile] [bigint] NULL,
[Landline] [bigint] NULL,
[Address] [nvarchar](max) NOT NULL,
[Pincode] [int] NOT NULL,
[VendorKey] AS ('v' +CONVERT([varchar](10),[vendorID])) PERSISTED
)
Of course, that will yield you some weird results when sorting, so you might consider padding the data with zeroes to make it a consistent length:
CREATE TABLE [dbo].[vendor_master]
(
[vendorID] [int] IDENTITY(1,1) NOT NULL,
[Vname] [nvarchar](50) NOT NULL,
[Email] [nvarchar](50) NOT NULL,
[Mobile] [bigint] NULL,
[Landline] [bigint] NULL,
[Address] [nvarchar](max) NOT NULL,
[Pincode] [int] NOT NULL,
[VendorKey] AS ('v' +RIGHT('000000000' + CONVERT([varchar](10),[vendorID]),10)) PERSISTED
)
You can't actually do what you are asking. Anyway, there is no reason to add the same letter to the front of a fields value. Instead add the letter in your SELECT statment:
SELECT CONCAT('v', vendorID) AS VendorIDWithPrefix, Vname, Email, Mobile
FROM vendor_master

Joining multiple columns in one table to a single column in another table

I am looking to create a view that pulls data from two tables "Schedule" and "Reference".
Schedule has 50+ columns (it's almost completely denormalized -- not my design), most of which contain a value that could be joined to a column in the Reference table.
How do I write the SQL statement to correctly join each column in Schedules to the single column in Reference?
The Schedule table is defined as:
CREATE TABLE [dbo].[Schedule](
[ID] [int] NOT NULL,
[SCHEDULEWEEK] [datetime] NOT NULL,
[EMPNO] [numeric](10, 0) NOT NULL,
[EMPLNAME] [varchar](32) NULL,
[EMPFNAME] [varchar](32) NULL,
[EMPSENDATE] [datetime] NULL,
[EMPHIREDATE] [datetime] NULL,
[EMPTYPE] [char](1) NULL,
[EMPSTATUS] [char](1) NULL,
[SNREFUSALS] [tinyint] NULL,
[QUALSTRING] [varchar](128) NULL,
[JOBOVERSHIFTTYPE] [bit] NULL,
[SHORTNOTICE] [bit] NULL,
[SHORTNOTICEWAP] [bit] NULL,
[SHORTNOTICEPHONE] [varchar](32) NULL,
[LEADHAND] [bit] NULL,
[DUALCURRENCY] [bit] NULL,
[MIN100WINDOW] [bit] NULL,
[STATHOLIDAY] [bit] NULL,
[AREAOVERHOURS] [bit] NULL,
[DOUBLEINTERZONES] [bit] NULL,
[MAXDAYSPERWEEK] [tinyint] NULL,
[MAXHOURSPERWEEK] [numeric](10, 2) NULL,
[MAXHOURSPERSHIFT] [numeric](10, 2) NULL,
[MAXDOUBLESPERWEEK] [tinyint] NULL,
[ASSIGNEDDAYS] [tinyint] NULL,
[ASSIGNEDHOURS] [numeric](10, 2) NULL,
[ASSIGNEDDOUBLES] [tinyint] NULL,
[ASSIGNEDLOAHOURS] [numeric](10, 2) NULL,
[SHIFTNO1] [int] NULL,
[TEXT1_1] [varchar](64) NULL,
[TEXT2_1] [varchar](64) NULL,
[DAYFLAG1] [bit] NULL,
[COMMENT1] [text] NULL,
[SHIFTNO2] [int] NULL,
[TEXT1_2] [varchar](64) NULL,
[TEXT2_2] [varchar](64) NULL,
[DAYFLAG2] [bit] NULL,
[COMMENT2] [text] NULL,
[SHIFTNO3] [int] NULL,
[TEXT1_3] [varchar](64) NULL,
[TEXT2_3] [varchar](64) NULL,
[DAYFLAG3] [bit] NULL,
[COMMENT3] [text] NULL,
[SHIFTNO4] [int] NULL,
[TEXT1_4] [varchar](64) NULL,
[TEXT2_4] [varchar](64) NULL,
[DAYFLAG4] [bit] NULL,
[COMMENT4] [text] NULL,
[SHIFTNO5] [int] NULL,
[TEXT1_5] [varchar](64) NULL,
[TEXT2_5] [varchar](64) NULL,
[DAYFLAG5] [bit] NULL,
[COMMENT5] [text] NULL,
[SHIFTNO6] [int] NULL,
[TEXT1_6] [varchar](64) NULL,
[TEXT2_6] [varchar](64) NULL,
[DAYFLAG6] [bit] NULL,
[COMMENT6] [text] NULL
-- Snip
) ON [PRIMARY]
And the Reference table is defined as:
CREATE TABLE [dbo].[Reference](
[ID] [int] NOT NULL,
[CODE] [varchar](21) NOT NULL,
[LOCATIONCODE] [varchar](4) NOT NULL,
[SCHAREACODE] [varchar](16) NOT NULL,
[LOCATIONNAME] [varchar](32) NOT NULL,
[FLTAREACODE] [varchar](16) NOT NULL
) ON [PRIMARY]
I am trying to join each [TEXT1_]/[TEXT2_] column in Schedule to the [SCHAREACODE] column in reference. All the reference table contains is a list of areas where the employee could work.
I think he means to join on the Reference table multiple times:
SELECT *
FROM Schedule AS S
INNER JOIN Reference AS R1
ON R1.ID = S.FirstID
INNER JOIN Reference AS R2
ON R2.ID = S.SecondID
INNER JOIN Reference AS R3
ON R3.ID = S.ThirdID
INNER JOIN Reference AS R4
ON R4.ID = S.ForthID
Your description is a bit lacking, so I'm going to assume that
Schedule has 50+ columns (it's almost completely denormalized -- not my design), most of which contain a value that could be joined to a column in the Reference table.
means that 1 of the 50+ columns in Schedule is a ReferenceId. So, given a table design like:
Schedule ( MaybeReferenceId1, MaybeReferenceId2, MaybeReferenceId3, ... )
Reference ( ReferenceId )
Something like:
SELECT *
FROM Schedule
JOIN Reference ON
Schedule.MaybeReferenceId1 = Reference.ReferenceId
OR Schedule.MaybeReferenceId2 = Reference.ReferenceId
OR Schedule.MaybeReferenceId3 = Reference.ReferenceId
OR Schedule.MaybeReferenceId4 = Reference.ReferenceId
...
would work. You could simplify it by using IN if your RDBMS supports it:
SELECT *
FROM Schedule
JOIN Reference ON
Reference.ReferenceId IN (
Schedule.MaybeReferenceId1,
Schedule.MaybeReferenceId2,
Schedule.MaybeReferenceId3,
Schedule.MaybeReferenceId4,
...
)
From updated question
Perhaps something like this? It will be messy no matter what you do.
SELECT S.ID
S.TEXT1_1,
TEXT1_1_RID = COALESCE((SELECT MAX(R.ID) FROM Reference R WHERE R.SCHAREACODE = S.TEXT1_1), 0),
S.TEXT1_2,
TEXT1_2_RID = COALESCE((SELECT MAX(R.ID) FROM Reference R WHERE R.SCHAREACODE = S.TEXT1_2), 0),
...
FROM Schedule S
Agree with TheSoftwareJedi, but can I just suggest using LEFT JOINs so that failures-to-match don't cause your Schedule row to disappear?
Of course, doing 28 JOINs is going to be a bit cumbersome whatever the details.
I'm not sure I'd call this "denormalized", more "abnormalized" ... :-)
Try a query like this:
select s.*, r.schareacode from schedule s,
where
s.text1_1 = s.schareacode
or s.text2_1 = s.schareacode
or s.textx_x = s.schareacode
..
You should be able to get the same results with traditional joins so I recommend you experiment with that as well.