How to speed up current query with index - sql

Question

I'm using a v12 server in Azure SQL Database, and I have the following table:
CREATE TABLE [dbo].[AudienceNiches](
[Id] [bigint] IDENTITY(1,1) NOT NULL,
[WebsiteId] [nvarchar](128) NOT NULL,
[VisitorId] [nvarchar](128) NOT NULL,
[VisitDate] [datetime] NOT NULL,
[Interest] [nvarchar](50) NULL,
[Gender] [float] NULL,
[AgeFrom18To24] [float] NULL,
[AgeFrom25To34] [float] NULL,
[AgeFrom45To54] [float] NULL,
[AgeFrom55To64] [float] NULL,
[AgeFrom65Plus] [float] NULL,
[AgeFrom35To44] [float] NULL,
CONSTRAINT [PK_AudienceNiches] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
)
I'm executing this query: (UPDATED QUERY)
`select a.interest, count(interest) from (
select visitorid, interest
from audienceNiches
WHERE WebsiteId = #websiteid
AND VisitDate >= #startdate
AND VisitDate <= #enddate
group by visitorid, interest) as a
group by a.interest`
And I have the following indexs (all ASC):
idx_WebsiteId_VisitDate_VisitorId
idx_WebsiteId_VisitDate
idx_VisitorId
idx_Interest
The problem is that my query return 18K rows aproximaly and takes 5 seconds, the whole table has 8.8M records, and if I expand a little the data the time increases a lot, so, what would be the best index to this query? What I'm missing?

Answer 1

The best index for this query is a composite index on these columns, in this order:
WebsiteId
VisitDate
Interest
VisitorId
This allows the query to be completely answered from the index. SqlServer can range scan on (WebsiteId, VisitDate) and then exclude null Interest and finally count distinct VisitorIds all from the index. The indexes entries will be in the correct order to allow these operations to occur efficiently.

Answer 2

It's difficult for me to write SQL without having the data to test against, but see if this gives the results you're looking for with a better execution time.
SELECT interest, count(distinct visitorid)
FROM audienceNiches
WHERE WebsiteId = #websiteid
AND VisitDate between #startdate and #enddate
AND interest is not null
GROUP BY interest

Answer 3

Indexes can require an almost infinite amount of understanding, but in your case I think you would see good performance gains by indexing the WebsiteId and VisitDate as separate indexes.
It's important though to make sure your indexes are in good shape. You need to maintain them by keeping statistics up to date, and rebuilding your indexes periodically.
Lastly, you should examine the query plan when tuning query performance. SQL Server will tell you if it thinks it would benefit from a column (or columns) being indexed, and also alert you to other performance related issues.
Press Ctrl+L from within Management Studio and see what's going on with the query.

Answer 4

Your query could be written in this way, because in final result set you do not pull column visitorid from table audienceNiches, so no need to write two different level of group by. Check with this query and let me know if still facing performance issue.
select interest, count(interest)
from audienceNiches
WHERE WebsiteId = #websiteid
AND VisitDate >= #startdate
AND VisitDate <= #enddate
group by interest

Answer 5

First off, your updated query can be effectively reduced to this:
select an.Interest, count(an.Interest)
from dbo.AudienceNiches an
where an.WebsiteId = #WebSiteId
and an.VisitDate between #startdate and #enddate
group by an.Interest;
Second, depending on the cardinality of your data, one of the following indices will provide the best possible performance:
create index IX_AudienceNiches_WebSiteId_VisitDate_Interest on dbo.AudienceNiches
(WebSiteId, VisitDate, Interest);
or
create index IX_AudienceNiches_VisitDate_WebSiteId_Interest on dbo.AudienceNiches
(VisitDate, WebSiteId, Interest);
As your data will grow, however, I think that eventually the latter one will become more efficient, on average.
P.S. Your table is severely denormalised in multiple aspects. I only hope you know what you are doing.