Select unique members from a set by weighted categories/properties - sql

Question

I have a collection of objects (5000+) with 7 different properties. Two properties are tertiary the rest are binary. Each object have all 7 properties specified. It is possible that in some scenarios binary property may become unary.
Once in awhile I need to select top N random objects from this collection weighted by the frequency of a label in each category against the objects' total.
Currently, I have all data in sql server table as object, propertyMask pairs; however, I can reorganize that any other way necessary.
Examples:
black blue yellow (1,2,4)
circle square triangle (8,16,32)
solid color/meshed color (64)
dashed contour/no contour (128)
etc. (256)
The data is :
object1|9 <- 1001 black circle only (all other properties are 0)
object2|81 <- 101 0001 black square with solid color (all other properties are 0)
object3|148 <- 1001 0100 yellow square with dashed contour
etc.
Say, I end up with 1k objects with 600 black, 300 yellow and 100 blue objects. And I need to select top 10 objects. If I just consider one property, I'll just take any 6 black, 3 yellow and 1 blue objects. But I have 6 other properties to consider and ensure I have right amount of circles, squares and triangles. Etc. At this point I don't even know how to approach this problem.
Any suggestions would be appreciated.
*EDIT:
I repopulated data in the following format
name | att1 | att2 | ...
obj1 | 1 | 8 | ...
obj2 | 2 | 16 | ...
obj3 | 1 | 32 | ...
Is there a way to select TOP N objects weighted by the frequency of each attribute? I have 7 attributes for each objects; no Null values.
Thanks!

Answer 1

It's messy, it does not always fetch the exact number of rows required or an perfect distribution but it comes pretty darn close.
So how does it work:
ValuesPivotted: pivot all the distinct values and give each row a random rownumber
TargetDistribution: for each distinct value determine how many you need
SelectRows: go through each row in ValuesPivotted on a per row basis see if the row is to be skipped because it would otherwise breach an target for an distinct value. Otherwise increment the Sum for each value applicable for that row.
DECLARE #TargetRowNum INT = 100;
WITH ValuesPivotted AS(
SELECT O.id
, RowNum = ROW_NUMBER() OVER (ORDER BY NEWID())
, [0] = CASE WHEN O.atr1 = 0 THEN 1 ELSE 0 END
, [1] = CASE WHEN O.atr1 = 1 THEN 1 ELSE 0 END
, [2] = CASE WHEN O.atr1 = 2 THEN 1 ELSE 0 END
, [4] = CASE WHEN O.atr2 = 4 THEN 1 ELSE 0 END
, [8] = CASE WHEN O.atr2 = 8 THEN 1 ELSE 0 END
, [16] = CASE WHEN O.atr3 = 16 THEN 1 ELSE 0 END
, [32] = CASE WHEN O.atr3 = 32 THEN 1 ELSE 0 END
, [64] = CASE WHEN O.atr4 = 64 THEN 1 ELSE 0 END
, [128] = CASE WHEN O.atr4 = 128 THEN 1 ELSE 0 END
FROM dbo.objects AS O
),
TargetDistribution AS (
SELECT Target0 = ROUND(CAST(SUM([0] ) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
, Target1 = ROUND(CAST(SUM([1] ) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
, Target2 = ROUND(CAST(SUM([2] ) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
, Target4 = ROUND(CAST(SUM([4] ) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
, Target8 = ROUND(CAST(SUM([8] ) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
, Target16 = ROUND(CAST(SUM([16] ) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
, Target32 = ROUND(CAST(SUM([32] ) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
, Target64 = ROUND(CAST(SUM([64] ) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
, Target128 = ROUND(CAST(SUM([128]) AS FLOAT) / COUNT(*) * #TargetRowNum, 0)
FROM ValuesPivotted
),
SelectRows AS(
SELECT VP.id
, RowNum
, KeepRow = 1
, Target0 , Sum0 = [0]
, Target1 , Sum1 = [1]
, Target2 , Sum2 = [2]
, Target4 , Sum4 = [4]
, Target8 , Sum8 = [8]
, Target16 , Sum16 = [16]
, Target32 , Sum32 = [32]
, Target64 , Sum64 = [64]
, Target128 , Sum128 = [128]
FROM ValuesPivotted AS VP
CROSS JOIN TargetDistribution AS TD
WHERE VP.RowNum = 1
UNION ALL
SELECT
VP.id
, VP.RowNum
, KeepRow = ISNULL(SkipRow.Value, 1)
, Target0 , Sum0 = Sum0 + ISNULL(SkipRow.Value, [0] )
, Target1 , Sum1 = Sum1 + ISNULL(SkipRow.Value, [1] )
, Target2 , Sum2 = Sum2 + ISNULL(SkipRow.Value, [2] )
, Target4 , Sum4 = Sum4 + ISNULL(SkipRow.Value, [4] )
, Target8 , Sum8 = Sum8 + ISNULL(SkipRow.Value, [8] )
, Target16 , Sum16 = Sum16 + ISNULL(SkipRow.Value, [16] )
, Target32 , Sum32 = Sum32 + ISNULL(SkipRow.Value, [32] )
, Target64 , Sum64 = Sum64 + ISNULL(SkipRow.Value, [64] )
, Target128 , Sum128 = Sum128 + ISNULL(SkipRow.Value, [128])
FROM SelectRows AS SR
INNER JOIN ValuesPivotted AS VP
ON VP.RowNum = SR.RowNum + 1
CROSS APPLY(
SELECT Value =
CASE WHEN Sum0 + [0] <= Target0
AND Sum1 + [1] <= Target1
AND Sum2 + [2] <= Target2
AND Sum4 + [4] <= Target4
AND Sum8 + [8] <= Target8
AND Sum16 + [16] <= Target16
AND Sum32 + [32] <= Target32
AND Sum64 + [64] <= Target64
AND Sum128 + [128] <= Target128
THEN NULL ELSE 0 END
) AS SkipRow
WHERE Sum0 < Target0
OR Sum1 < Target1
OR Sum2 < Target2
OR Sum4 < Target4
OR Sum8 < Target8
OR Sum16 < Target16
OR Sum32 < Target32
OR Sum64 < Target64
OR Sum128 < Target128
)
SELECT O.*
FROM SelectRows AS SR
INNER JOIN dbo.objects AS O
ON SR.id = O.id
WHERE SR.KeepRow = 1
OPTION(MAXRECURSION 0)
EDIT: The WHERE clause in SelectRows did not do what it was supposed to, stop the recursion when all targets were met, now it does.