Find group of N similar numbers in group of N+M numbers - sql

Question

I'm trying to find similar values from an array -not just one, but a group of them, while the sum of their element-wise differences is to be the lowest possible value
EXAMPLE:
0
2
4
6
8
9
11
15
16
19
pick 5 numbers
RESULT:
4
6
8
9
11
or
2
4
6
8
9
Where the sum of the element-wise difference of both groups is 7.
The problem is I need to select such group of 1500 numbers from an array of 2927 numbers and I'm not sure if algorithm which takes groups of 0-1500 (indexes)numbers and sums the differences, then goes i+1 until it reaches the 1427-2927 group is effective (finally I would check the smallest sum and which group it belongs to).
Note, that numbers are sorted (doesn't matter if ASC or DESC) and I'm trying to do that using PostgreSQL.
Thanks in advance.

Answer 1

SQL Fiddle
PostgreSQL 9.3 Schema Setup:
A small dataset of random data:
CREATE TABLE test (
id INT,
population INT
);
INSERT INTO TEST VALUES ( 1, 12 );
INSERT INTO TEST VALUES ( 2, 11 );
INSERT INTO TEST VALUES ( 3, 14 );
INSERT INTO TEST VALUES ( 4, 6 );
INSERT INTO TEST VALUES ( 5, 7 );
INSERT INTO TEST VALUES ( 6, 7 );
INSERT INTO TEST VALUES ( 7, 1 );
INSERT INTO TEST VALUES ( 8, 15 );
INSERT INTO TEST VALUES ( 9, 14 );
INSERT INTO TEST VALUES ( 10, 14 );
INSERT INTO TEST VALUES ( 11, 15 );
INSERT INTO TEST VALUES ( 12, 12 );
INSERT INTO TEST VALUES ( 13, 11 );
INSERT INTO TEST VALUES ( 14, 3 );
INSERT INTO TEST VALUES ( 15, 8 );
INSERT INTO TEST VALUES ( 16, 1 );
INSERT INTO TEST VALUES ( 17, 1 );
INSERT INTO TEST VALUES ( 18, 2 );
INSERT INTO TEST VALUES ( 19, 3 );
INSERT INTO TEST VALUES ( 20, 5 );
Query 1:
WITH ordered_sums AS (
SELECT ID,
POPULATION,
ROW_NUMBER() OVER ( ORDER BY POPULATION ) AS RN,
POPULATION - LAG(POPULATION,4) OVER ( ORDER BY POPULATION ) AS DIFFERENCE
FROM test
), minimum_rn AS (
SELECT DISTINCT FIRST_VALUE( RN ) OVER wnd AS optimal_rn
FROM ordered_sums
WINDOW wnd AS ( ORDER BY DIFFERENCE )
)
SELECT ID,
POPULATION
FROM ordered_sums o
INNER JOIN
minimum_rn m
ON ( o.RN BETWEEN m.OPTIMAL_RN - 4 AND m.OPTIMAL_RN )
Results:
| id | population |
|----|------------|
| 10 | 14 |
| 9 | 14 |
| 3 | 14 |
| 11 | 15 |
| 8 | 15 |
The query above will select 5 rows - to change it to select N rows then change the 4s in the LAG function and in the last line to N-1.

Answer 2

Assume the list is a[1], a[2], ..., a[N+M].
Calculate the minimal value of a[i+M-1]-a[i] for i=1 to N+1.
The value(s) of i for which the minimal value is reached are the first indices of M consecutive numbers in which the sum of the element-wise difference is minimal.
One key observation to understand this algorithm is that the "sum of the element-wise differences" of a sequence of sorted integers is simply the difference between the first and last element. E.g. for 4 6 8 9 11 it is 11-4=7.

Answer 3

This solution should work. Row_number() to get the order. self-Join on +1499, then order by the difference of sizes in the pair.
DECLARE #cities TABLE (
city VARCHAR(512)
,size INT
,rownum INT
)
INSERT INTO #cities
SELECT *
,row_number() OVER (
ORDER BY size
) rownum
FROM
rawdata
SELECT *
,d.size - c.size difference
FROM #cities c
INNER JOIN #cities d ON c.rownum + 1499 = d.rownum
WHERE c.rownum <=2927-1499
ORDER BY d.size - c.size