Efficiently running an SQL query over multiple inputs - sql

Question

Hi I've got a simulation snapshot that is currently stored in an PostgreSQL database as a table the schema for the snapshot table is
simdb=> \d isonew_4.snapshot_102
Table "isonew_4.snapshot_102"
Column | Type | Modifiers
--------+---------+-----------
id | integer |
x | real |
y | real |
z | real |
vx | real |
vy | real |
vz | real |
pot | real |
mass | real |
Indexes:
"snapshot_102_id_idx" btree (id) WITH (fillfactor=100)
I've got a query that calculates the mass enclosed for a single radius fine:
SELECT SUM(mass) AS mass
FROM isonew_4.snapshot_102 AS s
WHERE SQRT(s.x^2 + s.y^2 + s.z^2) < {radius}
However I would like to run this over a number number of different radii.
Since the table has around 100 million rows it's something that I would prefer to do as a SQL query rather than grabbing all of the particles and using something like numpy.histogram in python to do the binning on my machine locally.

Answer 1

Method #1
This query might work, with for example 10,20 and 25 as the successive values for the radius:
WITH r(radius) as (values (10),(20),(25))
SELECT radius, SUM(mass) AS mass
FROM isonew_4.snapshot_102 AS s CROSS JOIN r
WHERE SQRT(s.x^2 + s.y^2 + s.z^2) < radius
GROUP BY radius;
The output has two columns: radius and corresponding sum(mass).
Method #2
If the query is too slow because of the CROSS JOIN with the list (presumably, EXPLAIN or better EXPLAIN ANALYZE would tell for sure), a different approach that certainly guarantees a single scan of the big table is to gather all results in a single row, one column per radius, with a generated query looking like this:
SELECT
sum(case when r < 10 then s.mass else 0 end) as radius10,
sum(case when r < 20 then s.mass else 0 end) as radius20,
sum(case when r < 25 then s.mass else 0 end) as radius25
FROM (select mass,SQRT(x^2 + y^2 + z^2) as r from isonew_4.snapshot_102) AS s
Method #3
If it's not practical, another completely different approach that might be worth trying would be to pre-compute SQRT(x^2 + y^2 + z^2) in a btree functional index in the hope that the SQL engine can use it with the inequality comparison. Whether this happens and if the query would be faster or not depends mainly on the data distribution.
create index radius_idx on isonew_4.snapshot_102(SQRT(x^2 + y^2 + z^2));
Then use the first query, either repeated with single radius each time, or method #1 with the GROUP BY and all values at once. If the values are very selective, the execution might be way faster than even a single large sequential scan.