Compute weighted average by group in SQLite - sql

Question

I have quarterly data on portfolio holdings, let's call the table holdings,
portfolio date security dollar_amount
p1 03/31/2001 security1 50
p1 03/31/2001 security2 100
p2 03/31/2001 security1 25
p2 03/31/2001 security2 50
p1 06/30/2001 security1 50
p1 06/30/2001 security2 100
p1 06/30/2001 security3 50
p2 06/30/2001 security1 25
p2 06/30/2001 security3 50
and data on monthly returns for each security, let's call it returns
security date return
security1 03/31/2001 1
security2 03/31/2001 -1
security3 03/31/2001 2
security1 04/30/2001 3
security2 04/30/2001 -1
security3 04/30/2001 2
security1 05/31/2001 1
security2 05/31/2001 2
security3 05/31/2001 -1
security1 06/30/2001 2
security2 06/30/2001 -1
security3 06/30/2001 3
security1 07/31/2001 2
security2 07/31/2001 -3
security3 07/31/2001 1
security1 08/30/2001 2
security2 08/30/2001 -3
security3 08/30/2001 2
For each portfolio, here p1 and p2, I want to compute monthly weighted average returns for each portfolio: SUM(dollar_amount * return) / SUM(dollar_amount). However, I want to take into account that there are quarterly changes in holdings, that is the weights should adjust every quarter.
Desired output:
portfolio date return
p1 03/31/2001 1/3*1 + 2/3*(-1) = -1/2
p2 03/31/2001 1/3*1 + 2/3*(-1) = -1/2
p1 04/30/2001 1/3*3 + 2/3*(-1) = 1/3
p2 04/30/2001 1/3*3 + 2/3*(-1) = 1/3
p3 05/31/2001 1/3*1 + 2/3*2 = 5/3
p4 05/31/2001 1/3*1 + 2/3*2 = 5/3
-- rebalancing, i.e. adjusting the weights according to holding data --
p1 06/30/2001 1/4*2 + 1/2*(-1) + 1/4*3 = 3/4
p2 06/30/2001 1/3*2 + 2/3*3 = 8/3
p1 07/31/2001 1/4*2 + 1/2*(-3) + 1/4*1 = -3/4
p2 07/31/2001 1/3*2 + 2/3*1 = 4/3
p3 08/30/2001 1/4*2 + 1/2*(-3) + 1/4*2 = -1/2
p4 08/30/2001 1/3*2 + 2/3*2 = 2
My final query will have to work with 53 quarters of holdings data and thus 159 months. The number of unique portfolios and securities are up to 13,000.
My question is whether there is a meaningful way to do it in a single SQLite query. If not, what do you think is the best way to do it?
The problems for me are
joining only the relevant (monthly) returns data for each quarter, e.g. returns from 03/31/2001, 04/30/2001, 05/31/2001 for the portfolio weights from 03/31/2001; otherwise the data would explode.
that the weighted average returns have to be computed per group, where a group is defined by quarter and portfolio.
The only way I can think of is to query the weighted average return per date and portfolio, such that I would have to loop through all of these combinations. I am aware that this is a computationally costly job, but I am looking for the fastest solution here.
Thanks for your help! I am using Python, sqlalchemy, sqlite3.

Answer 1

This is my take at it but Sqlite doesn't have much support for working with dates so it feels a little inefficient. I could not get it to work with your date format so I hade to change to 'yyyy-mm-dd' but maybe that is installation specific
SELECT portfolio, r.date, 1.0 * SUM(dollar_amount * return) / SUM(dollar_amount)
FROM returns r
JOIN holdings h ON h.security = r.security AND
strftime('%Y',h.date) = strftime('%Y',r.date) AND
CAST(strftime('%m',r.date) as int) BETWEEN cast(strftime('%m',h.date) as int) AND CAST(strftime('%m',h.date) as int) + 2
GROUP BY portfolio, r.date
ORDER BY r.date