Laravel sum query using join and distinct - sql

I have 2 tables to query using join, namely loans and amortizations table.
Loans table has many amortizations, so loans.id is a reference to amortizations.loan_id in amortizations table.
Loan id:3 exist twice in amortizations table, each with a value=10,800.
Loan id:5 also exist twice in amortizations table, each with a value=11,100 and the same goes with Loan id:11 that also exist twice in amortizations table each with a value=5400.
When you add this up, the total is 54,600. But I wanted the ids to be distinct so that the values won't add twice. So the total would be equal to 27,300. My existing query below returns a 54,600 value instead of 27,300 even using distinct() in Laravel. How would I achieve this?
Controller query
$oneToThirtyDue = Amortization::distinct('amortizations.loan_id')
->join('loans', 'loans.id', '=', 'amortizations.loan_id')
->select('loans.loan_type',\DB::raw('SUM(loans.loan_balance) as total_val'))
->where('amortizations.payment_status',0)
->where('amortizations.schedule', '<', date('Y-m-d'))
->where('amortizations.past_due', '<=', 30)
->groupBy('loans.loan_type')
->orderBy('loans.loan_type')
->get();
Please help. Thanks a lot.

$idsDue = Amortization::distinct()
->select('loan_id')
->where('payment_status',0)
->where('schedule', '<', date('Y-m-d'))
->where('past_due', '<=', 30)
->get();
$oneToThirtyDue = Loan::select('loan_type',\DB::raw('SUM(loan_balance) as total_val'))
->whereIn('id',$idsDue)
->groupBy('loan_type')
->orderBy('loan_type')
->get();

I believe this problem is that the max will be calculated for all rows in the group by before the distinct happens. Meanwhile distinct will run on the whole row, after the max so i does not help.
Therefor i believe a sub query can solve most of your troubles. The approach is to cut down the duplicate in the Amortization before joining. This will make the join be a 1 to 1, instead of it sometimes being 2-1 when you have duplicates. Reversing the join for ease and using a group by for avoiding duplicates and getting extra columns with max as they are not in the group by. Got a similar query working on local testing on some test tables, so this should be possible to achieve.
$subQuery = Amortization::select(
'loan_id',
DB::raw('max(amortizations.id) as id'),
DB::raw('max(amortizations.payment_status) as payment_status'),
DB::raw('max(amortizations.schedule) as schedule'),
DB::raw('max(amortizations.past_due) as past_due')
)->groupBy('loan_id');
Loan::join(DB::raw("($subQuery->toSql()) as amortizations"), 'amortizations.loan_id', '=', 'loans.id')
->select('loans.loan_type',\DB::raw('SUM(loans.loan_balance) as total_val'))
->where('amortizations.payment_status',0)
->where('amortizations.schedule', '<', date('Y-m-d'))
->where('amortizations.past_due', '<=', 30)
->orderBy('loans.loan_type');
This has to be done with a DB::raw, but i can not see any better approach as of now, sub queries outside of select statements is a grey area in Laravel.

Related

how to get total sum of two tables and group responses with eloquent Laravel?

I am trying to create a query using eloquent where I need to know the total points summed by id, username and session_id.
but the result I get is not correct.
my code
$query = DB::table('advisors')
->select('advisors.id','advisors.nombre_comercial','session_details.session_id',
DB::raw('SUM(session_details.spent_points + template_sales.price_points) AS suma_total'))
->join('consultancy_requests','advisors.id','=','consultancy_requests.advisor_id')
->whereBetween('consultancy_requests.created_at',[$from,$to])
->join('receipts','consultancy_requests.id','=','receipts.session_id')
->where('receipts.status',NULL)
->whereBetween('receipts.created_at',[$from,$to])
->join('session_details','consultancy_requests.consultancy_id','=','session_details.session_id')
->whereBetween('session_details.created_at',[$from,$to])
->join('template_sales','session_details.session_id','=','template_sales.session_id')
->whereBetween('template_sales.created_at',[$from,$to])
->groupBy('advisors.id','advisors.nombre_comercial','session_details.session_id')
->get();
code responses
session_details table
template_sales table
this is the correct answer i want to get.
There are a couple of mistakes in your query that I noticed. For instance you don't need to SUM (session_details.spent_points + template_sales.price_points) because this is already performing the addition.
Instead of pointing out all these, let's break your problem down into smaller pieces; when query seems complicated it would be a good idea to break it down for better understanding. There seems to be a couple of tables but I will base my answer on the two tables provided, and that should give you a starting point.
Essentially, what you want is,
Get the sum of spent_points per session_id; so you need to group by session_id and sum(spent_points)
$sumSpentPointsQuery = DB::table('session_details')
->select('session_id', DB::raw('SUM(spent_points) as sum_spent_points'))
->groupBy('session_id');
Get the sum of price_points per session_id; so you need to group by session_id and sum(price_points)
$sumPricePointsQuery = DB::table('template_sales')
->select('session_id', DB::raw('SUM(price_points) as sum_price_points'))
->groupBy('session_id');
Now we need to get the addition of sum_spent_points and sum_price_points. This time our tables would be the results we got from the sub queries. So we can work with Laravel's fromSub and joinSub to get the result we want.
DB::query()
->select('ssp.session_id as session_id', DB::raw('sum_spent_points + sum_price_points as suma_total') )
->fromSub($sumSpentPointsQuery, 'ssp')
->joinSub($sumPricePointsQuery, 'spp', function($join){
$join->on('ssp.session_id', '=', 'spp.session_id');
})->get();
This query should produce the sql that represents this:
select ssp.session_id as session_id, (sum_spent_points + sum_price_points) as suma_total
from
(select session_id, sum(spent_points) as sum_spent_points
from session_details group by session_id) ssp
inner join
(select session_id, sum(price_points) as sum_price_points
from template_sales group by session_id) spp
on ssp.session_id = spp.session_id ;
Hope this kicks you in the right direction.

Eloquent sum of derived column in select and also where clause

I have a table sales which has a related receipts table.
A sale can have many receipts. If the sum of the receipts is less than the sales amount then there is a balance outstanding.
I have the SQL that works just how I want. Having headaches trying to shoehorn this into eloquent.
SELECT `sales`.`id`, `sales`.`created_at`,`sales`.`updated_at`,`sales`.`sales_date`,`sales`.`gross`, `sales`.`net_amount`,`sales`.`vat_amount`,`sales`.`vat_rate`,`sales`.`description`,
(SELECT SUM(receipt_gross_amount) as received_gross FROM `receipts` INNER JOIN `sales` WHERE `receipts`.`sales_id` = `sales`.`id`) as received
FROM `sales`
) as unsettled_invoices
WHERE unsettled_invoices.gross > COALESCE(unsettled_invoices.received,0)
I won't bore you with what I have tried already as it's not relevant and just causes php to hang and issue "killed" in the command line...
This seems to do the job ...
$unsettledsalessentries = DB::table('sales')
->leftjoin('receipts', 'sales.id','=', 'receipts.sales_id')
->select('sales.id','sales.gross','sales.description',DB::raw('SUM(COALESCE(receipts.receipt_gross_amount,0)) as total_receipts'))
->groupBy('sales.id','sales.gross','sales.description')
->havingRaw('sales.gross > total_receipts')
->get();

Negative comparison with ActiveRecord

I have two tables:
sales
period_id
customer_id
product_id
value
total_sales
period_id
customer_id
product_id
value
I want to select every combination of period_id and customer_id that exists on sales but don't exists on total_sales. I believe there's a short way to do this. But every approach I thought involves N+1 queries.
How can I do this?
In my opinion this is precisely the case where it's reasonable to diverge from the purist ActiveRecord approach and bring some SQL to the table. This should do the trick perfectly well:
Sale.find_by_sql("
SELECT * FROM sales WHERE NOT EXISTS (
SELECT 1 FROM total_sales
WHERE total_sales.period_id = sales.period_id AND
total_sales.customer_id = sales.customer_id)
")
I suppose it also can be done in a more ActiveRecordish style, but it will most certainly lose clarity of the code.
Of course if you ever drop this to you codebase, you really should wrap it in a method and leave in the model, so that SQL code at least doesn't bleed into the controller.
Hope this helps!
P.S. Oh, here's the SQL fiddle I used to test this out: http://sqlfiddle.com/#!15/eaf1d/1
How about you first get the list that matches, then use that to get the rows that you actually want:
unwanted_ids = Sale.joins('inner join total_sales on sales.period_id = total_sales.period_id and sales.customer_id = total_sales.customer_id').pluck('distinct sales.id')
wanted_rows = unwanted_ids.any? Sale.where('id not in (?)', unwanted_ids) : Sale.all
wanted_rows.select(:period_id, :customer_id) #if you only want these two
This way you only need two queries.
EDIT:
You can also do this:
Sale.joins('left outer join total_sales on sales.period_id = total_sales.period_id and sales.customer_id = total_sales.customer_id').where('total_sales.period_id is null')
Does it in one query

PGError: ERROR: column “recipes.id” must appear in the GROUP BY clause or be used in an aggregate function

I have this SQL query in my DB which is causing a problem with PostgreSQL on heroku, Causing the page not to load with the above error in the heroku logs. I am using postgreSQL 9.1.6 so previous bugs have apparently been fixed
def self.top_countries
joins(:recipes).
select('countries.*, count(*) AS recipes_count').
group('countries.id').
order('recipes_count DESC')
end
I am unsure on how to refactor this so that it will work.Could anyone advise please?
Thank You
def self.top_countries
joins(:recipes).
select('countries.id, count(*) AS recipes_count').
group('countries.id').
order('recipes_count DESC')
This generates the SQL
select countries.id, count(*) AS recipes_count
from countries
join recipes on countries.id = recipes.country_id
group by countries.id
order by recipes_count
You'll notice that you only have 2 columns in the SELECT.
Not being a Heroku expert, I suspect you can get it to work by explicitly listing all column that you need from countries, and grouping by the full column list i.e.
def self.top_countries
joins(:recipes).
select('countries.id, countries.name, countries.other, count(*) AS recipes_count').
group('countries.id, countries.name, countries.other').
order('recipes_count DESC')
There might be a more concise way to join the original answer (top part) with another join to top_countries on countries.id to get the rest of the columns after the group by.

Why do I have to use DISTINCT for this to work?

here's my problem: I have an SQL query that makes 4 calls to a lookup table to return their values from a list of combinations in another table. I finally got this working, and for some reason, when I run the query without DISTINCT, I get a ton of data back, so I'm guessing that I'm either missing something or not doing this correctly. It would be really great if this would not only work, but also return the list alphabetically by the first colour name.
I'm putting my SQL here I hope I've explained this well enough:
SELECT DISTINCT
colour1.ColourID AS colour1_ColourID,
colour1.ColourName AS colour1_ColourName,
colour1.ColourHex AS colour1_ColourHex,
colour1.ManufacturerColourID AS colour1_ManufacturerColourID,
colour2.ColourID AS colour2_ColourID,
colour2.ColourName AS colour2_ColourName,
colour2.ColourHex AS colour2_ColourHex,
colour2.QEColourID2 AS colour2_QEColourID2,
colour3.ColourID AS colour3_ColourID,
colour3.ColourName AS colour3_ColourName,
colour3.ColourHex AS colour3_ColourHex,
colour3.QEColourID3 AS colour3_QEColourID3,
colour4.ColourID AS colour4_ColourID,
colour4.ColourName AS colour4_ColourName,
colour4.ColourHex AS colour4_ColourHex,
colour4.QEColourID4 AS colour4_QEColourID4,
Combinations.ID,
Combinations.ManufacturerColourID AS Combinations_ManufacturerColourID,
Combinations.QEColourID2 AS Combinations_QEColourID2,
Combinations.QEColourID3 AS Combinations_QEColourID3,
Combinations.QEColourID4 AS Combinations_QEColourID4,
Combinations.ColourSupplierID,
ColourSuppliers.ColourSupplier
FROM
ColourSuppliers INNER JOIN
(
colour4 INNER JOIN
(
colour3 INNER JOIN
(
colour2 INNER JOIN
(
colour1 INNER JOIN Combinations ON
colour1.ColourID=Combinations.ManufacturerColourID
) ON colour2.ColourID=Combinations.QEColourID2
) ON colour3.ColourID=Combinations.QEColourID3
) ON colour4.ColourID=Combinations.QEColourID4
) ON ColourSuppliers.ColourSupplierID=Combinations.ColourSupplierID
WHERE Combinations.ColourSupplierID = ?
Thanks
Steph
It looks as though you've probably got multiple records for each set of four colour combinations in the Combinations table - posting the structure of the table might help us to work it out.
Adding the clause order by colour1.ColourName to the end of the query should sort it alphabetically by the first colour name.
My guess (and it is a guess because your SQL query is very wide!) is that you're getting the cartesian product.