GAMS: Getting the n'th element of a vector - gams-math

I have a dataset in parameter variable
set t /t1 * t5/;
parameter mydata(t) /t1 10, t2 20, t3 30, t4 40, t5 50/;
How can I, the easiest way, get the n'th element of mydata. I am looking for something like:
display mydata(3)
where I will get the value 30

set t /t1 * t5/;
parameter mydata(t) /t1 10, t2 20, t3 30, t4 40, t5 50/;
scalar x;
x = sum(t$(ord(t)=3), mydata(t));
display x;

Related

Update rows with nested queries

I need to update the rows with SG='All' by perform some recalculations on those rows (for example the sum of the rows with SG!='All' groupped by column G).
I tried doing it in the following manner but I get an error:
WITH UpdateData as
(Select * from Tbl
where SG!='All')
update Tbl
Set Val =(select SUM(ROUND(ud.Val,2)) group by ud.G)
from UpdateData ud
where Tbl.SG='All' and ud.G = Tbl.G
Msg 164, Level 15, State 1, Line 6
Each GROUP BY expression must contain at least one column that is not an outer reference.
And here is how the table looks like:
G SG I Val
B All All 142.215
B T1 123 10.401
B T1 123 60.957
B T2 220 70.857
D All All 96.003
D T3 666 80.158
D T5 700 15.845
After the update, on the first row the value should be:
select ROUND(10.401,2) + ROUND(60.957,2) + ROUND(70.857,2) = 10.4+60.96+70.86 =142.22 instead of 142.215
The real situation is more complex and I want to avoid recalculating the all rows, so I prefer doing an update rather than delete and readd them. Thanks a lot
Works like this. Thanks Mihai!
WITH UpdateData as
(Select * from Tbl
where SG!='All')
update Tbl
Set Val =(select SUM(ROUND(ud.Val,2))
from UpdateData ud
where Tbl.SG='All' and ud.G = Tbl.G
group by ud.G)
where Tbl.SG='All'
hope this will work,
update Tbl
set Tbl.Val = = t.Value
from
(select SUM(ROUND(ud.Val,2)) as Value, ud.G
from UpdateData ud
where ud.G = Tbl.G
group by ud.G) as t
where Tbl.SG='All'

Inner join 2 tables by 2 categories and multiple columns

I have 2 tables: Candidates and Jobs.
In Jobs there are columns Profession and Subprofession.
For each row in Candidates there are 8 columns:
Selected_Profession1, Selected_Subprofession1,
Selected_Profession2, Selected_Subprofession2,
Selected_Profession3, Selected_Subprofession3,
Selected_Profession4, Selected_Subprofession4
I would like to make a query that would select all the jobs whose profession and subprofession are in one of the respective fields in the Candidates table.
So let's say we have the following Jobs table:
(profession subprofession) -----> (100, 200)
(100, 201)
(101, 200)
(101, 201)
and the following Candidates table:
(prof1 subprof1 prof2 subprof2 prof3 subprof3 prof4 subprof4) ---->
(100, 200, 300, 400, 100, 200, 100, 300)
(101, 200, 102, 200, 300, 200, 200, 300)
(100, 200, 300, 400, 101, 201, 100, 300)
(101, 101, 200, 200, 300, 300, 400, 400)
The query will return rows 1, 3 and 4 from the Jobs table (because Candidate 1 has the pair 100, 200 and candidate 2 has the pair 101, 200 and candidate 3 has the pair 101, 201).
Hope this is clear enough...
You can do the join on multiple fields with an or condition:
select j.*
from jobs j join
candidates c
on (j.prof = c.prof1 and j.subprof = c.subprof1) or
(j.prof = c.prof2 and j.subprof = c.subprof2) or
(j.prof = c.prof3 and j.subprof = c.subprof3) or
(j.prof = c.prof4 and j.subprof = c.subprof4);
If you have large tables, the performance on this will not be very good. You can fix the data structure to get better performance, by having a CandidateProf table, where each prof/subprof pair is on a different row.
With the data structure you have, you would get better performance with separate joins for each prof/subprof grouping, particularly by having an index on the pair. The problem is the select clause. So:
select distinct j.*
from jobs j lef outer join
candidates c1
on (j.prof = c1.prof1 and j.subprof = c1.subprof1) left outer join
candidates c2
on (j.prof = c2.prof2 and j.subprof = c2.subprof2) left outer join
. . .
where c1.prof1 is not null or c2.prof1 is not null or
c3.prof1 is not null or c4.prof1 is not null
And you need to remove duplicates because one candidate might have multiple qualifications.
If your data structures were normalised, this kind of query becomes easier, and your database becomes more flexible.
IE : Your table should look more like
CandidateID ProfessionOrder Profession SubProfession
1 1 100 200
1 2 300 400
...
2 1 101 200
The following query based on your current data structures firstly normalises the candidate/professions table, and then joins in order to demonstrate the ease of finding the solution with normalised data structures.
select
candidateid
from
jobs
inner join
(
select
candidateid, prof1 as profession, subprof1 as subprofession
from candidates
union
select
candidateid, prof2 , subprof2
from candidates
union
select
candidateid, prof3 , subprof3
from candidates
union
select
candidateid, prof4 , subprof4
from candidates
) candidates
on jobs.profession = candidates.profession
and jobs.subprofession = candidates.subprofession

How to divide two values from the same column in SQL

As the title states I would like to divide two values with each other that are in the same column.
E.g.
A B C D
Shirts 2011 85 0
Shirts 2012 92 percent change from 2011 to 2012
Shirts 2013 100 percent change from 2012 to 2013
Pants 2011 31 0
Pants 2012 42 percent change from 2011 to 2012
Pants 2013 55 percent change from 2012 to 2013
Jacket 2011 10 0
Jacket 2012 16 percent change from 2011 to 2012
Jacket 2013 18 percent change from 2012 to 2013
In this example column D would be a derived from column C, where the value of 2012 is subtracted from 2011, and then times by a 100 to get the percent.
I don't know how to set the query up I tried doing a bunch of sub-queries but didn't know how to link them together. Any help would be greatly appreciated.
Here's an option to get what you're after
SELECT t1.a, t1.b, t1.c,
CASE WHEN t2.c IS NULL THEN 0 ELSE t1.c - t2.c END AS d,
CASE WHEN t2.c IS NULL THEN 0 ELSE (1.0 * t1.c - t2.c) / t1.c * 100.0 END AS pct
FROM t t1
LEFT OUTER JOIN t t2 ON t1.a = t2.a
AND t1.b = t2.b + 1
SQL Fiddle Example
If I'm understanding your requirements correctly, you could use a recursive cte to accomplish your results. This uses ROW_NUMBER() and partitions by (groups by) column A.
This method would best work if you can't guarantee sequential years in column B. If you always have sequential years, then bobs provides the best alternative.
with cte as (
select A, B, C,
ROW_NUMBER() OVER (PARTITION BY A ORDER BY B) rn
from yourtable
),
recursive_cte as (
select A, B, C, 0 prc
from cte
where rn = 1
union all
select y.A, y.B, y.C, y.C-c.C prc
from cte y
join cte c on y.a=c.a and y.rn=c.rn+1
)
select *
from recursive_cte
order by a, b
SQL Fiddle Demo
This will return the difference between each year and the previous grouped by column A. It uses a 2nd CTE just for simplicity. If you want the actual percent changed, update the formula above.
--EDIT
Since it sounds like you're looking for percentage growth, try replacing y.C-c.C prc with this formula instead:
cast(y.C as decimal(10,2))/cast(c.C as decimal(10,2))
Use CAST if your data types are integers.
You can accomplish this with a self-join:
SELECT curr.ClothingType, curr.SalesYear, curr.NumberSold
, COALESCE((CAST((curr.NumberSold - prev.NumberSold) AS NUMERIC(5,2)) / prev.NumberSold) * 100, 0) AS PercentChange
FROM #Clothes AS curr
LEFT JOIN #Clothes AS prev ON curr.ClothingType = prev.ClothingType
AND curr.SalesYear = prev.SalesYear + 1
SQL Fiddle Example
I agree with Shivan, but only because I don't know how to do it without PHP. With PHP it's just a matter of pulling the two totals, doing the math and sending a query back with the answers.
ALSO BE SURE TO READ THIS BECAUSE IT'S THE MORE IMPORTANT PART
Your math's wrong. (Shirts2011 - Shirts2012) * 100 doesn't give you the percent change. It just gives you 100 times the difference of the two (actually, the negative difference based on your wording). What you want, math-wise, is:
((Item2012-Item2011)/Item2011)*100
I know the outside parens aren't necessary but they make it easier to read. The percent change is the same as 100 times the ratio of the difference of the two to the initial.
Edited to add PHP code.
The PHP looks gross, but it gets the job done.
$conn = new PDO(HOST:DATABASE, USERNAME, PASSWORD);
$query = "SELECT A, B, C FROM yourTable";
$st = $conn->prepare($query);
$st->execute();
$list = array();
while($row = $st->fetch(PDO::FETCH_ASSOC))
$list[] = $row;
$fill = array();
for($i=1; $i<count($list); $i++)
if($list[$i]['A'] == $list[$i-1][A])
$fill[] = $array('A' => $list[$i]['A'],
'B' => $list[$i]['B'],
'D' => ($list[$i]['C']-$list[$i-1]['C'])/$list[$i-1]['C']*100);
$update = new PDO(HOST:DATABASE, USERNAME, PASSWORD);
for($i=0; $i<count($fill); $i++){
$query = "UPDATE yourTable SET D = " . $fill[$i]['D'] . " WHERE A = " . $fill[$i]['A'] . " AND B = " . $fill[$i]['B'];
$st = $update->prepare($query);
$st->execute();
}
Just initialize your table where the 2011 D columns = 0.
UPDATE yourTable SET D = 0 WHERE B = 2011;
And before the flames start, I know a foreach would have worked; I just think a standard for looks better (and allows you to skip the first 2011 value).
Try this:
SELECT t1.type A, t1.year B, t1.value C,
IF(
EXISTS(
SELECT 1 FROM table t2 WHERE t1.type = t2.type AND t1.year = (t2.year - 1)
),
((SELECT t3.year FROM table t3 WHERE t1.type = t3.type AND t1.year = (t3.year - 1)) - t1.year)/t1.year,
0
) D
FROM table t1
Not super efficient, but it can be done.
You could try this too:
UPDATE mytable t1
INNER JOIN mytable t2
ON t1.A = t2.A AND t2.B = t1.B - 1
SET t1.D = ((t1.C - t2.C) / t2.C) * 100;
see fiddle.
Assuming you already have 0 as a default for D. If not, just set it to 0 for all rows before running this query.
Try this:
select Y2013.*, Y2012.C, Y2013.C, Y2013.C/Y2011.C
from (
select * from data where B = year(getdate())
) Y2013
left join (
select * from data where B = year(getdate()) - 1
) Y2012 on Y2012.A = Y2013.A
left join (
select * from data where B = year(getdate()) - 2
) Y2011 on Y2011.A = Y2012.A
and Y2011.A = Y2013.A

Parsing simplified SQL queries with Perl into SQLite

I am trying to turn a "simplified" sql query into a working SQLite one to use against XnViews databases, meaning the database layout is at best suboptimal for what I'm trying to do AND I can't change anything about that.
Example would be "(cat_10 and cat_12) and (cat_5 or cat_7)".
This should be used against the table "t3", which has the fields "if" (fileID) and "ic" (categoryID).
The entries look like this:
if, ic
7, 10
7, 12
7, 4
9, 10
9, 12
9, 5
10, 10
10, 12
10, 7
The simplified query above should only select the files 9 and 10 as 7 does have the wanted categories 10 and 12 but has neither 5 nor 7.
The actual problem now is building that hell of a query statement because it took me already some hours to simply get an AND between two categories working.
SELECT if FROM t3 WHERE ic IN (10, 12) GROUP BY if HAVING count(if) = 2
This gives me all fileIDs that contain category 10 and 12, but I have no idea how I should combine that with the remaining " and (cat_5 or cat_7)".
When I planned these simplified sql statements (made by a click-it-together-builder made in html and js) I was planning to simply replace "cat_5" with "t3.ic = 5" and leave the rest as it is.
Of course I didn't forsee that it wouldn't work as where checks the entry as a whole and there can't be ic = 5 AND ic = 7. That pretty much broke everything.
So I'm wondering if anyone would have an idea how I could translate these simple queries in actual working ones, keeping in mind that it might not be limited to ( x and y ) pairs.
Edit: I worked out how to do the example I've given, I think atleast:
SELECT if FROM t3 WHERE ic IN (10, 12) GROUP BY if HAVING count(if) = 2
INTERSECT
SELECT if FROM t3 WHERE ic IN (5, 7) GROUP BY if
But the main problem now is resolving the ( ) in the right order.
Edit 2: I think I'm giving grouping the categories into one field with group_concat() a try, then I should be able to simply to cats LIKE " " AND which would be small blocks I could easy throw together, then just the brackets and it should work. Highlighting the 'should'.
Your original query doesn't do what is intended. WHERE ic IN (10, 12) GROUP BY if HAVING count(if) = 2 would yield the right result even when you have ics in 10 and 10 again but not 12 at all. This is against your textual description of what you want. This is where an inner query to fetch results for 12 and 10 is needed. You can test your query to fail in the fiddle link I have posted below.
Bit tricky, but this is how I would interpret it straightforward.
SELECT DISTINCT ifc
FROM t3
WHERE ifc IN (
SELECT ifc
FROM t3
WHERE ic = 10
GROUP BY ifc
HAVING COUNT(*) > 0
INTERSECT
SELECT ifc
FROM t3
WHERE ic = 12
GROUP BY ifc
HAVING COUNT(*) > 0
)
AND ic IN (5, 7)
Try fiddle
I did not bring in any optimization, you may try yours. The fiddle link is of Postgres but this should work ( did not get SQLite to work in my browser :( )
Edit: CL. points out an interesting thing about not having to include HAVING clauses in the inner query which is true. I was interpreting OP's requirement in SQL terms with an intent to make things clear without thinking of any optimizations.
Here is a better looking query:
SELECT DISTINCT ifc
FROM t3
WHERE ifc IN (
SELECT ifc
FROM t3
WHERE ic = 10
INTERSECT
SELECT ifc
FROM t3
WHERE ic = 12
)
AND ic IN (5, 7)
Ok I got it working as I originally planned surprisingly.
SELECT Folders.Pathname || Images.Filename AS File FROM Images
JOIN Folders ON Images.FolderID = Folders.FolderID
LEFT JOIN (
SELECT f, Cats, t4.if AS Tagged FROM t2
JOIN (
SELECT if, ' ' || group_concat(ic,' ') || ' ' AS Cats FROM t3 GROUP BY if
) st3 ON t2.i = st3.if
LEFT JOIN t4 ON t2.i = t4.if
) st2 ON File = st2.f
$selectWhereImage $sqlqry
ORDER BY ModifiedDate $order LIMIT $offset, $limit
I know this is one hell of a query but it combines all things I'd be looking for (category ids, tagged or not, rating, color) sortable by date with the full filepath as result.
It's probably a horrible way to do it but if anyone finds a better working way where I can simply replace placeholders like "cat_5" while keeping the rest like it is, needed for brackets and operators, then I'm all ears :D
Oh and $selectWhereImage contains just a longer WHERE that limits File to be ending with an imageformat, $sqlqry is the refittet thing from above, cat_5 would just turn into cats LIKE '% 5 %', due to the additional spaces left and right of cats I can match any number without finding "1" in "10" since " 1 " isn't in " 10 " :D
A hackish approach which would be simpler and I believe faster is
SELECT DISTINCT ifc
FROM t3
WHERE ifc IN (
SELECT ifc
FROM t3
WHERE ic = 10
)
AND ifc IN (
SELECT ifc
FROM t3
WHERE ic = 12
)
AND ic IN (5, 7)
If you have to use an intersect as you have done you should change your upper query which is wrong. Since you have to ensure every if has a 10 and 12 as ic then you can't get away without two separate queries for that. Something like:
SELECT ifc
FROM t3
WHERE ifc IN (
SELECT ifc
FROM t3
WHERE ic = 10
)
AND ifc IN (
SELECT ifc
FROM t3
WHERE ic = 12
)
INTERSECT
SELECT ifc FROM t3 WHERE ic IN (5, 7)
The INTERSECT will handle the group by here so you dont have to explicitly add but this will not be as efficient as my other queries. If you have to get away with subqueries, you can use JOIN:
SELECT DISTINCT t.ifc
FROM t3 AS t
JOIN t3 AS v ON v.ifc = t.ifc
JOIN t3 AS p ON p.ifc = t.ifc
WHERE v.ic = 10 AND p.ic = 12 AND t.ic IN (5, 7)
The second one has the advantage that it works on databases that doesn't know INTERSECT like MySQL.

How can I write this query as a full join instead of a union left/right join?

here's the code, showing the inputs and the required output.
Basically, I'm trying to self-join to match the results of my broker's statement with my internal records. So left set of columns is broker's list, right side is my list. If broker has a position, and I don't, NULLs on the right. If I have a position and broker doesn't, NULLs on the left.
The left join + right join + union works exactly as I want. Seems like there should be some voodoo to allow a full join to get that without two selects, but I can't figure it out.
drop table MatchPositions
go
create table MatchPositions
(
mt_source varchar (10),
mt_symbol varchar (10),
mt_qty float,
mt_price float
)
go
insert into MatchPositions values ('BROKER', 'IBM', 100, 50.25)
insert into MatchPositions values ('BROKER', 'MSFT', 75, 30)
insert into MatchPositions values ('BROKER', 'GOOG', 25, 500)
insert into MatchPositions values ('BROKER', 'SPY', 200, 113)
insert into MatchPositions values ('MODEL', 'MSFT', 75, 30)
insert into MatchPositions values ('MODEL', 'GOOG', 25, 500)
insert into MatchPositions values ('MODEL', 'GLD', 300, 150)
go
select * from MatchPositions b
left join MatchPositions m on b.mt_symbol = m.mt_symbol and m.mt_source = 'MODEL'
where b.mt_source = 'BROKER'
union
select * from MatchPositions b
right join MatchPositions m on b.mt_symbol = m.mt_symbol and b.mt_source = 'BROKER'
where m.mt_source = 'MODEL'
and here's the expected output:
mt_source mt_symbol mt_qty mt_price mt_source mt_symbol mt_qty mt_price
---------- ---------- ---------------------- ---------------------- ---------- ---------- ---------------------- ----------------------
NULL NULL NULL NULL MODEL GLD 300 150
BROKER GOOG 25 500 MODEL GOOG 25 500
BROKER IBM 100 50.25 NULL NULL NULL NULL
BROKER MSFT 75 30 MODEL MSFT 75 30
BROKER SPY 200 113 NULL NULL NULL NULL
;WITH T1 AS
(
SELECT *
FROM MatchPositions
WHERE mt_source = 'BROKER'
), T2 AS
(
SELECT *
FROM MatchPositions
WHERE mt_source = 'MODEL'
)
SELECT *
FROM T1 FULL JOIN T2 ON T1.mt_symbol = T2.mt_symbol
Possibly using an isnull function:
SELECT *
FROM MatchPositions b
FULL JOIN MatchPositions m on b.mt_symbol = m.mt_symbol
and b.mt_source != m.mt_source
WHERE isnull(b.mt_source, 'BROKER') = 'BROKER'
and isnull(m.mt_source, 'MODEL') = 'MODEL'
SELECT *
FROM MatchPositions b
FULL JOIN MatchPositions m ON b.mt_symbol = m.mt_symbol
AND b.mt_source = 'BROKER'
AND m.mt_source = 'MODEL'
This filters the table into the 'BROKER' and 'MODEL' parts before outer joining them.
Try this:
select *
from MatchPositions broker
full join MatchPositions model on model.mt_symbol = broker.mt_symbol
and model.mt_source <> broker.mt_source
where ( broker.mt_source = 'BROKER' or broker.MT_SOURCE is null )
and ( model.mt_source = 'MODEL' or model.MT_SOURCE is null )
From the first logical source table, you want either broker rows, or missing rows.
From the second logical source table, you want either model rows or missing rows.
If your RDBMS supports FULL JOIN (also known as FULL OUTER JOIN):
SELECT *
FROM (SELECT * FROM MatchPositions WHERE mt_source = 'BROKER') b
FULL
JOIN (SELECT * FROM MatchPositions WHERE mt_source = 'MODEL' ) m
ON b.mt_symbol = m.mt_symbol
This solution is basically same as Martin's, just uses a different syntax, which may be helpful in case your RDBMS doesn't support CTEs.