I have two queries which links to different databases
query = "select name ,ctry from xxxx where xxxx"
cursor.execute(query)
results1 = list(cursor.fetchall())
for row in results1:
query1 = "SELECT sessionname, country FROM xxx where and sessions.sessionname = '"+row[0] +"'"
cur.execute(query1)
results2.append(cur.fetchall())
How to connect them if they have common value(sessionname and name) and save it's output to file. Both queries are located in different dbo (oracle, postgresql)
My code is here :
try:
query = """select smat.s_name "SQLITE name" ,smed.m_ctry as "Country", smed.m_name "HDD Label" from smart.smed2smat ss, smart.smed smed, smart.smat smat where ss.M2S_SMAT=smat.s_id and ss.m2s_smed=smed.m_id and smed.m_name like '{0}%' order by smat.s_name""" .format(line_name)
cursor.execute(query)
columns = [i[0] for i in cursor.description]
results1 = cursor.fetchall()
for row in results1:
query1 = "SELECT sessions.sessionname, projects.country , projects.projectname FROM momatracks.sessions, momatracks.projects, momatracks.sessionsgeo where sessions.projectid = projects.id and sessionsgeo.sessionname = sessions.sessionname and sessions.sessionname = '"+row[0] +"' order by sessions.sessionname"
cur.execute(query1)
results2 =cur.fetchall()
print "results1 -----> \n:", row
tmp=[]
output_items = []
for tmp in results2:
print "---> \n", tmp
try:
stations_dict = dict([(item[0], item[1:]) for item in tmp])
for item in row:
output_item = list(item) + stations_dict.get(item[0], [])
output_items.append(output_item)
except Exception, f:
print str (f)
cursor.close()
cur.close()
except Exception, g:
print str ( g )
except Exception, e:
print str ( e )
My results from row and tmp are :
row - WE246JP_2015_10_11__14_53_33', 'NLD', '031_025_SQLITE_NLD1510_03INDIA
and
tmp - WE246JP_2015_10_11__14_53_33', 'NLD', 'NLD15_N2C1-4_NL'
How to properly connect them? I want output look like this :
output_items - WE246JP_2015_10_11__14_53_33', 'NLD', '031_025_SQLITE_NLD1510_03INDIA', 'NLD15_N2C1-4_NL'
At the moment i get this error :
can only concatenate list (not "str") to list
Also value station_dict looks like this :( And this is not what i intended to do
'W': 'E246JP_2015_10_11__15_23_33', 'N': 'LD15_N2C1-4_NL3'
I know there is something wrong with my code which is simmilar to join. Can anyone explain this to me ? I used method below :
http://forums.devshed.com/python-programming-11/join-arrays-based-common-value-sql-left-join-943177.html
If the sessions are exactly the same in both databases then just zip the results:
query = """
select
smat.s_name "SQLITE name",
smed.m_ctry as "Country",
smed.m_name "HDD Label"
from
smart.smed2smat ss
inner join
smart.smed smed on ss.M2S_SMAT = smat.s_id
inner join
smart.smat smat on ss.m2s_smed = smed.m_id
where smed.m_name like '{0}%'
order by smat.s_name
""".format(line_name)
cursor.execute(query)
results1 = cursor.fetchall()
query1 = """
select
sessions.sessionname,
projects.country,
projects.projectname
from
momatracks.sessions,
inner join
momatracks.projects on sessions.projectid = projects.id
inner join
momatracks.sessionsgeo on sessionsgeo.sessionname = sessions.sessionname
where sessions.sessionname in {}
order by sessions.sessionname
""".format(tuple([row[0] for row in results1]))
cur.execute(query1)
results2 = cur.fetchall()
zipped = zip(results1, results2)
output_list = [(m[0][0], m[0][1], m[0][2], m[1][2]) for m in zipped]
If the sessions are different then make each result a dictionary to join.
I think you can use a subquery here. There's no way for me to test it, but I think it should look like this:
SELECT *
FROM (SELECT smat.s_name "SQLITE name" ,
smed.m_ctry as "Country",
smed.m_name "HDD Label"
FROM smart.smed2smat ss,
smart.smed smed,
smart.smat smat
WHERE ss.M2S_SMAT=smat.s_id
AND ss.m2s_smed=smed.m_id
AND smed.m_name like '{0}%'
ORDER BY smat.s_name) t1,
(SELECT sessions.sessionname,
projects.country ,
projects.projectname
FROM momatracks.sessions,
momatracks.projects,
momatracks.sessionsgeo
WHERE sessions.projectid = projects.id
AND sessionsgeo.sessionname = sessions.sessionname
AND sessions.sessionname = '"+row[0] +"'
ORDER BY sessions.sessionname) t2
WHERE t1."SQLITE name" = t2.sessionname ;
Related
I am trying to run the following sql command on a powershell script:
I set the following as a variable:
$DBSUM = select x.[example 1], y.example2, z.example3
from (select count (distinct valueA) AS "example 1"
from dbo.table1
) as x,
(select count (distinct ValueB) AS "example2"
from dbo.table2
) as y,
(select count (distinct ValueC) AS "example3"
from dbo.table3
) as z
after some other commands, i have the following:
$SqlConnectionSUM = New-Object System.Data.SqlClient.SqlConnection
$SqlConnectionSUM.ConnectionString = "Server = $SQLServer; Database =
$SQLDB; Integrated Security = True"
$SqlCmdSUM = New-Object System.Data.SqlClient.SqlCommand
$SqlCmdSUM.CommandText = $DBSUM
$SqlCmdSUM.Connection = $SqlConnectionSUM
$SqlAdapterSUM = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapterSUM.SelectCommand = $SqlCmdSUM
$DataSetSUM = New-Object System.Data.DataSet
$null = $SqlAdapterSUM.Fill($DataSetSUM)
$DataSetSUM.Tables[0] | select * -ExcludeProperty RowError, RowState,
HasErrors, ItemArray,Table | ConvertTo-Html -Head $HtmlHead | Out-File
"$PSScriptRoot\filelocation\example.html"
The above command works perfectly in SQL, but when I run on my powershell I get the error:
+ ... .example3 from (select count (distinct ValueA) AS "example1 ...
+ ~~~~~~
Unexpected token 'example' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : UnexpectedToken
Please can anyone advise?
Thank you.
Thank you to Luuk and Astentx for your replies.
I have now working ok, the issue was the " and spaces on example 1
I replaced the spaces with _ and removed the " and t works absolutely fine.
Here is the result:
$DBSUM = select x.example_1, y.example2, z.example3
from (select count (distinct valueA) AS example_1
from dbo.table1
) as x,
(select count (distinct ValueB) AS example2
from dbo.table2
) as y,
(select count (distinct ValueC) AS example3
from dbo.table3
) as z
Thank you all.
I am trying to construct a double count left join query in Slick 3.1 similar to this:
SELECT
COUNT(p.id) AS replies,
COUNT(f.id) AS images
FROM posts AS p
LEFT JOIN files AS f
ON p.id = f.post_id
WHERE thread = :thread_id
Join part of the query is quite simple and looks like this:
val joinQ = (threadId: Rep[Long]) =>
(postDAO.posts joinLeft fileRecordDAO.fileRecords on (_.id === _.postId))
.filter(_._1.thread === threadId)
Using joinQ(1L).map(_._1.id).length generates COUNT(1) which counts all rows - not the result I want to obtain. Using joinQ(1L).map(_._1.id).countDistinct generates COUNT(DISTINCT p.id) which is somewhat what I'm looking for, but trying to do two of these generates this monstrosity:
select x2.x3, x4.x5
from (select count(distinct x6.`id`) as x3
from `posts` x6
left outer join `files` x7 on x6.`id` = x7.`post_id`
where x6.`thread` = 1) x2,
(select
count(distinct (case when (x8.`id` is null) then null else 1 end)) as x5
from `posts` x9
left outer join `files` x8 on x9.`id` = x8.`post_id`
where x9.`thread` = 1) x4
Here's the double countDistinct code:
val q = (threadId: Rep[Long]) => {
val join = joinQ(threadId)
val q1 = join.map(_._1.id).countDistinct // map arg type is
val q2 = join.map(_._2).countDistinct // (Post, Rep[Option[FileRecord]])
(q1, q2)
}
Any ideas? :)
Count is aggregation function, so you also need grouping by some field (e.x. id). Try next query:
def joinQ(threadId: Long) = (postDAO.posts joinLeft fileRecordDAO.fileRecords on (_.id === _.postId))
.filter(_._1.thread === threadId)
val q = (threadId: Long) => {
joinQ(threadId).groupBy(_._1.id).map {
case (id, qry) => (id, qry.map(_._1.id).countDistinct, qry.map(_._2.map(_.id)).countDistinct)
}
}
It generates next sql:
SELECT x2.`id`,
Count(DISTINCT x2.`id`),
Count(DISTINCT x3.`id`)
FROM `posts` x2
LEFT OUTER JOIN `files` x3
ON x2.`id` = x3.`post_id`
WHERE x2.`thread` = 10
GROUP BY x2.`id`
I am trying to translate this sql into a slick 3.1 style collection query (single call). This sql (postgres) returns what I am looking for:
select
minDate.min as lastModified,
(select count("id") from "Items" where "orderId" = 1) as totalItemCount,
(select count("id") from "Items" where "orderId" = 1 and "dateModified" >= minDate.min) as addedCount
from
(select min("dateModified") as "min" from "Items" where "orderId" = 1 and "state" = 'new') as minDate
Returns: for a specified set of Items (from orderId), returns:
date of item last modified
total number of items
number of items added since the lastModified
But after many attempts, I can't figure out how to translate this to a single slick-style query
This codes
import scala.slick.driver.PostgresDriver
case class Item(id: Int, orderId: Int, state: String, dateModified: Int)
object SlickComplexQuery {
def main(args: Array[String]) = {
val driver = PostgresDriver
import driver.simple._
class ItemsTable(tag: Tag) extends Table[Item](tag, "Items"){
def id = column[Int]("id")
def orderId = column[Int]("orderId")
def state = column[String]("state")
def dateModified = column[Int]("dateModified")
def * = (id, orderId, state, dateModified) <> (Item.tupled, Item.unapply)
}
val items = TableQuery[ItemsTable]
val query1 = items
.filter(i => i.orderId === 1 && i.state === "new")
.map(_.dateModified)
.min
val query2 = items
.filter(_.orderId === 1)
.map(_.id)
.length
val query3 = items
.filter(i => i.orderId === 1 && i.dateModified >= query1)
.map(_.id)
.length
val query = Query(query1, query2, query3)
results in such query:
select x2.x3, x4.x5, x6.x7
from (select min(x8.x9) as x3
from (select x10."dateModified" as x9
from "Items" x10
where (x10."orderId" = 1) and (x10."state" = 'new')) x8) x2,
(select count(1) as x5
from (select x11."id" as x12
from "Items" x11
where x11."orderId" = 1) x13) x4,
(select count(1) as x7
from (select x14."id" as x15
from "Items" x14, (select min(x16.x17) as x18
from (select x19."dateModified" as x17
from "Items" x19
where (x19."orderId" = 1) and (x19."state" = 'new')) x16) x20
where (x14."orderId" = 1) and (x14."dateModified" >= x20.x18)) x21) x6
This query is much alike yours, slick 2.0 was used.
I have two tables. One is todays prices and one is yesterdays prices. There is a job which updates each table over night. Yesterdays prices is a copy of todays before it gets updated.
I am trying to create a table which shows the differences between the tables.
To do this I am using a full outer join. I am filtering down my criteria to make it faster as both tables are over 48 million rows.
My current code is like this.
WITH differ AS
(
SELECT
tAP.CustomerID, tAP.ProductID, yAP.CustomerID AS 'Yest_CustomerID', yAP.ProductID AS 'Yest_ProductID',
tAP.PDG, tAP.DiscPct1, tAP.DiscPct2,
yAP.DiscPct1 AS 'Yest_DiscPct1',
yAP.DiscPct2 AS 'Yest_DiscPct2',
CONVERT(DECIMAL(18,2),tAP.DiscPct1 - yAP.DiscPct1) AS 'DiscPct1_Difference',
CONVERT(DECIMAL(18,2),tAP.DiscPct2 - yAP.DiscPct2) AS 'DiscPct2_Difference',
tAP.internalSPANumber, yAP.internalSPANumber AS 'Yest_InternalSPANumber', tAP.SPAUniqueReference,
tAP.Project,
tAP.ExpiryDate,tAP.Subaddress, tAP.PriceType, yAP.PriceType AS 'Yest_PriceType', tAP.ListPrice,
tAP.NettPrice, yAP.NettPrice AS 'Yest_NettPrice',
CONVERT(DECIMAL(18,2),tAP.NettPrice- yAP.NettPrice) AS 'NettPrice_Difference'
FROM tbl_Prices tAP FULL OUTER JOIN tbl_Prices_Yesterday yAP ON tAP.CustomerId = yAP.CustomerID AND tAP.ProductID = yAP.ProductID
AND tAP.PDG = yAP.PDG AND tAP.Project = yAP.Project AND tAP.PriceType = yAP.PriceType
WHERE (((tAP.DiscPct1 <> yAP.DiscPct1)
OR (tAP.DiscPct2 <> yAP.DiscPct2)
OR (yAP.CustomerID IS NULL)
OR (tAP.CustomerID IS NULL)
OR (tAP.NettPrice <> yAP.NettPrice)
OR (tAP.InternalSPANumber <> yAP.InternalSPANumber))
AND
(
tAP.ProductID = '10238610' OR tAP.ProductID = '10238620'
OR tAP.ProductID = '10238621' OR tAP.ProductID = '10238687'
OR tAP.ProductID = '10238688' OR yAP.ProductID = '10238610'
OR yAP.ProductID = '10238620' OR yAP.ProductID = '10238621'
OR yAP.ProductID = '10238687' OR yAP.ProductID = '10238688')
)
)
SELECT * INTO tbl_Difference FROM differ
Anyway to make this faster?
Can you filter the 2 large tables before you try to join them using AND filter in your where?
WITH tAPcte AS
(
SELECT * -- fields you need
FROM tbl_Prices
WHERE ProductID IN ('10238610',
'10238620',
'10238621',
'10238687',
'10238688')
),
yAPcte AS
(
SELECT * -- fields you need
FROM tbl_Prices_Yesterday
WHERE ProductID IN ('10238610',
'10238620',
'10238621',
'10238687',
'10238688')
)
differ AS
(
SELECT
tAP.CustomerID, tAP.ProductID, yAP.CustomerID AS 'Yest_CustomerID', yAP.ProductID AS 'Yest_ProductID',
tAP.PDG, tAP.DiscPct1, tAP.DiscPct2,
yAP.DiscPct1 AS 'Yest_DiscPct1',
yAP.DiscPct2 AS 'Yest_DiscPct2',
CONVERT(DECIMAL(18,2),tAP.DiscPct1 - yAP.DiscPct1) AS 'DiscPct1_Difference',
CONVERT(DECIMAL(18,2),tAP.DiscPct2 - yAP.DiscPct2) AS 'DiscPct2_Difference',
tAP.internalSPANumber, yAP.internalSPANumber AS 'Yest_InternalSPANumber', tAP.SPAUniqueReference,
tAP.Project,
tAP.ExpiryDate,tAP.Subaddress, tAP.PriceType, yAP.PriceType AS 'Yest_PriceType', tAP.ListPrice,
tAP.NettPrice, yAP.NettPrice AS 'Yest_NettPrice',
CONVERT(DECIMAL(18,2),tAP.NettPrice- yAP.NettPrice) AS 'NettPrice_Difference'
FROM tAPcte tAP FULL OUTER JOIN yAPcte yAP ON tAP.CustomerId = yAP.CustomerID AND tAP.ProductID = yAP.ProductID
AND tAP.PDG = yAP.PDG AND tAP.Project = yAP.Project AND tAP.PriceType = yAP.PriceType
WHERE (((tAP.DiscPct1 <> yAP.DiscPct1)
OR (tAP.DiscPct2 <> yAP.DiscPct2)
OR (yAP.CustomerID IS NULL)
OR (tAP.CustomerID IS NULL)
OR (tAP.NettPrice <> yAP.NettPrice)
OR (tAP.InternalSPANumber <> yAP.InternalSPANumber))
)
SELECT * INTO tbl_Difference FROM differ
SELECT
PC_SL_ACNO, -- DB ITEM
SLNAME, -- ACCOUNT NAME:
SL_TOTAL_AMOUNT -- TOTAL AMOUNT:
FROM GLAS_PDC_CHEQUES
WHERE PC_COMP_CODE=:parameter.COMP_CODE
AND pc_bank_from = :block02.pb_bank_code
AND pc_due_date between :block01.date_from
AND :block01.date_to
AND nvl(pc_discd,'X') IN(‘X’, 'R')
GROUP BY
pc_comp_code, pc_sl_ldgr_code, pc_sl_acno
ORDER BY pc_sl_acno
ACCOUNT NAME:
BEGIN
SELECT COAD_PTY_FULL_NAME INTO :BLOCK03.SLNAME
FROM GLAS_PTY_ADDRESS,GLAS_SBLGR_MASTERS
WHERE SLMA_COMP_CODE = :PARAMETER.COMP_CODE
AND SLMA_ADDR_ID = COAD_ADDR_ID
AND SLMA_ADDR_TYPE = COAD_ADDR_TYPE
AND SLMA_ACNO = :BLOCK03.PC_SL_ACNO
AND SLMA_COMP_CODE = COAD_COMP_CODE;
EXCEPTION WHEN OTHERS THEN NULL;
END;
TOTAL AMOUNT:
BEGIN
SELECT SUM(PC_AMOUNT) INTO :SL_TOTAL_AMOUNT
FROM GLAS_PDC_CHEQUES
WHERE PC_DUE_DATE BETWEEN :BLOCK01.DATE_FROM AND :BLOCK01.DATE_TO
AND PC_BANK_FROM = :block02.PB_BANK_CODE
AND PC_SL_ACNO = :BLOCK03.PC_SL_ACNO
AND NVL(PC_DISCD,'X') = 'R'
AND PC_COMP_CODE = :PARAMETER.COMP_CODE;
EXCEPTION WHEN OTHERS THEN :block03.SL_TOTAL_AMOUNT := 0;
END;
How can I join the three tables?
You'll have to adjust depending on precisely what criteria and required fields you have for your query or queries.
SELECT
c.PC_SL_ACNO,
a.COAD_PTY_FULL_NAME,
SUM(c.PC_AMOUNT)
FROM
GLAS_PDC_CHEQUES c
LEFT JOIN
GLAS_SBLGR_MASTERS m
ON ( c.PC_SL_ACNO = m.SLMA_ACNO
AND c.PC_COMP_CODE = m.SLMA_COMP_CODE
)
LEFT JOIN
GLAS_PTY_ADDRESS a
ON ( m.SLMA_ADDR_ID = a.COAD_ADDR_ID
AND m.SLMA_COMP_CODE = a.COAD_COMP_CODE
AND m.SLMA_ADDR_TYPE = a.COAD_ADDR_TYPE
)
WHERE
c.PC_COMP_CODE = :PARAMETER.COMP_CODE
AND c.PC_SL_ACNO = :BLOCK03.PC_SL_ACNO
AND c.PC_BANK_FROM = :BLOCK02.PB_BANK_CODE
AND NVL(c.PC_DISCD,'X') IN (‘X’, 'R')
AND c.PC_DUE_DATE BETWEEN :BLOCK01.DATE_FROM AND :BLOCK01.DATE_TO
GROUP BY
c.PC_SL_ACNO, -- not sure which grouping exactly you need.
a.COAD_PTY_FULL_NAME
ORDER BY
c.PC_SL_ACNO
I notice that in the first query you have pc_comp_code as a search criterion, and on the leading edge of your grouping - is that what you need?
This is a bit of an 'estimate' due to the enigmatic nature of your question!