Failed to annotate a left join query in django 1.8 - sql

I try to a new field praise_cnt to 'Store' in django 1.8 but failed. Here is my code.
I've two models:
class Store(models.Model):
...
class Works(models.Model):
store = models.ForeignKey(Store)
...
The query is :
sql = 'select store.*, sum(works.praise_cnt) as praise_cnt from store left join works on store.id = works.store_id ' \
'where store.is_active=1 and store.audit_status=1 ' \
'group by store.id order by praise_cnt desc limit %d,%d' %(offset, rows)
store_list = Store.objects.raw(sql)
for store in store_list:
print store.praise_cnt
The sql is right but the value of store is always 'None'. So how to add a praise_cnt field to store in this case? Thanks
UPDATE: I solve this issue by myself. I replaced 'sum(works.praise_cnt) as praise_cnt' with 'sum(works.praise_cnt) as cnt' and it works. I seems that django doesn't support annotate a new field with same name.

Related

How to write an Open SQL statement with substring in the JOIN ON condition? [duplicate]

I have the following select statement in ABAP:
SELECT munic~mandt VREFER BIS AB ZZELECDATE ZZCERTDATE CONSYEAR ZDIMO ZZONE_M ZZONE_T USAGE_M USAGE_T M2MC M2MT M2RET EXEMPTMCMT EXEMPRET CHARGEMCMT
INTO corresponding fields of table GT_INSTMUNIC_F
FROM ZCI00_INSTMUNIC AS MUNIC
INNER JOIN EVER AS EV on
MUNIC~POD = EV~VREFER(9).
"where EV~BSTATUS = '14' or EV~BSTATUS = '32'.
My problem with the above statement is that does not recognize the substring/offset operation on the 'ON' clause. If i remove the '(9) then
it recognizes the field, otherwise it gives error:
Field ev~refer is unknown. It is neither in one of the specified tables
nor defined by a "DATA" statement. I have also tried doing something similar in the 'Where' clause, receiving a similar error:
LOOP AT gt_instmunic.
clear wa_gt_instmunic_f.
wa_gt_instmunic_f-mandt = gt_instmunic-mandt.
wa_gt_instmunic_f-bis = gt_instmunic-bis.
wa_gt_instmunic_f-ab = gt_instmunic-ab.
wa_gt_instmunic_f-zzelecdate = gt_instmunic-zzelecdate.
wa_gt_instmunic_f-ZZCERTDATE = gt_instmunic-ZZCERTDATE.
wa_gt_instmunic_f-CONSYEAR = gt_instmunic-CONSYEAR.
wa_gt_instmunic_f-ZDIMO = gt_instmunic-ZDIMO.
wa_gt_instmunic_f-ZZONE_M = gt_instmunic-ZZONE_M.
wa_gt_instmunic_f-ZZONE_T = gt_instmunic-ZZONE_T.
wa_gt_instmunic_f-USAGE_M = gt_instmunic-USAGE_M.
wa_gt_instmunic_f-USAGE_T = gt_instmunic-USAGE_T.
temp_pod = gt_instmunic-pod.
SELECT vrefer
FROM ever
INTO wa_gt_instmunic_f-vrefer
WHERE ( vrefer(9) LIKE temp_pod ). " PROBLEM WITH SUBSTRING
"AND ( BSTATUS = '14' OR BSTATUS = '32' ).
ENDSELECT.
WRITE: / sy-dbcnt.
WRITE: / 'wa is: ', wa_gt_instmunic_f.
WRITE: / 'wa-ever is: ', wa_gt_instmunic_f-vrefer.
APPEND wa_gt_instmunic_f TO gt_instmunic_f.
WRITE: / wa_gt_instmunic_f-vrefer.
ENDLOOP.
itab_size = lines( gt_instmunic_f ).
WRITE: / 'Internal table populated with', itab_size, ' lines'.
The basic task i want to implement is to modify a specific field on one table,
pulling values from another. They have a common field ( pod = vrefer(9) ). Thanks in advance for your time.
If you are on a late enough NetWeaver version, it works on 7.51, you can use the OpenSQL function LEFT or SUBSTRING. Your query would look something like:
SELECT munic~mandt VREFER BIS AB ZZELECDATE ZZCERTDATE CONSYEAR ZDIMO ZZONE_M ZZONE_T USAGE_M USAGE_T M2MC M2MT M2RET EXEMPTMCMT EXEMPRET CHARGEMCMT
FROM ZCI00_INSTMUNIC AS MUNIC
INNER JOIN ever AS ev
ON MUNIC~POD EQ LEFT( EV~VREFER, 9 )
INTO corresponding fields of table GT_INSTMUNIC_F.
Note that the INTO clause needs to move to the end of the command as well.
field(9) is a subset operation that is processed by the ABAP environment and can not be translated into a database-level SQL statement (at least not at the moment, but I'd be surprised if it ever will be). Your best bet is either to select the datasets separately and merge them manually (if both are approximately equally large) or pre-select one and use a FAE/IN clause.
They have a common field ( pod = vrefer(9) )
This is a wrong assumption, because they both are not fields, but a field an other thing.
If you really need to do that task through SQL, I'll suggest you to check native SQL sentences like SUBSTRING and check if you can manage to use them within an EXEC_SQL or (better) the CL_SQL* classes.

Django ORM raw query how manage quotes

In my Django project i have to implement a complex RAW query for extract data, for doing this i use the raw django function.
The original query is:
SELECT FT."Riferimento1", FT."Riferimento2", "CodProdotto", "QuantitaFatturata", "PrezzoUnit", "DataFattura", "NumeroFattura", "CodCli"
FROM public.idocuments_as_fatturetestata FT
LEFT JOIN public.idocuments_as_fatturerighe FR ON FT."Riferimento1" = FR."Riferimento1" AND FT."Riferimento2" = FR."Riferimento2"
WHERE FT."CodCli" = '12192';
if i run directly in pgadmin SQL all was done
In django i implement in this fashion:
>>> from idocuments.models import as_fatturetestata as asf
>>> a = asf.objects.raw("SELECT FT."Riferimento1",FT."Riferimento2","CodProdotto","QuantitaFatturata","PrezzoUnit","DataFattura","NumeroFattura","CodCli" FROM public.idocuments_as_fatturetestata FT LEFT JOIN public.idocuments_as_fatturerighe FR ON FT."Riferimento1" = FR."Riferimento1" AND FT."Riferimento2" = FR."Riferimento2" WHERE FT."CodCli" = '12192'")
but i get:
SyntaxError: invalid syntax
maybe the problem is how to manage quotes but i don't understand how transport the sql query into raw directive.
Someone can help me please?
so many thanks in advance
Try using single quotes and escaping the single quotes within the string with \:
a = asf.objects.raw('"SELECT FT."Riferimento1",FT."Riferimento2","CodProdotto","QuantitaFatturata","PrezzoUnit","DataFattura","NumeroFattura","CodCli" FROM public.idocuments_as_fatturetestata FT LEFT JOIN public.idocuments_as_fatturerighe FR ON FT."Riferimento1" = FR."Riferimento1" AND FT."Riferimento2" = FR."Riferimento2" WHERE FT."CodCli" = \'12192\'"')

Doctrine group data

I would like to group doctrine data using the most performatic way. In this following query I need to create an object (or array) that returns the following SiteBundle:DJ with their childs SiteBundle:HJ followed by WHERE filters correctly. I am not sure if I need to use sub-query or group by clause, but I am using the following query:
$sql = ' SELECT dj FROM SiteBundle:DJ dj '
. ' LEFT JOIN SiteBundle:HJ hj WITH dj.id = hj.date '
. ' LEFT JOIN SiteBundle:Job job WITH job.id = dj.job '
. ' LEFT JOIN SiteBundle:Dia d WITH d.id = dj.dia '
. ' WHERE job.active = 1 and job.id = :job '
. ' AND (hj.hour BETWEEN :ini and :fim) AND (d.data >= :now)'
. ' GROUP BY dj.id, hj.id, d.data ORDER BY d.data ASC';
Returns:
array:2 [▼
0 => DJ {#1584 ▶}
1 => DJ {#1580 ▶}
]
$dj->getHJ() returns all the SiteBundle:HJ related to this DJ, but I need only the ones I've filtered in where clause.
How to group them?
what you expected to happen is actually a bad idea in the global context.
doctrine is first and foremost about correctness. even though you filter something in your query (and don't select it), doctrine will provide the full collection when you (essentially) call $object->getCollection() because it's the correct collection for that object. queries of the past don't really matter here.
However, the persistent collection you usually get from doctrine for relations can be filtered: https://symfonycasts.com/screencast/doctrine-relations/collection-criteria
(however, your filter might not be representable by the options it provides)
or you filter the collection manually, i.e. write a new function for that entity that returns a filtered collection.
You have a ";" at the end of the where clause, remove it and re-try.

Pyodbc and Access with query parameter that contains a period

I recently found a bug with some Access SQL queries that I can't seem to track down. I have a fairly straightforward SQL query that I use to retrieve data from an access database that's "managed" in an older application (ie the data is already in the database and I have no real control over what's in there).
import pyodbc
MDB = '******.MDB'
DRV = '{Microsoft Access Driver (*.mdb)}'
PWD = ''
con = pyodbc.connect('DRIVER={};DBQ={};PWD={}'.format(DRV, MDB, PWD))
sql = ('SELECT Estim.PartNo, Estim.Descrip, Estim.CustCode, Estim.User_Text1, Estim.Revision, ' +
'Estim.Comments, Routing.PartNo AS RPartNo, Routing.StepNo, Routing.WorkCntr, Routing.VendCode, ' +
'Routing.Descrip AS StepDescrip, Routing.SetupTime, Routing.CycleTime, ' +
'Routing.WorkOrVend, ' +
'Materials.PartNo as MatPartNo, Materials.SubPartNo, Materials.Qty, ' +
'Materials.Unit, Materials.TotalQty, Materials.ItemNo, Materials.Vendor ' +
'FROM (( Estim ' +
'INNER JOIN Routing ON Estim.PartNo = Routing.PartNo ) ' +
'INNER JOIN Materials ON Estim.PartNo = Materials.PartNo )')
if 'PartNo' in kwargs:
key = kwargs['PartNo']
sql = sql + 'WHERE Estim.PartNo=?'
cursor = con.cursor().execute(sql, key)
# use this for debuging only
num = 0
for row in cursor.fetchall():
num += 1
return num
This works fine for all PartNo except when PartNo contains a decimal point. Curiously, when PartNo contains a decimal point AND a hyphen, I get the appropriate record(s).
kwargs['PartNo'] = "100.100-2" # returns 1 record
kwargs['PartNo'] = "200.100" # returns 0 records
Both PartNos exist when viewed in the other application, so I know there should be records returned for both queries.
My first thought was to ensure kwargs['PartNo'] is a string key = str(kwargs['PartNo']) with no change.
I also tried to places quotes around the 'PartNo' value with no success. key = '\'' + kwargs['PartNo'] + '\''
Finally, I tried to escape the . with no success (I realize this would break most queries, but I'm just trying to track down the issue with a single period) key = str(kwargs['partNo']).replace('.', '"."')
I know using query parameters should handle all the escaping for me, but at this point, I'm just trying to figure out what's going on. Any thoughts on this?
So the issue isn't with the query parameters - everything works as it should. The problem is with the SQL statement. I incorrectly assumed - and never checked - that there was a record in the Materials table that matched PartNo.
INNER JOIN Materials ON Estim.PartNo = Materials.PartNo
will only return a record if PartNo is found in both tables, which in this particular case it is not.
Changing it to
LEFT OUTER JOIN Materials ON Estim.PartNo = Materials.PartNo
produces the expected results. See this for info on JOINS. https://msdn.microsoft.com/en-us/library/bb243855(v=office.12).aspx
As for print (repr(key)) - flask handles the kwarg type upstream properly
api.add_resource(PartAPI, '/api/v1.0/part/<string:PartNo>'
so when I ran this in the browser, I got the "full length" strings. When run in the cmd line using python -c ....... I was not handling the argument type properly as Gord pointed out, so it was truncating the trailing zeros. I didn't think the flask portion was relevant, so I never added that in the original question.

DOCTRINE : IN statement ordered?

I am using Symfony2 and doctrine 2, and I have a problem with this query :
$query = $em->createQuery('SELECT a FROM MyBundle:Artiste a WHERE a.id IN (4,12,1)');
$result = $query->getArrayResult();
And I always get results order by a.id, ie 1 then 4 then 12 while I would like to display the results ordered as the list of ids : 4 then 12 then 1.
UPDATE
Thanks to #Bram Gerritsent comment, I register a custom DQL function FIELD, so here is what I have done :
In MyBundle/DQL/Field.php, I have inserted the following code (https://github.com/beberlei/DoctrineExtensions/blob/master/lib/DoctrineExtensions/Query/Mysql/Field.php) (I've just changed the namespace to be namespace MyBundle\DQL;
Then, I add the following in my config.yml as shown in the Symfony2 documentation (http://symfony.com/doc/2.0/cookbook/doctrine/custom_dql_functions.html)
orm:
auto_generate_proxy_classes: "%kernel.debug%"
entity_managers:
default:
auto_mapping: true
dql:
string_functions:
field: MyBundle\DQL\Field
So, I wrote the following query $query = $em->createQuery('SELECT a FROM MyBundle:Artiste a WHERE a.id IN (4,12,1) ORDER BY FIELD(4,12,1)'); but I'm getting this error : [Syntax Error] line 0, col 75: Error: Expected end of string, got '('
You need to have a look into MySql FIELD function.
In native MySql you would do something like this:
ORDER BY FIELD(a.id,4,12,1)
The field function isn't part of the Doctrine 2 distribution, but you can get it from the DoctrineExtensions.
See this StackOverflow post for more information about using the FIELD function in Doctrine 2
EDIT
I have tested it using your query but got the same syntax error. The following query works for me. Not sure why you cannot use ORDER BY field(a.id,4,12,1) directly, but you have to create a HIDDEN field in your select first.
SELECT a, field(a.id,4,12,1) as HIDDEN field FROM MyBundle:Artiste a WHERE a.id IN (4,12,1) ORDER BY field
EDIT2
I have done some more debugging and researching and the DQL parser doesn't seem to support string functions in the order by clause. I've fixed the issue and created a Pull Request.
Not so nice as FIELD function but should work:
SELECT output.a FROM (
SELECT a, ( CASE WHEN a.id = 4 THEN 1 WHEN a.id = 12 THEN 2 a.id = 1 THEN 3 END ) ord FROM MyBundle:Artiste a WHERE a.id IN (4,12,1)) output ORDER BY output.ord