I'm trying to code the following query using Zend_Db_Select:
SELECT p1.SKU,
(
SELECT ',' + Status
FROM "Products" "p2"
WHERE p2.SKU = p1.SKU
ORDER BY "Status"
FOR XML PATH ('')
) AS "Statuses"
FROM "Products" p1
GROUP BY SKU
This is what I have so far:
$s1 = $products->select()
->setIntegrityCheck(false)
->from(array('p2' => 'Products'),
new Zend_Db_Expr("',' + Status")
)
->where('p2.SKU = p1.SKU')
->order('Status');
$s2 = $products->select()
->from(array('p1' => 'Products'),
array('p1.SKU',
'Statuses' => new Zend_Db_Expr('(' . $s1 . ')')
)
)
->group('SKU');
echo $s2;
$dbRowSet = $Products->fetchAll($s2);
That gives me this:
SELECT "p1"."SKU",
(
SELECT ',' + Status
FROM "Products" AS "p2"
WHERE (p2.SKU = p1.SKU)
ORDER BY "Status" ASC
) AS "Statuses"
FROM "Products" AS "p1"
GROUP BY "SKU"
I can't figure out how to get the required FOR XML PATH ('').
Also, isn't using the . operator with $s1 calling __toString(), instead of leaving it as a native Zend_Db_Select object. Is there any other way to get the parens around $s1?
Alternatively, is there another way to do this whole query? I want to return each SKU and a concatenated grouping of all Statuses (a la GROUP_CONCAT() in MySQL). The table is huge, so iterating over them in PHP takes an unacceptably long time.
Unfortunately Zend Framework does not have a great support for constructing MS SQL specific queries. What you could do is use Zend_Db_Adapter_Abstract::query() and skip the object oriented query abstraction altogether. Alternatively, you could extend the Zend_Db_Select, add the appropriate code to Zend_Db_Select::$_parts and Zend_Db_Select::_render*, however you would still have an incomplete support.
I don't quite understand what exactly are you doing in the second code example, since the $2 variable is not assigned at all.
Don't worry about the __toString() being called when you use . for string construction; the expression will ultimately be subjected to string conversion either way.
Looks like I was close, and with KSiimson's comments about string conversion in mind, this works:
$s1 = $products->select()
->setIntegrityCheck(false)
->from(array('p2' => 'Products'),
new Zend_Db_Expr("',' + Status")
)
->where('p2.SKU = p1.SKU')
->order('Status');
$s2 = $products->select()
->from(array('p1' => 'Products'),
array('p1.SKU',
'Statuses' => new Zend_Db_Expr('(' . $s1 .
" FOR XML PATH(''))")
)
)
->group('SKU');
echo $s2;
$dbRowSet = $Products->fetchAll($s2);
This just concats the FOR XML PATH clause with the first query as a string. Not quite as elegant as I was hoping for, but "perfect is the enemy of good".
Related
I have several OR in my SQL statement so I want to save a chuck of it in a cfsavecontent. Here is that part:
<cfsavecontent variable="checkDepartment">
<cfif #wrkDept# EQ #dept[2][1]#>
Department = 'Health' AND
<cfelse>
Department = '#wrkDept#' AND
</cfif>
</cfsavecontent>
But the error I get on the page shows 2 sets of apostrophes around the word Health.
SQL
SELECT COUNT(*) AS numItems
FROM IT_PROJECTS
WHERE
Department = ''Health'' AND
status = 'Cancelled'
Can anyone help me to only get a single apostrophe? Thanks
So this answer seems a lot more complicated than it really is. And without knowing specifically what your query looks like (re:OR conditions), I'm not really sure how to structure it. It can be better. The goal should be to make one single trip to your SQL server with the query that makes the most sense for the data you're trying to get. I'm not sure what you are trying to do with cfsavecontent, but I don't think you need it.
The bulk of my example query (https://trycf.com/gist/4e1f46bfa84a6748aced0f9ee8221c6d/acf2016?theme=monokai) is setup. I chose to go with a cfscript format, because as Redtopia said, I also find it much easier to build a dynamic query in cfscript.
After initial setup, I basically just script out the variables I'll use in my final queryExecute().
// Base query.
qry = "SELECT count(*) AS theCount FROM IT_PROJECTS WHERE 1=1 " ;
// This is our dynamic filter that we build below.
qfilter = {} ;
// Query options.
opts = { "dbtype":"query" } ;
After we have our base, I build up the dynamic part of the query. This is the part that will likely change quite a bit depending on your current needs and setup.
For the first part, I basically replaced your cfif with a ternary evaluation. I'm not sure how your data plays into the evaluation of dept or where that array comes from. But from there I build a basic included statement of the query and set up the queryparam values for it. Then I add a second check that will pick a different set of values for the query (currently based on even/odd seconds). Again, I'm not sure of the intent of your query here, so I just made something dynamic.
//////////// BUILD DYNAMIC FILTER ////////////
qdept = ( wrkDept == dept[2][1] ) ? 'Health' : wrkDept ;
/// This one is an included filter:
qry &= " AND department = :dpt AND status = :sts " ;
qfilter.dpt = {"value":qdept,"cfsqltype":"CFSQLVARCHAR"} ;
qfilter.sts = {"value":"Cancelled","cfsqltype":"CFSQLVARCHAR"} ;
/// Adding Dynamic ORs
// Dynamically set status based on even/odd seconds.
qStatus = ( now().second()%2==0) ? "Cancelled" : "Active" ;
qry &= " OR ( department = :dpt2 AND status = :sts2 ) " ;
qfilter.dpt2 = {value:"IT",cfsqltype:"CFSQLVARCHAR"} ;
qfilter.sts2 = {value:qStatus,cfsqltype:"CFSQLVARCHAR"} ;
This gives us a SQL string that looks like:
SELECT count(*) AS theCount
FROM IT_PROJECTS
WHERE 1=1
AND department = :dpt AND status = :sts
OR
( department = :dpt2 AND status = :sts2 )
With a SQL statement, the placement of AND and OR conditions can greatly impact the results. Use parenthesis to group conditions how you need them.
After we've built the query string, we just have to plug it and our queryparams into the queryExecute().
result = queryExecute( qry , qfilter , opts ) ;
And if we want to output our data, we can go:
writeOutput("There are " & result.theCount & " records." ) ;
Which gives us:
There are 8 records.
Again, I don't know what your main conditions look like. If you can give me an example of a query with a bunch of ORs and ANDs, I'll try to modify this for you.
I would like to group doctrine data using the most performatic way. In this following query I need to create an object (or array) that returns the following SiteBundle:DJ with their childs SiteBundle:HJ followed by WHERE filters correctly. I am not sure if I need to use sub-query or group by clause, but I am using the following query:
$sql = ' SELECT dj FROM SiteBundle:DJ dj '
. ' LEFT JOIN SiteBundle:HJ hj WITH dj.id = hj.date '
. ' LEFT JOIN SiteBundle:Job job WITH job.id = dj.job '
. ' LEFT JOIN SiteBundle:Dia d WITH d.id = dj.dia '
. ' WHERE job.active = 1 and job.id = :job '
. ' AND (hj.hour BETWEEN :ini and :fim) AND (d.data >= :now)'
. ' GROUP BY dj.id, hj.id, d.data ORDER BY d.data ASC';
Returns:
array:2 [▼
0 => DJ {#1584 ▶}
1 => DJ {#1580 ▶}
]
$dj->getHJ() returns all the SiteBundle:HJ related to this DJ, but I need only the ones I've filtered in where clause.
How to group them?
what you expected to happen is actually a bad idea in the global context.
doctrine is first and foremost about correctness. even though you filter something in your query (and don't select it), doctrine will provide the full collection when you (essentially) call $object->getCollection() because it's the correct collection for that object. queries of the past don't really matter here.
However, the persistent collection you usually get from doctrine for relations can be filtered: https://symfonycasts.com/screencast/doctrine-relations/collection-criteria
(however, your filter might not be representable by the options it provides)
or you filter the collection manually, i.e. write a new function for that entity that returns a filtered collection.
You have a ";" at the end of the where clause, remove it and re-try.
I am inside a function in a controller.
So from the Form, I get a value for a variable, say:
$x = "whatever";
Then I need to embed that variable (so, its value), in the WHERE statement. If I hardcode the value, it brings a correct result, but I have tried in all ways to insert that variable without success. Well, supposing that I manage to use that variable, then I will have to look into binding to avoid sql injection, but so far, I would say, see if that variable can get used in the query.
I have tried, double quotes, concatenation . $vx . , curly braces {$x}, the variable plain like this $variable, but either gives syntax errors in some cases, (concatenation), or if I just embed the variable like this where author = $x, it tells me that it can't find the column named $x
$x = "whatever";
$results = DB::select(DB::raw('SELECT
t.id, t.AvgStyle, r.RateDesc
FROM (
SELECT
p.id, ROUND(AVG(s.Value)) AS AvgStyle
FROM posts p
INNER JOIN styles s
ON s.post_id = p.id
WHERE author = $x
GROUP BY p.id
) t
INNER JOIN rates r
ON r.digit = t.AvgStyle'
));
This appears to be a simple PHP variable interpolation issue.
DB::raw() wants literally raw SQL. So there are a couple of issues that need to be fixed in the SQL string you are passing.
PHP Variable interpolation (injecting variables into a string) only happens if you use double quotes around the string. With single quotes it becomes a string constant.
If Author is a char/varchar, then SQL syntax requires quotes around the string in your raw SQL statement. Query builders typically take care of these issues for you, but you are going around them.
So the "fixed" version of this would be:
$x = "whatever";
$results = DB::select(DB::raw("SELECT
t.id, t.AvgStyle, r.RateDesc
FROM (
SELECT
p.id, ROUND(AVG(s.Value)) AS AvgStyle
FROM posts p
INNER JOIN styles s
ON s.post_id = p.id
WHERE author = '$x'
GROUP BY p.id
) t
INNER JOIN rates r
ON r.digit = t.AvgStyle"
));
Like all interpolation, this opens you up to the possibility of SQL injection if the variable being interpolated comes from user input. From the original question it is unclear whether this is a problem.
DB::select() has an option that allows you to pass an array of parameters that is inherently safe from SQL injection. In that case the solution would be:
$x = "whatever";
$results = DB::select(DB::raw("SELECT
t.id, t.AvgStyle, r.RateDesc
FROM (
SELECT
p.id, ROUND(AVG(s.Value)) AS AvgStyle
FROM posts p
INNER JOIN styles s
ON s.post_id = p.id
WHERE author = :author
GROUP BY p.id
) t
INNER JOIN rates r
ON r.digit = t.AvgStyle"
),
array('author' => $x)
);
Regarding this tutorial
$results = DB::select( DB::raw("SELECT * FROM some_table WHERE some_col = :somevariable"), array(
'somevariable' => $someVariable,
));
This is one example for you to insert variable in a raw sql laravel
$query_result = Event::select(
DB::raw('(CASE WHEN status = "draft" THEN "draft"
WHEN events.end_time <= \''.$now.'\' THEN "closed"
ELSE "available"
END) AS status'))
->orderBy('status')
->get();
I am new to Pig Latin.
I want to extract all lines that match a filter criteria (have a word "line_token" ) from log files and then from these matching lines extract two different fields meeting two separate field match criteria . Since the lines aren't structured well I am loading them as a char array.
When I try to run the following code - I get an error
"Invalid resource schema: bag schema must have tuple as its field"
I have tried to perform an explicit cast to a tuple but that does not work
input_lines = LOAD '/inputdir/' AS ( line:chararray);
filtered_lines = FILTER input_lines BY (line MATCHES '.*line_token1.*' );
tokenized_lines = FOREACH filtered_lines GENERATE FLATTEN(TOKENIZE(line)) AS tok_line;
my_wordbag = FOREACH tokenized_lines {
word1 = FILTER tok_line BY ( $0 MATCHES '.*word_token1.*' ) ;
word2 = FILTER tok_line BY ( $0 MATCHES '.*word_token1.*' ) ;
GENERATE word1 , word2 as my_tuple ;
-- I also tried --> GENERATE (word1 , word2) as my_tuple ;
}
dump my_wordbag;
I suppose I am taking a very wrong approach.
Please note - my logs aren't structured well - so I cant mend the way I load
Post loading and initial filtering for lines of interest ( which is straightforward) - I guess I need to do something different rather than tokenize line and iterate through fields trying to find fields.
Or maybe I should use joins ?
Also if I know the structure of line beforehand well as all text fields, then will loading it differently ( not as a chararray) make it an easier problem ?
For now I made a compromise - I added a extra filter clause in my original - line filter and settled for picking just one field from line. When I get back to it I will try with joins and post that code ... - here's my working code that gets me a useful output - but not all that I want.
-- read input lines from poorly structured log
input_lines = LOAD '/log-in-dir-in-hdfs' AS ( line:chararray) ;
-- Filter for line filter criteria and date interested in passed as arg
filtered_lines = FILTER input_lines BY (
( line MATCHES '.*line_filter1*' )
AND ( line MATCHES '.*line_filter2.*' )
AND ( line MATCHES '.*$forDate.*' )
) ;
-- Tokenize every line
tok_lines = FOREACH filtered_lines
GENERATE TOKENIZE(line) AS tok_line;
-- Pick up specific field frm tokenized line based on column filter criteria
fnames = FOREACH tok_lines {
fname = FILTER tok_line BY ( $0 MATCHES '.*field_selection.*' ) ;
GENERATE FLATTEN(fname) as nnfname;
}
-- Count occurances of that field and store it with field name
-- My original intent is to store another field name as well
-- I will do that once I figure how to put both of them in a tuple
flgroup = FOREACH fnames
GENERATE FLATTEN(TOKENIZE((chararray)$0)) as cfname;
grpfnames = group flgroup by cfname;
readcounts = FOREACH grpfnames GENERATE COUNT(flgroup), group ;
STORE readcounts INTO '/out-dir-in-hdfs';
As I understand, after the FLATTEN operation, you have single line (tok_line) in each row and you want to extract 2 words from each line. REGEX_EXTRACT will help you achieve this. I'm not a REGEX expert so will leave writing the REGEX part up to you.
data = FOREACH tokenized_lines
GENERATE
REGEX_EXTRACT(tok_line, <first word regex goes here>) as firstWord,
REGEX_EXTRACT(tok_line, <second word regex goes here>) as secondWord;
I hope this helps.
You must refer to the alias, not the column.
So:
word1 = FILTER tokenized_lines BY ( $0 MATCHES '.*word_token1.*' ) ;
word1 and word2 are going to be aliases as well, not columns.
How do you need the output to look like?
I want to change the query to return multiply values in extra_fields, how can I change the regex? Also I don't understand what extra_fields is - is it a field? If so why it is not called with the table prefix like i.extra_fields?
SELECT i.*,
CASE WHEN i.modified = 0 THEN i.created ELSE i.modified END AS lastChanged,
c.name AS categoryname,
c.id AS categoryid,
c.alias AS categoryalias,
c.params AS categoryparams
FROM #__k2_items AS i
LEFT JOIN #__k2_categories AS c ON c.id = i.catid
WHERE i.published = 1
AND i.access IN(1,1)
AND i.trash = 0
AND c.published = 1
AND c.access IN(1,1)
AND c.trash = 0
AND (i.publish_up = '0000-00-00 00:00:00'
OR i.publish_up <= '2013-06-12 22:45:19'
)
AND (i.publish_down = '0000-00-00 00:00:00'
OR i.publish_down >= '2013-06-12 22:45:19'
)
AND extra_fields REGEXP BINARY '(.*{"id":"2","value":\["[^\"]*1[^\"]*","[^\"]*2[^\"]*","[^\"]*3[^\"]*"\]}.*)'
ORDER BY i.id DESC
The extra_fields is a column of the #__k2_items table. The table qualifier can be omitted, because it is not ambiguous in this query. The column is JSON encoded. That is a serialization format used to store information which is not searchable by design. Applying a RegExp may work one day, but fail another day, since there is no guarantee for id preceeding value (as in your example).
The right way
The right way to filter this is to ignore the extra_fields condition in the SQL query an evaluate in the resultset instead. Example:
$rows = $db->loadObjectList('id');
foreach ($rows as $id => $row) {
$extra_fields = json_decode($row->extra_fields);
if ($extra_fields->id != 2) {
unset($rows[$id]);
}
}
The short way
If you can't change the database layout (which is true for extensions you want to keep updateable), you must split the condition into two, because there is no guarantee for a certain order of the subfields. For some reason, one day value may occur before id. So change your query to
...
AND extra_fields LIKE '%"id":"2"%'
AND extra_fields REGEXP BINARY '"value":\[("[^\"]*[123][^\"]*",?)+\]'
Prepare an intermediate table to hold the contents of extra_fields. Each extra_fields field will be converted into a series of records. Then do a join.
Create a trigger and cronjob to keep the temp table in sync.
Another way is to write UDF in Perl that will decode the field, but AFAIK it is not indexable in mysql.
Using an external search engine is out of scope.
Ok, i didnt want to change the db strucure, i gost some help and changed the regex intoAND extra_fields REGEXP BINARY '(.*{"id":"2","value":\[("[^\"]*[123][^\"]*",?)+\]}.*)'
and i got the right resaults
Thanks