I have some problems with the new version of SQL, which uses the only_full_group_by option. I have two tables: sensor and data. The sensor table HAS_MANY data, so the data table has a foreignkey to the pk of sensor table. Here is one relation:
'avg' => array(self::HAS_MANY, 'Data', 'sensor_id', 'select' => 'AVG(value) AS avg, date AS date', 'group' => 'date', 'order' => 'date desc')
Yii is complaining because there is a column in the SELECT statement that is not aggrgated. This column is the pk of the data table, which seems to be automatically added in the select of the generated query, in fact the query is:
SELECT AVG(value) AS avg, date as date, id_data FROM `data` `avg` WHERE ... GROUP BY date ORDER ...
What I want to do is to remove the added pk or wrap it with ANY_VALUE, so that the only_full_group_by option does not complain anymore.
Thank you
Found the answer by myself, writing it here to make it available to everyone.
To solve the problem you just need to add the id in the SELECT clause and give it the same alias given by the yii generate query. So, in my query I have
SELECT id as t0_c1 ...
I changed it to
SELECT ANY_VALUE(id) as t0_c1 ...
Related
this is an example of data structure in my SQL table
In fact I have many users in my table and some of them have incorrect order of steps (user number 2 in the picture). How can I select all such users? The logic is to select all users that have date of sign_in earlier than date of registration? I suppose regular WHERE clause won't work here. Maybe there is a special function for such cases?
I can see two approaches to solve the problem. For reference this is how I imagine the table might look like
create table users (
user_id int,
action text,
date decimal
);
Use a self join. In this we're basically fetching the records with 'registration' action and adding a self join on matching user_id and 'sign_in' action. Because of the join, the data for each of the action is now available in the same row so this allows you to compare in the where clause
select u1.*
from users u1
join users u2 on u1.user_id = u2.user_id and u2.action = 'sign_in'
where u1.action = 'registration' and u2.date < u1.date;
Use crosstab* function of postgres. This allows you to transpose rows into columns hence gives the ability to compare in the where clause. Personally I think this is more elegant and extensive in the sense that it'll allow you to make other comparisons as well if needed without adding another join. Looking at the cost using "explain", this comes out to be more efficient as well.
SELECT *
FROM crosstab(
'select user_id, action, date
from users
order by user_id, action'
) AS ct(user_id int, del_account decimal, registration decimal, sign_in decimal)
where sign_in < registration;
*Note: In order to use crosstab however you may need superuser access to the database to create the extension. You can do so by running the following query only once
CREATE EXTENSION IF NOT EXISTS tablefunc;
Hope this helps. Let me know in the comments if there's any confusion
Your question is a bit vague yet the problem is generic enough.
First let's make your actions comparable and sortable in the right sequence, for example '1.registration', '2.sign_in', '3.del_account' instead of 'registration', 'sign_in', 'del_account'. Even better, use action codes, 2 for sign_in, 1 for registration etc.
Then you can detect misplaced actions and select the list of distinct user_id-s who did them.
select distinct user_id from
(
select user_id,
action > lead(action) over (partition by user_id order by "date") as misplaced
from the_table
) as t
where misplaced;
This approach would work for ay number of action steps, not only 3.
If you create a case statement for the action column you can get date of sign_in earlier than date of registration
https://dbfiddle.uk/?rdbms=postgres_9.6&fiddle=1e112d51825f5d3185e445d97d4e9c78
select * from (
select ROW_NUMBER() OVER(PARTITION BY user_id ORDER BY date ) as udid,case when action='registration' then 1
when action='sign_in' then 2
when action='delete' then 3
ELSE 4
end as stsord,*
from duptuser
) as drt where stsord!=udid
I need this sql query in a function in my controller in ruby on rails 2.3.
SELECT name, created_at AS "date" FROM table1 WHERE created_at = '2015-07-15' ORDER by created_at;
This is because I have two more queries before this one, and they have a field named 'date', and I need to concatenate every one and sort them by the field named 'date', and just this last one table don't have a field named 'date'.
Ok, I found this solution that helps me, for another person maybe need it..
table1.find_by_sql["SELECT name, created_at AS 'date' FROM table1 WHERE created_at = '2015-07-15' ORDER by created_at"]
Then I can sort all my query results (including the previous tables) by the field 'date'..
The models are Person and Team with a M:1 relationship.
The query that fails:
db.Person.findAll({
attributes: [ [sequelize.fn('COUNT', sequelize.col('*')), 'count']],
include: [{model: db.Team, required: true}], // to force inner join
group: ['team.team_id']
}).complete(function(err, data) {
...
});
The generated SQL is:
SELECT "person"."person_id",
COUNT(*) AS "count",
"team"."team_id" AS "team.team_id",
"team"."team_name" AS "team.team_name",
"team"."team_email" AS "team.team_email",
"team"."team_lead" AS "team.team_lead"
FROM "person" AS "person" INNER JOIN "team" AS "team"
ON "person"."team_id" = "team"."team_id"
GROUP BY "team"."team_id";
Obviously, the person.person_id included in the SELECT clause, screws it up, with Postgres complaining correctly that:
ERROR: column "person.person_id" must appear in the `GROUP BY` clause or be used in an aggregate function
It seems that the attributes option is taken into account since the COUNT appears correctly, but all the rest of the columns in the SELECT clause are added by default.
Is there another way (besides attributes) to explicitly define which columns appear in the SELECT clause or is this a bug?
I'm using Sequelize v2.0.3.
Sequelize will always add the primary key to the selected fields. Currently there is no way to disable that.
Perhaps adding DISTINCT ON as suggested here https://stackoverflow.com/a/19723716/800016 to person_id could fix the issue?
Otherwise, feel free to open an issue on the sequelize bug tracker
If I have an array of IDs from a table (table1) in my database. Is there a way of querying another table (table2) to select all the records where a column equals a value equal to one of the IDs from table1.
My code so far is:
LabQuestion.where("product_id=#{Product.where(:brand_id => brand_id).pluck(:id)}")
In this code, I am trying to retrieve all the Lab Questions which are linked to all the products from a brand. This code does not work, but I've tried to demonstrate my needs.
Assuming you have setup your relations properly, you can use joins to join the two tables and query them like this:
LabQuestion.joins(:product).where(:products => { :brand_id => brand_id })
You can use includes instead of joins as below
LabQuestion.includes(:product).where(:products => { :brand_id => brand_id })
I have Articles that have_many Metrics. I wish to order the Articles by a specific Metric.value when Metric.name = "score". (Metric records various article stats as 'name' and 'value' pairs. An Article can have multiple metrics, and even multiple 'scores', although I'm only interested in ordering by the most recent.)
class Article
has_many :metrics
class Metric
# name :string(255)
# value :decimal(, )
belongs_to :article
I'm struggling to write a scope to do this - any ideas? Something like this?
scope :highest_score, joins(:metrics).order('metrics.value DESC')
.where('metrics.name = "score"')
UPDATE:
An article may have many "scores" stored in the metrics table (as they are calculated weekly/monthly/yearly etc.) but I'm only interested in using the first-found (most recent) "score" for any one article. The Metric model has a default_scope that ensures DESCending ordering.
Fixed typo on quote location for 'metrics.value DESC'.
Talking to my phone-a-friend uber rails hacker, it looks likely I need a raw SQL query for this. Now I'm in way over my head... (I'm using Postgres if that helps.)
Thanks!
UPDATE 2:
Thanks to Erwin's great SQL query suggestion I have a raw SQL query that works:
SELECT a.*
FROM articles a
LEFT JOIN (
SELECT DISTINCT ON (article_id)
article_id, value
FROM metrics m
WHERE name = 'score'
ORDER BY article_id, date_created DESC
) m ON m.article_id = a.id
ORDER BY m.value DESC;
article_list_by_desc_score = ActiveRecord::Base.connection.execute(sql)
Which gives an array of hashes representing article data (but not article objects??).
Follow-up question:
Any way of translating this back into an activerecord query for Rails? (so I can then use it in a scope)
SOLUTION UPDATE:
In case anyone is looking for the final ActiveRecord query - many thanks to Mattherick who helped me in this question. The final working query is:
scope :highest_score, joins(:metrics).where("metrics.name"
=> "score").order("metrics.value desc").group("metrics.article_id",
"articles.id", "metrics.value", "metrics.date_created")
.order("metrics.date_created desc")
Thanks everyone!
The query could work like this:
SELECT a.*
FROM article a
LEFT JOIN (
SELECT DISTINCT ON (article_id)
article_id, value
FROM metrics m
WHERE name = 'score'
ORDER BY article_id, date_created DESC
) m ON m.metrics_id = a.metrics_id
ORDER BY m.value DESC;
First, retrieve the "most recent" value for name = 'score' per article in the subquery m. More explanation for the used technique in this related answer:
Select first row in each GROUP BY group?
You seem to fall victim to a very basic misconception though:
but I'm only interested in using the first-found (most recent) "score"
for any one article. The Metric model has a default_scope that ensures DESCending ordering.
There is no "natural order" in a table. In a SELECT, you need to ORDER BY well defined criteria. For the purpose of this query I am assuming a column metrics.date_created. If you have nothing of the sort, you have no way to define "most recent" and are forced to fall back to an arbitrary pick from multiple qualifying rows:
ORDER BY article_id
This is not reliable. Postgres will pick a row as it choses. May change with any update to the table or any change in the query plan.
Next, LEFT JOIN to the the table article and ORDER BY value. NULL sorts last, so articles without qualifying value go last.
Note: some not-so-smart ORMs (and I am afraid Ruby's ActiveRecord is one of them) use the non-descriptive and non-distinctive id as name for the primary key. You'll have to adapt to your actual column names, which you didn't provide.
Performance
Should be decent. This is a "simple" query as far as Postgres is concerned. A partial multicolumn index on table metrics would make it faster:
CREATE INDEX metrics_some_name_idx ON metrics(article_id, date_created)
WHERE name = 'score';
Columns in this order. In PostgreSQL 9.2+ you could add the column value to make index-only scans possible:
CREATE INDEX metrics_some_name_idx ON metrics(article_id, date_created, value)
WHERE name = 'score';