I'm trying to get rows in a Table.
Imagine I got two records (t1 and t2). I want to get rows that do not have the t1.start_hour BETWEEN t2.start_hour and t2.finish_hour. I basically want only to get the occurrences that don not have conflict in hours with another one.
This is the table:
create_table "occurrences", :force => true do |t|
t.string "start_hour"
t.string "finish_hour"
t.date "start_date"
t.date "finish_date"
t.datetime "created_at", :null => false
t.datetime "updated_at", :null => false
t.integer "activity_id"
end
And this is the SQL query I came up so far:
Occurrence.find_by_sql("SELECT * FROM occurrences t1 INNER JOIN occurrences t2 ON (t1.start_hour NOT BETWEEN t2.start_hour and t2.finish_hour)")
It gives me duplicate results. I'm not be able to remove them and get the correct answer.
Thanks for the help in advance.
Example
INPUT
#<Occurrence id: 1, start_hour: "19:00", finish_hour: "20:20", start_date: "2012-05-30", finish_date: "2012-05-30", created_at: "2012-05-30 09:58:19", updated_at: "2012-05-30 09:58:19", activity_id: 1>,
#<Occurrence id: 2, start_hour: "19:30", finish_hour: "20:10", start_date: "2012-05-30", finish_date: "2012-05-30", created_at: "2012-05-30 09:58:19", updated_at: "2012-05-30 09:58:19", activity_id: 2>,
#<Occurrence id: 3, start_hour: "22:00", finish_hour: "23:20", start_date: "2012-05-30", finish_date: "2012-05-30", created_at: "2012-05-30 09:58:20", updated_at: "2012-05-30 09:58:20", activity_id: 3>
OUTPUT
#<Occurrence id: 1, start_hour: "19:00", finish_hour: "20:20", start_date: "2012-05-30", finish_date: "2012-05-30", created_at: "2012-05-30 09:58:19", updated_at: "2012-05-30 09:58:19", activity_id: 1>,
#<Occurrence id: 3, start_hour: "22:00", finish_hour: "23:20", start_date: "2012-05-30", finish_date: "2012-05-30", created_at: "2012-05-30 09:58:20", updated_at: "2012-05-30 09:58:20", activity_id: 3>
The record with the start_hour = 19:30 does not output because is between 19:00 and 20:20 of another one.
EDIT:
I Got the solution:
Occurrence.find_by_sql("SELECT start_hour FROM occurrences WHERE start_hour NOT IN (SELECT t2.start_hour FROM occurrences t1 INNER JOIN occurrences t2 ON ((t1.activity_id <> t2.activity_id AND t2.start_hour BETWEEN t1.start_hour and t1.finish_hour)))")
Thanks for the help
Let assume there are 3 records in table. (I am taking integer in place of datatime as this will be easier)
id start end
1 1 3
2 4 5
3 7 9
When I will try to find the rows were start is not in between start and end, than for id =1 both first row will be true and will come in result. Similarly for row with id =2 (start=4) both rows will qualify (making third row come twice in result) Same will happen for third row and you will end up with six rows.
Its not very clear what you are trying to achieve here, but putting distinct will remove the duplicate.
EDIT: You might consider putting inner joins on start and finish date.
not tested (from memory)
you need to exlude the record itself --> t1.activity_id <> t2.activity_id
left join or you won't get the good ones
where there is no right side
would need to test this :p
SELECT * FROM occurrences t1
left JOIN occurrences t2
ON (t1.activity_id <> t2.activity_id and t1.start_hour BETWEEN t2.start_hour and t2.finish_hour)
where t2.activity_id is null
Related
I am currently trying to list different assets on a rails application. I want to access the name, and last values of each asset, all of that ordered by marketcap.
here is my database structure
`create_table "asset_values", force: :cascade do |t|
t.integer "date"
t.integer "asset_id"
t.float "price"
t.float "marketcap"
t.float "volume"
t.datetime "created_at", precision: 6, null: false
t.datetime "updated_at", precision: 6, null: false
end
create_table "assets", force: :cascade do |t|
t.string "name"
t.string "symbol"
t.datetime "created_at", precision: 6, null: false
t.datetime "updated_at", precision: 6, null: false
end`
date is in unix time
Here is my model structure corresponding to asset
has_many :asset_value
validates :name, :symbol, uniqueness: true
validates :name, :symbol, presence: true
and here is my model structure corresponding to asset_values
belongs_to :asset, foreign_key: "asset_id"
EXAMPLE:
Assets:
id: 1, name: Bitcoin, symbol: BTC
id: 2, name: Ethereum, symbol: ETH
Asset_values:
id:1, price: 12000, marketcap: 211 000 000 000, volume 100, asset_id: 1, date: 1 598 300 000
id:2, price: 11000, marketcap: 210 000 000 000, volume 110, asset_id: 1, date: 1 598 200 000
id:3, price: 400, marketcap: 42 000 000 000, volume 20, asset_id: 2, date: 1 598 300 000
id:4, price: 420, marketcap: 43 000 000 000, volume 20, asset_id: 2, date: 1 598 200 000
The_table_i_want :
id: 1, name: Bitcoin, symbol: BTC, price: 12000, marketcap: 211 000 000 000, volume 100, date: 1 598 300 000
id: 2, name: Ethereum, symbol: ETH, price: 400, marketcap: 42 000 000 000, volume 20, date: 1 598 300 000
I tried different calls in order to access the data but none of them is working...
here is what i think is best
sql = ("
SELECT a.id, a.name, a.symbol, v.price, v.marketcap, v.volume, v.date
FROM assets AS a
INNER JOIN asset_values AS v ON a.id = v.asset_id
INNER JOIN (
SELECT MAX(date)
FROM asset_values
GROUP BY asset_id
) AS p
ON a.id = p.asset_id
ORDER BY v.marketcap DESC
")
#data = ActiveRecord::Base.connection.execute(sql)
I am using postgresql and rails 6
Let me know if these adjustments to your query produce the results you would expect
SELECT a.id
,a.name
,a.symbol
,v.price
,v.marketcap
,v.volume
,v.date
FROM assets AS a
INNER JOIN asset_values AS v
ON a.id = v.asset_id
INNER JOIN (
SELECT asset_id
,MAX(date) as max_date
FROM asset_values
GROUP BY asset_id
) AS p
ON v.asset_id = p.asset_id
AND v.date = p.max_date
ORDER BY v.marketcap DESC
If you are using postgresql 9.3+ you can use lateral join
SELECT a.id
,a.name
,a.symbol
,v.price
,v.marketcap
,v.volume
,v.date
FROM assets AS a
INNER JOIN LATERAL
( SELECT asset_id
,price
,marketcap
,volume
,date
FROM asset_values
WHERE asset_id=a.id
ORDER BY created_at ASC LIMIT 1 ) v
ON v.asset_id=a.id
ORDER BY v.marketcap DESC
Following command
Fight.last.fight_logs.where(item_id: nil)
generates sql:
Fight Load (0.3ms) SELECT "fights".* FROM "fights" ORDER BY "fights"."id" DESC LIMIT $1 [["LIMIT", 1]]
FightLog Load (0.2ms) SELECT "fight_logs".* FROM "fight_logs" WHERE "fight_logs"."fight_id" = $1 AND "fight_logs"."item_id" IS NULL LIMIT $2 [["fight_id", 27], ["LIMIT", 11]]
and returns:
#<ActiveRecord::AssociationRelation [#<FightLog id: 30, fight_id: 27, attack: 0, block: 0, item_id: nil, user_id: 1, damage: 11.0, created_at: "2017-11-02 16:20:55", updated_at: "2017-11-02 16:20:57">, #<FightLog id: 31, fight_id: 27, attack: 0, block: 0, item_id: nil, user_id: 20, damage: 3.0, created_at: "2017-11-02 16:20:57", updated_at: "2017-11-02 16:20:57">, #<FightLog id: 33, fight_id: 27, attack: 0, block: 0, item_id: nil, user_id: 1, damage: 1.0, created_at: "2017-11-02 16:21:40", updated_at: "2017-11-02 16:21:40">, #<FightLog id: 32, fight_id: 27, attack: 0, block: 0, item_id: nil, user_id: 20, damage: 7.0, created_at: "2017-11-02 16:21:33", updated_at: "2017-11-02 16:21:40">, #<FightLog id: 34, fight_id: 27, attack: 0, block: 0, item_id: nil, user_id: 1, damage: 12.0, created_at: "2017-11-02 16:21:47", updated_at: "2017-11-02 16:21:48">, #<FightLog id: 35, fight_id: 27, attack: 0, block: 0, item_id: nil, user_id: 20, damage: 14.0, created_at: "2017-11-02 16:21:48", updated_at: "2017-11-02 16:21:48">]>
but
Fight.last.fight_logs.where.not(item_id: 1)
generates sql:
Fight Load (1.0ms) SELECT "fights".* FROM "fights" ORDER BY "fights"."id" DESC LIMIT $1 [["LIMIT", 1]]
FightLog Load (0.8ms) SELECT "fight_logs".* FROM "fight_logs" WHERE "fight_logs"."fight_id" = $1 AND ("fight_logs"."item_id" != $2) LIMIT $3 [["fight_id", 27], ["item_id", 1], ["LIMIT", 11]]
and returns:
#<ActiveRecord::AssociationRelation []>
How it is possible? What i'm doing wrong?
You should specify NULL value in your query since you have it in your database:
Fight.last.fight_logs.where('item_id != ? OR item_id IS NULL', 1)
This is just how SQL works:
select 1 != NULL;
+-----------+
| 1 != NULL |
+-----------+
| NULL |
+-----------+
You can look at this answer to clarify the issue.
Also, I would recommend avoiding using default NULL values in your database, there is nice answer about it. You can simply use default: 0, null: false your case.
I am getting an error when running a query in SQLite vs Postgres with Active Record. There is an aggregate function error in Postgres but nothing in SQLite. My question is, what is SQLite doing here? Is it ignoring the group? Notice that GROUP BY "song" doesn't return an error.
SQLite output:
Playlist.where(user_id:user.id).group(:song)
Playlist Load (0.3ms) SELECT "playlists".* FROM "playlists" WHERE
"playlists"."user_id" = ? GROUP BY "song" [["user_id", 1]]
=> #<ActiveRecord::Relation [#<Playlist id: 2, user_id: 1, song_id: 1,
created_at: "2017-04-27 01:18:01", updated_at: "2017-04-27 01:18:01">]>
Playlist.where(user_id:user.id).group(:song_id)
Playlist Load (0.4ms) SELECT "playlists"."id", "playlists"."user_id",
"playlists"."song_id" FROM "playlists" WHERE "playlists"."user_id" = ?
GROUP BY "song" [["user_id", 1]]
=> #<ActiveRecord::Relation [#<Playlist id: 2, user_id: 1, song_id:
1, created_at: "2017-04-27 01:18:01">]>
Playlist.where(user_id:user.id).group(:id,:song_id)
Playlist Load (0.2ms) SELECT "playlists".* FROM "playlists" WHERE
"playlists"."user_id" = ? GROUP BY "playlists"."id",
"playlists"."song_id" [["user_id", 1]]
=> #<ActiveRecord::Relation [#<Playlist id: 1, user_id: 1, song_id: 1,
created_at: "2017-04-27 01:18:00", updated_at: "2017-04-27 01:18:00">,
#<Playlist id: 2, user_id: 1, song_id: 1, created_at: "2017-04-27
01:18:01", updated_at: "2017-04-27 01:18:01">]>
Playlist.where(user_id:user.id).group(:song).count
(0.2ms) SELECT COUNT(*) AS count_all, "playlists"."song_id" AS
playlists_song_id FROM "playlists" WHERE "playlists"."user_id" = ?
GROUP BY "playlists"."song_id" [["user_id", 1]]
Song Load (2.1ms) SELECT "songs".* FROM "songs"
WHERE "songs"."id" = ? LIMIT 1 [["id", 1]]
=> {#<Song id: 1, title: "A song", artist: "Artist", user_id: 1,
created_at: "2017-04-27 01:17:39">=>2}
Postgres:
Playlist.where(user_id:user.id).group(:song_id)
ActiveRecord::StatementInvalid: PG::GroupingError: ERROR: column
"playlists.id" must appear in the GROUP BY clause or be used in an
aggregate function
SELECT "playlists".* FROM "playlists" WHERE "playlists"."user_id" = $1
GROUP BY "playlists"."song_id"
Playlist.where(user_id:user.id).group(:id,:song_id)
Playlist Load (0.6ms) SELECT "playlists".* FROM "playlists" WHERE
"playlists"."user_id" = $1 GROUP BY "playlists"."id",
"playlists"."song_id" [["user_id", 1]]
=> #<ActiveRecord::Relation [#<Playlist id: 1, user_id: 1, song_id: 1,
created_at: "2017-04-27 01:25:34", updated_at: "2017-04-27 01:25:34">,
#<Playlist id: 2, user_id: 1, song_id: 1, created_at: "2017-04-27
01:25:36", updated_at: "2017-04-27 01:25:36">]>
Playlist.where(user_id:user.id).group(:song).count
(0.5ms) SELECT COUNT(*) AS count_all, "playlists"."song_id" AS
playlists_song_id FROM "playlists" WHERE "playlists"."user_id" = $1
GROUP BY "playlists"."song_id" [["user_id", 1]]
Song Load (0.3ms) SELECT "songs".* FROM "songs" WHERE "songs"."id" =
$1 LIMIT 1 [["id", 1]]
=> {#<Song id: 1, title: "A song", artist: "Artist", user_id: 1,
created_at: "2017-04-27 01:25:26", updated_at: "2017-04-27
01:25:26">=>2}
A common rule for group by: you may select only columns listed in group by and aggregate functions. This approach supported by sql standard and will work in any sql database.
If you are selecting columns not listed in a group by clause, for example select * … group by song_id, it may work in some databases and will not work in others.
If you specify selected columns in your code it will work:
Playlist.
where(user_id:user.id).
select("song_id").
group(:song_id)
You may read details about group by in PostgreSQL documentation: https://www.postgresql.org/docs/current/static/sql-select.html#SQL-GROUPBY
Let's assume such table content where for the same product_id, we have as many rows than updates during status==1 (published) and finally status==0 (unpublished) and then becomes==2 (deleted)
{id: <auto>, product_id: 1, last_updated: 2015-12-1, status: 1, price: 1}
{id: <auto>, product_id: 2, last_updated: 2015-12-1, status: 1, price: 10}
{id: <auto>, product_id: 1, last_updated: 2015-12-2, status: 1, price: 2}
{id: <auto>, product_id: 1, last_updated: 2015-12-3, status: 0, price: 2}
{id: <auto>, product_id: 2, last_updated: 2015-12-2, status: 0, price: 10}
{id: <auto>, product_id: 3, last_updated: 2015-12-2, status: 1, price: 123}
{id: <auto>, product_id: 1, last_updated: 2015-12-4, status: 2, price: 2}
{id: <auto>, product_id: 2, last_updated: 2015-12-4, status: 2, price: 10}
Now, I am trying to find a way, maybe using a secondary compound index, do get for example, given a date like in col1 (using r.time)
DATE STATUS==1 STATUS==0 STATUS==2
2015-12-1 [101, 102] [] []
2015-12-2 [103, 106] [105] []
2015-12-3 [106] [104, 105] []
2015-12-4 [] [] [107, 108]
The difficulty here, is that a product_id document is still to be considered as the most recent status as long as its last_updated date is less or equal to the provided date.
I try by grouping by product_id, then take the max('last_updated'), then only keep each reduction unique document if status==1
I have in mind to have an index for each status / given_date
Or another solution, would be to insert in another table the result of an aggregation which would only store a unique document per date, containing all the initial documents ids matching the same criteria, and so on...
And then later perform joins using these intermediate records to fetch the values of each product_id at the given date/status.
something like:
{
date: <date_object>,
documents: [
{id: document_id, status: 1},
{id: document_id, status: 1},
{id: document_id, status: 2},
{id: document_id, status: 0},
...
]
}
Please advise
Edit 1:
This is an example of a query I try to run to analyse my data, here it is for example to get an overview of the statuses for each group with more than 1 document:
r.db('test').table('products_10k_sample')
.group({index: 'product_id'})
.orderBy(r.desc('last_updated'))
.ungroup()
.map(function(x){
return r.branch(
x('reduction').count().gt(1),
x('reduction').map(function(m){
return [m('last_updated').toISO8601(), m('status'), m('product_id')]
}),
null
)
})
the result produced by
locations_by_geocoder = Location.near([params[:latitude], params[:longitude]], params[:radius], :units => :km)
and
locations_by_where = Location.where(:id => [7, 10, 9, 8])
are the same
location_by_geocoder output is
[#<Location id: 7, country: nil, state: nil, city: nil, locality: nil, name: "Coffee On Canvas", latitude: "12.93516201597191", longitude: "77.63097871416265", created_at: "2014-03-29 15:20:34", updated_at: "2014-03-29 15:20:34", address: "#84, S.T.Bed Layout, 4th Block Koramangala", id_location: "cd49dfa966245b12">,
#<Location id: 10, country: nil, state: nil, city: nil, locality: nil, name: nil, latitude: "12.93516201597191", longitude: "77.63097871416265", created_at: "2014-03-30 14:47:03", updated_at: "2014-03-30 15:20:52", address: nil, id_location: "60f7f024d37c74f9">,
#<Location id: 9, country: nil, state: nil, city: nil, locality: nil, name: "Cafe Coffee Day", latitude: "12.93434671368444", longitude: "77.6256083701297", created_at: "2014-03-29 15:41:42", updated_at: "2014-03-29 15:41:42", address: "Mango Suites,", id_location: "8db810b6762fd784">,
#<Location id: 8, country: nil, state: nil, city: nil, locality: nil, name: "Cafe Coffee Day", latitude: "12.932626125142486", longitude: "77.62338638305664", created_at: "2014-03-29 15:31:53", updated_at: "2014-03-29 15:31:53", address: "BPCL RAJAJINAGAR,Dr.RajkumarRoad, Rajajinagar", id_location: "ff71f5a0b4de9e1b">]
location_by_where output is
[#<Location id: 7, country: nil, state: nil, city: nil, locality: nil, name: "Coffee On Canvas", latitude: "12.93516201597191", longitude: "77.63097871416265", created_at: "2014-03-29 15:20:34", updated_at: "2014-03-29 15:20:34", address: "#84, S.T.Bed Layout, 4th Block Koramangala", id_location: "cd49dfa966245b12">,
#<Location id: 8, country: nil, state: nil, city: nil, locality: nil, name: "Cafe Coffee Day", latitude: "12.932626125142486", longitude: "77.62338638305664", created_at: "2014-03-29 15:31:53", updated_at: "2014-03-29 15:31:53", address: "BPCL RAJAJINAGAR,Dr.RajkumarRoad, Rajajinagar", id_location: "ff71f5a0b4de9e1b">,
#<Location id: 9, country: nil, state: nil, city: nil, locality: nil, name: "Cafe Coffee Day", latitude: "12.93434671368444", longitude: "77.6256083701297", created_at: "2014-03-29 15:41:42", updated_at: "2014-03-29 15:41:42", address: "Mango Suites,", id_location: "8db810b6762fd784">,
#<Location id: 10, country: nil, state: nil, city: nil, locality: nil, name: nil, latitude: "12.93516201597191", longitude: "77.63097871416265", created_at: "2014-03-30 14:47:03", updated_at: "2014-03-30 15:20:52", address: nil, id_location: "60f7f024d37c74f9">]
but while using it in a join both produces different results
books = books.joins(:location).merge(locations_by_geocoder)
produces
[#<Book id: 10, created_at: "2014-03-30 14:47:03", updated_at: "2014-03-30 15:20:52">]
query syntax is
SELECT locations.*, 6371.0 * 2 * ASIN(SQRT(POWER(SIN((12.9398981 - locations.latitude) * PI() / 180 / 2), 2) + COS(12.9398981 * PI() / 180) * COS(locations.latitude * PI() / 180) * POWER(SIN((77.6275559 - locations.longitude) * PI() / 180 / 2), 2))) AS distance, CAST(DEGREES(ATAN2( RADIANS(locations.longitude - 77.6275559), RADIANS(locations.latitude - 12.9398981))) + 360 AS decimal) % 360 AS bearing FROM `books` INNER JOIN `locations` ON `locations`.`id` = `books`.`location_id` WHERE (locations.latitude BETWEEN 12.930904883940814 AND 12.948891316059187 AND locations.longitude BETWEEN 77.61832835502157 AND 77.63678344497843 AND 6371.0 * 2 * ASIN(SQRT(POWER(SIN((12.9398981 - locations.latitude) * PI() / 180 / 2), 2) + COS(12.9398981 * PI() / 180) * COS(locations.latitude * PI() / 180) * POWER(SIN((77.6275559 - locations.longitude) * PI() / 180 / 2), 2))) <= '1') ORDER BY distance ASC
where as
books = books.joins(:location).merge(locations_by_where)
produces
[#<Book id: 11, title: "ggn", author: "steven jobs,steve jobs", isbn_10: "1572846933", isbn_13: "1932841660", edition: "1", print: nil, publication_year: 2011, publication_month: "2011", condition: "", value: nil, status: nil, stage: nil, description: "", visits: 38, user_id: 5, prefered_place: nil, prefered_time: nil, created_at: "2014-01-01 08:13:00", updated_at: "2014-03-30 14:47:35", rating: nil, image: nil, publisher: nil, goodreads_id: nil, ext_image_url: nil, pages: nil, language_code: nil, barter_type: nil, location_id: 10, id_book: nil>]
and query syntax is
SELECT `books`.* FROM `books` INNER JOIN `locations` ON `locations`.`id` = `books`.`location_id` WHERE `locations`.`id` IN (7, 10, 9, 8)
even though the output of locations_by_where and locations_by_geocoder are same
both are ActiveRecord::Relation::ActiveRecord_Relation_Location
the issue is connected to near scope in locations.near
https://github.com/alexreisner/geocoder/issues/627#issuecomment-39048249