I need to convert this SQL statement to an SQLAlchemy ORM query:
SELECT zone, count(fall_status)
FROM events
WHERE events.timestamp BETWEEN 1663460366 AND 1663546766
AND events.fall_status = 1
GROUP by zone
First you need to create an engine:
from sqlalchemy import create_engine
engine = engine = create_engine('sqlite:///...')
Then you can use
from sqlalchemy.sql import text
with engine.connect() as conn:
statement = text("SELECT zone, count(fall_status) from events WHERE events.timestamp BETWEEN 1663460366 AND 1663546766 AND events.fall_status = 1 GROUP by zone")
conn.execute(statement)
q = (
db.query(
Events.zone,
func.count(Events.fall_status),
)
.filter(Events.fall_status == 1)
.filter(Events.timestamp.between(to_date,from_date))
.group_by(Events.zone)
).all()
see func.count
Related
My bigquery query is :
SELECT d.type AS `error_type`, count('d.type') AS `count`
FROM `table_android`, unnest(`table_android`.`exceptions`) AS `d`
WHERE `table_android`.`event_timestamp` BETWEEN '2022-12-15' AND '2022-12-20' GROUP BY `error_type` ORDER BY `count` desc;
This query is working fine in bigquery editor. But same version of query with sqlalchemy I could not get same results.
sqlalchemy query :
sa.select(
sa.literal_column("d.type").label("Error_Type"),
sa.func.count("Error_Type").label("count"),
)
.select_from(sa.func.unnest(table_android.c.exceptions).alias("d"))
.group_by("Error_Type")
.order_by(desc("count"))
.where(table_android.c.event_timestamp.between('2022-12-15', '2022-12-20'))
.cte("main_table")
Correct result :
Wrong result:
I am using python-bigquery-sqlalchemy library. table_android.exceptions column struct is like that :
column types :
And this is render of sqlalchemy query :
SELECT `d`.`type` AS `Error_Type`, count(`d`.`type`) AS `count` FROM `table_android`, unnest(`table_android`.`exceptions`) AS `d` WHERE `table_android`.`event_timestamp` BETWEEN '2022-12-05' AND '2022-12-20' GROUP BY `Error_Type` ORDER BY `count` DESC
I see correct result in bigquery editor. But sqlalchemy is not shows correct result. How should i edit my sqlalchemy query for correct results ?
I don't have bigquery so I can't test this but I think you want column and not literal_column. Also I think you can create an implicit CROSS JOIN by including both the table and the unnested column in the select_from.
# I'm testing with postgresql but it doesn't have easy to make structs
from sqlalchemy.dialects import postgresql
table_android = Table("table_android", metadata,
Column("id", Integer, primary_key=True),
Column("exceptions", postgresql.ARRAY(String)),
Column("event_timestamp", Date))
from sqlalchemy.sql import column, func, select
d = func.unnest(table_android.c.exceptions).alias("d")
# You might be able to do d.column["type"].label("Error_Type")
type_col = column("d.type").label("Error_Type")
count_col = func.count(type_col).label("count")
print (select(
type_col,
count_col,
)
.select_from(table_android, d)
.group_by(type_col)
.order_by(count_col)
.where(table_android.c.event_timestamp.between('2022-12-15', '2022-12-20'))
.cte("main_table"))
I have a SQL query and when writing in Django ORM it returns an error. But the SQL query works perfectly on MySQL Command Line Client. Would anyone please explain the error or working of CASE and When in Django ORM?
SQL query:-
SELECT CASE WHEN LENGTH(au.first_name) < 1 THEN au.username ELSE concat(au.first_name,' ',au.last_name)
END AS fullname FROM rewards_usercard ru RIGHT JOIN auth_user au ON ru.user_id = au.id;
Django Models code:-
from django.db.models import Case, When, Q, CharField, Value
from django.db.models.functions import Length, Concat
from django.db.models.lookups import LessThan
queryset = UserCard.objects.annotate(
full_name = Case(
When(condition=Q(
LessThan(Length('user__first_name'),1)
), then='user__username'),
default = Concat('user__first_name', Value(' '), 'user__last_name'),
output_field=CharField()
)
)
Error:-
cannot unpack non-iterable LessThan object
Try this:
from django.db.models.functions import Coalesce, Concat
from django.db.models import (Case, CharField, When, Value)
data = UserCard.objects.all().annotate(fullname=Coalesce(Case(When(user__first_name__in=[None, ""], then="user__username"),default=Concat('user__first_name',Value(" "), 'user__last_name'),output_field=CharField()),"user__username")).values('fullname')
Trying to implement pandas.read_sql function.
I created a clickhouse table and filled it:
create table regions
(
date DateTime Default now(),
region String
)
engine = MergeTree()
PARTITION BY toYYYYMM(date)
ORDER BY tuple()
SETTINGS index_granularity = 8192;
insert into regions (region) values ('Asia'), ('Europe')
Then python code:
import pandas as pd
from sqlalchemy import create_engine
uri = 'clickhouse://default:#localhost/default'
engine = create_engine(uri)
query = 'select * from regions'
pd.read_sql(query, engine)
As the result I expected to get a dataframe with columns date and region but all I get is empty dataframe:
Empty DataFrame
Columns: [2021-01-08 09:24:33, Asia]
Index: []
UPD. It occured that defining clickhouse+native solves the problem.
Can it be solved without +native?
There is encient issue https://github.com/xzkostyan/clickhouse-sqlalchemy/issues/10. Also there is a hint which assumes to add FORMAT TabSeparatedWithNamesAndTypes at the end of a query. So the init query will be look like this:
select *
from regions
FORMAT TabSeparatedWithNamesAndTypes
I'm trying to extract keywords from the SQL database and CONTAINS and LIKE are not worrking.
import pandas as pd
from sqlalchemy import create_engine
engine = create_engine('sqlite://', echo=False)
df = pd.read_csv('data.csv')
sql = df.to_sql('table1', con=engine,index=True)
q1 = engine.execute('SELECT * FROM table1 WHERE features LIKE "Swimming" ').fetchall()
q2 = engine.execute('SELECT * FROM table1 WHERE CONTAINS(features, "Swimming") ').fetchall()
I've tred both the ways but i'm not getting the answer. And i get this error OperationalError: (sqlite3.OperationalError) no such function: CONTAINS [SQL: 'SELECT * FROM table1 WHERE CONTAINS(features, "Swimming") '] (Background on this error at: http://sqlalche.me/e/e3q8)
You need to use double quote around your SQL statement and single quotes inside that.
q1 = engine.execute("SELECT * FROM table1 WHERE features LIKE '%Swimming%' ").fetchall()
You might want to change 'Swimming' to '%Swimming%' so that it can be matched to any value that has Swimming in it.
I want to convert a resultproxy object into a subquery for use with an ORM query. What's the procedure to do this? I can't really find any examples.
A result proxy is something that you get when a SELECT statement is passed to the database and executed - at that point, there's no more "query" so in a literal sense, a result proxy isn't a subquery. However, that SELECT statement which you have executed, can instead not be executed and passed into another query as a subquery pretty easily:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class A(Base):
__tablename__ = 'a'
id = Column(Integer, primary_key=True)
data = Column(String)
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
sess = Session(e)
# test data.
sess.add_all([A(data="d%d" % i) for i in range(10)])
sess.commit()
# a query.
subq = sess.query(A.id).filter(A.data.in_(["d3", "d4", "d5"]))
# execute subquery, gives us a list of tuples
list_of_tuples = subq.all()
# execute subquery via Core, gives us a ResultProxy
result_proxy = sess.execute(subq)
# inject subq into another Query as a subquery()
q = sess.query(A).filter(A.data.in_(subq.subquery()))
a_from_subquery = q.all()