After reading the following link https://github.com/googleapis/python-bigquery-sqlalchemy I managed to query a table stored on Google BigQuery by using SLQAlchemy. Now I would like to create a SQLAlchemy Users model on Google BigQuery so that I can use Flask-login features (e.g. UserMixin) in order to validate whether a user is authenticated, active, etc. but in this case my database is stored on BigQuery instead of a traditional SQL database (mainly due to billing costs as I find Google Cloud SQL way more expensive than Google BigQuery).
This is my code:
from sqlalchemy.engine import create_engine
from flask_sqlalchemy import SQLAlchemy
engine = create_engine('bigquery://my_project',credentials_path='my_credentials.json')
db = SQLAlchemy()
class Users(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(15), unique=True, nullable = False)
email = db.Column(db.String(50), unique=True)
Users.metadata.create_all(engine)
Unfortunately I'm getting the following error:
DatabaseError: (google.cloud.bigquery.dbapi.exceptions.DatabaseError) 400 Table "users" must be qualified with a dataset (e.g. dataset.table).
I tried to modify the engine variable as follows:
engine = create_engine('bigquery://my_project.my_dataset',credentials_path='my_credentials.json')
but then I get the following error (I'm intrigued by the None part):
ValueError: table_id must be a fully-qualified ID in standard SQL format, e.g., "project.dataset.table_id", got my_project.my_dataset.None.users
Does anyone know how can I create a SQLAlchemy model on Google BigQuery?
Can you try this:
from sqlalchemy.engine import create_engine
from flask_sqlalchemy import SQLAlchemy
from pybigquery.api import ApiClient
from flask import Flask
db = SQLAlchemy()
#ToDo:Change project name and dataset name.
engine = create_engine('bigquery://my-project/my-dataset')
db = SQLAlchemy()
class Users(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(15), unique=True, nullable = False)
email = db.Column(db.String(50), unique=True)
Users.metadata.create_all(engine)
I solved the error by adding the dataset name.
engine = create_engine('bigquery://my-project/my-dataset')
Related
I am new to Django. I am to create some sort of todo app. I am having trouble setting IntegerField as optional field. As far as I can see the problem is when I try to save to save the object to the database. I get error: NOT NULL constraint failed: lista_row.quantity. I have made (and migrated) migrations. Here is my models.py:
from django.db import models
from django.contrib import admin
class Row(models.Model):
name = models.CharField(max_length=200)
quantity = models.IntegerField(null=False, blank=True)
If you want to make it optional, use this instead:
quantity = models.IntegerField(blank=True, null=True)
class BlogPost(db.Model):
users = db.relationship(User)
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('users.id'), nullable=False)
Username= db.Column(db.Text)
Password = db.Column(db.LargeBinary(length=500),nullable=True)
Strength=db.Column(db.Text)
I have these columns in data base table and when I want to retrieve the data :
user = User.query.filter_by(username=current_user.username).first_or_404()
posts=BlogPost.query.filter_by(user_id=current_user.id).all()
When I try to access each value in posts for example post[0], the datatype shown is:
<class 'password.models.BlogPost'>
I am trying to make a password manager and I have encrypted the passwords before storing it into the database and by doing so it changes to base64(bytes) and hence I want it to retrieve it in the datatype in which it was stored,so that I can decrypt it, however when I Try decrypting it it says that token must be in bytes , can someone help me solve this issue.
I have another 'USER' table for storing user data, which is email,password,etc.
When you access post[0], you are accessing the entire post object. Try using post[0].password.
So I'm just trying to make sense of the output of the SQLAlchemy ORM methods after creating a model, committing some entries and running queries. Most queries are fine...I'm getting back a list but for some it just returns an object (see below). I know this sounds obvious but is this normal behavior? I'm specifically referring to the filter_by query as you can see below...
#sample_app.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app=Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI']='...'
db = SQLAlchemy(app)
class Person(db.Model):
__tablename__='persons'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(), nullable=False)
def __repr__(self):
return f'<Person Id: {self.id}, name: {self.name}>'
db.create_all()
#Run some basic commands in interactive mode with model already populated
python3
from sample_app import db,Person
#add a bunch of persons
person1=Person(name='Amy')
person2=...
db.session.add(person1)
db.session.commit()
...
#Run queries
Person.query.all() #returns all persons as a list
Person.query.first() #returns first item in the list
Person.query.filter_by(name='Amy')
#returns <flask_sqlalchemy.Basequery object at 0xsadfjasdfsd>
So why am I not getting the same type of output for the third query for 'Amy'? is that normal behavior for the filter_by method?
Thanks
You didn’t execute the query in the last example. The all method brings back all object selected by the query, first is the first. You’ve specified a filter in the last example, but you didn’t execute a method which processes the query and returns a result [set].
If there are more than one Amy’s, you get all the matches with all() or the first with first(). If you had a filter which should yield a unique record, you could also use .one()
I have these models:
class Person(models.Model):
user = models.OneToOneField(User)
class Post(models.Model):
author = models.ForeignKey(Person, null=False)
text = models.TextField(null=False, blank=False)
and want a Queryset at least with the below fields,
author.user.username
text
I have read select_related() queries but when I try to use that with this view can't get username field
posts = Post.objects.select_related('person__user')[:10]
can I use Django query or have to use SQL raw ?
Thanks for any help
You can serialize like this:
import json
from django.core.serializers.json import DjangoJSONEncoder
json_data = json.dumps(list(Post.objects.values('author__user__username', 'text')[:10]), cls=DjangoJSONEncoder)
select_related should be called with the field names, not the type.
posts = Post.objects.select_related('author__user')[:10]
for post in posts:
print(post.person.user.username)
print(post.text)
All the select_related does is ensure that the foreign fields can be accessed without extra queries (select_related constructs joins to the relevant tables).
I'm trying to figure out how to write a hook to query the database before inserting a row from the ORM. I hope to achieve something similar to this:
class Table(Base):
id = Column(Integer, primary_key=True)
value = Column(Integer, nullable=False)
def before_insert_hook(self, session):
"""Some arbitrary queries and code. For example:"""
if self.value is None:
self.value = session.query(func.avg(Table.value))\
.filter(Table.value > 100).scalar()
I've been reading up in the SQLAlchemy docs about ORM events and such, but I can't figure out how to use them to achieve this.
Looks like you want ORM Events:
from sqlalchemy import event
class Table(Base):
...
#event.listens_for(Table, 'before_insert')
def do_stuff(mapper, connect, target):
# target is an instance of Table
target.value = ...