SQLAlchemy create multi column index using naming_convention - indexing

I am trying to create multi (i.e. 2-column index) for a model based table. But I'd like not to give specific name for this index. I'd like that naming_convention and alembic revision --autogenerate would do it's job with naming index. So far I have code like this:
from sqlalchemy import MetaData
from sqlalchemy.ext.declarative import as_declarative
from sqlalchemy.schema import Index
metadata = MetaData(
naming_convention={
'pk': '%(table_name)s_pk',
'ix': '%(table_name)s_%(column_0_N_name)s_ix',
},
)
#as_declarative(metadata=metadata)
class Base:
pass
class Foo(Base):
id = Column(Integer, primary_key=True)
col1 = Column('Col1', Integer)
col2 = Column('Col2', DateTime)
Index(
metadata.naming_convention['ix'] % {
'table_name': Foo.__tablename__,
'column_0_N_name': Foo.col1.expression.name + "_" + Foo.col2.expression.name
},
Foo.col1,
Foo.col2,
)
So I'd like to avoid the 'creating name' part of code:
metadata.naming_convention['ix'] % {
'table_name': Foo.__tablename__,
'column_0_N_name': Foo.col1.expression.name + "_" + Foo.col2.expression.name
}

after more search there is very simple solution. According to github comment in SQLAlchemy issue if you would like to create index by Index() you silmply need to pass name=None argument, and fill arguments for columns.
so the code above should look like (part that stays the same):
from sqlalchemy import MetaData
from sqlalchemy.ext.declarative import as_declarative
from sqlalchemy.schema import Index
metadata = MetaData(
naming_convention={
'pk': '%(table_name)s_pk',
'ix': '%(table_name)s_%(column_0_N_name)s_ix',
},
)
#as_declarative(metadata=metadata)
class Base:
pass
Option 1 (declare multi column index out of table model)
class Foo(Base):
id = Column(Integer, primary_key=True)
col1 = Column('Col1', Integer)
col2 = Column('Col2', DateTime)
Index(None, Foo.col1, Foo.col2)
Option 2 (declare multi column index inside of table model)
class Foo(Base):
id = Column(Integer, primary_key=True)
col1 = Column('Col1', Integer)
col2 = Column('Col2', DateTime)
__table_args__ = (
Index(None, 'Col1', 'Col2'),
)
Then index name (in both options) would be:
Foo_Col1_Col2_ix
I have not found this kind of solution in SQLALchemy documentation (maybe there is?), but it's good that there are some answers on github issues in SQLAlchemy github :)

Related

Kotlin, Pre-packaged database has an invalid schema: Column order is wrong?

I exported a copy of the database from the emulator and moved the test database to an external file. This has been working well for sometime, however, today something changed and this error appeared.
Pre-packaged database has an invalid schema: tableLinkUserToPassword
Expected:
TableInfo
{
name='tableLinkUserToPassword',
columns = {
userId = Column { name='userId', type='INTEGER', affinity='3', notNull=true, primaryKeyPosition=1, defaultValue='null' },
password = Column { name='password', type='TEXT', affinity='2', notNull=true, primaryKeyPosition=0, defaultValue='null' }
},
foreignKeys = [ForeignKey
{
referenceTable='tableUser', onDelete='CASCADE', onUpdate='NO ACTION', columnNames=[userId], referenceColumnNames=[userId]
}
],
indices = [Index { name='index_tableLinkUserToPassword_password', unique=false, columns=[password], orders=[ASC] },
Index { name='index_tableLinkUserToPassword_userId', unique=false, columns=[userId], orders=[ASC]}
]
}
Found:
TableInfo{name='tableLinkUserToPassword', columns={password=Column{name='password', type='TEXT', affinity='2', notNull=false, primaryKeyPosition=0, defaultValue='null'}, userId=Column{name='userId', type='INTEGER', affinity='3', notNull=false, primaryKeyPosition=1, defaultValue='null'}}, foreignKeys=[ForeignKey{referenceTable='tableUser', onDelete='CASCADE', onUpdate='NO ACTION', columnNames=[userId], referenceColumnNames=[userId]}], indices=[Index{name='index_tableLinkUserToPassword_userId', unique=false, columns=[userId], orders=[ASC]}, Index{name='index_tableLinkUserToPassword_password', unique=false, columns=[password], orders=[ASC]}]}
The UserId and the Password columns are switched in order. And notNull=false seems not to match.
I have DB Browser but how would I change the order or columns?
Entity:
import androidx.room.ColumnInfo
import androidx.room.Entity
import androidx.room.ForeignKey
import androidx.room.PrimaryKey
#Entity(
tableName = "tableLinkUserToPassword",
foreignKeys = [
ForeignKey(
entity = EntityUser::class,
parentColumns = ["userId"],
childColumns = ["userId"],
onDelete = ForeignKey.CASCADE
)
]
)
data class EntityLinkUserToPassword(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(index = true)
val userId: Int,
val password: String,
)
Thanks for any assistance
The order in which the columns appear is not an issue, it's the order in which they are extracted (I think). It is the reported values that matter.
However, what can be seen, as an example, is that the password column EXPECTED (what is extracted from the #Entity annotated class) has the NOT NULL constraint (i.e. it must not be null) as per notNull=true, whilst it found that there is no NOT NULL constraint coded for the password column in the FOUND (i.e. the pre-packaged database) as per notNull=false.
So you either have to change the EntityLinkUserToPassword class to allow null, or change the pre-packaged database to have NOT NULL coded on the password column.
e.g. val password: String?,
You need to check ALL columns for discrepancies between the found and expected.
P.S. a second index on the userId column is a waste and inefficient. The PrimaryKey is an index. So there is no need for the #ColumnInfo annotation.
However, again there is another discrepancy the second index on the pre-packaged database is on the password column. So you should have the #ColumnInfo annotation moved to apply to the password val/column e.g. I believe that you want :-
data class EntityLinkUserToPassword(
#PrimaryKey(autoGenerate = true)
val userId: Int,
#ColumnInfo(index = true)
val password: String?,
)
Note the above is based upon observation, the suggested code has not been tested and is not necessarily full comprehensive, so may contain omissions and or errors.
I have DB Broswer but how would I chnage the order or columns?
You would/could :-
Rename the EntityLinkUserToPassword table e.g. ALTER TABLE EntityLinkUserToPassword RENAME TO renamed_EntityLinkUserToPassword;
Use UPDATE renamed_EntityLinkUserToPassword SET password = 'a suitable default value' WHERE password IS NULL;
This so that you don't get NOT NULL constraint conflicts when copying the data.
Create the new table with the correct schema (see later)
Use INSERT INTO EntityLinkUserToPassword SELECT * FROM renamed_EntityLinkUserToPassword ORDER BY userId ASC;
DROP TABLE IF EXISTS renamed_EntityLinkUserToPassword'
Getting the correct schema
With the classes annotated with #Entity coded as required AND defined in the entities parameter of the #Database annotation, compile the project.
using the Android View in Android Studio look at the Java(generated) for a class the same name as the #Database annotated class but suffixed with _Impl.
Look for the createAllTables method. The SQL for the creation of the tables is hard coded. Copy it and this will be the EXACT SQL.

Marshmallow - Sort field values by declared order

I've read the docs and searched this site but cannot seem to find a solution to sorting field values by the order in which they are declared. The docs state that adding ordered = True to the class Meta will solve this problem -
class MySchema(Schema):
class Meta:
ordered = True
However, I am not using class Meta in my schema. My schema simply looks like -
class MySchema(Schema):
id = fields.Integer()
name = fields.Str()
category = fields.Str()
So in this situation, how and where would I set ordered = True? Thanks!
I solved the issue by changing my schema class to -
class MySchema(Schema):
class Meta:
ordered = True
id = fields.Integer()
name = fields.Str()
category = fields.Str()
and then also adding JSON_SORT_KEYS=False to my app's config.py file.

How to check if value is already exist in database (SQLAlchemy)

I am trying to check if unique element is already present in postgresdb.
My method in views.py is
def bestfriend(username):
print username
user = Users.query.filter_by(username = username).first()
if user == None:
flash('bestfriend not found.')
return redirect(url_for('index'))
print user
u = g.user.friend(user)
#print bestfriend.id
if u is None:
#flash('Cannot be friend %(username)s.', username = username)
return redirect(url_for('user', username = username))
if db.session.query(bestfriend).filter(bestfriend.id==u.id).first():
flash('Already Exist')
return redirect(url_for('index'))
db.session.add(u)
db.session.commit()
flash('Your bestfriend has been added.')
return redirect(url_for('user', username = username))
My model.py is
bestfriend= db.Table('bestfriend',
db.Column('id',db.Integer, primary_key = True),
db.Column('friendid', db.Integer, db.ForeignKey('users.id'))
)
class Users(db.Model):
__tablename__ = "users"
id = db.Column(db.Integer, primary_key = True)
username = db.Column('username', db.String(20), unique=True , index=True)
password = db.Column('password' , db.String(10))
email = db.Column('email',db.String(50),unique=True , index=True)
registered_on = db.Column('registered_on' , db.DateTime)
posts = db.relationship('Post', backref = 'author', lazy = 'dynamic')
followed = db.relationship('Users',
secondary = followers,
primaryjoin = (followers.c.follower_id == id),
secondaryjoin = (followers.c.followed_id == id),
backref = db.backref('followers', lazy = 'dynamic'),
lazy = 'dynamic')
bestfriends = db.relationship('Users',
secondary = bestfriend,
primaryjoin = (bestfriend.c.friendid == id),
secondaryjoin = (bestfriend.c.id == id),
backref = db.backref('bestfriend', lazy = 'dynamic'),
lazy = 'dynamic')
I am able to insert value in database table bestfriend
Table "public.bestfriend"
Column | Type | Modifiers
----------+---------+-----------
id | integer | not null
friendid | integer |
Indexes:
"bestfriend_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
"bestfriend_friendid_fkey" FOREIGN KEY (friendid) REFERENCES users(id)
but after insertion, I have to check if record is already present, using this line
if db.session.query(bestfriend).filter(bestfriend.id==u.id).first():
I am not sure about this statement. I have already tried similar solutions How to elegantly check the existence of an object/instance/variable and simultaneously assign it to variable if it exists in python? but it didn't worked for me.
I didn't receive any reply so adding some more info. I just want to check if id = 8 is already present in bestfriend table
app=> select * from bestfriend;
id | friendid
----+----------
8 | 11
Your statement does not work because bestfriend is a Table, not a mapped class. To fix the typo, just add c. to the query:
exists = db.session.query(bestfriend).filter(bestfriend.c.id==u.id).first()
Given that your relationship is defined as dynamic, you can do it easier with the following:
exists = user.bestfriends.filter(Users.id == u.id).one()
Side remark: Note that your bestfriend table is not structured correctly to allow many-to-many relationship. In order to do that, please change the definition of the table as per below:
bestfriend = db.Table('bestfriend',
db.Column('id',db.Integer, db.ForeignKey('users.id'), primary_key = True),
db.Column('friendid', db.Integer, db.ForeignKey('users.id'), primary_key = True)
)
Another remark: I do not understand the code flow: you add a friend at step u = g.user.friend(user), but later check if it exists. Shall logic not be reversed?
If you want to find if a record exists in a table, You can do it like this:
select 1 from table_name;
or better
select 1 from table_name where rownum=1;
The image below shows the query with output.
If there are any data, it will return 1 in the column.

Django auth.models.User query by full name but Chinese name

In Chinese, the name format is 'LastnameFirstname' (number of characters for both Lastname and Firstname are varied), while in English, it is 'Firstname LastName'. We can see that in Chinese, the first name last name is swapped (which is not a problem in query here), and the first name last name is NOT separated by whitespace (which caused me this problem).
In SQL, we can do this:
SELECT *
FROM USER
WHERE Concat(last_name, first_name) = 'LastnameFirstName';
But how can I do this in Django? Given a FULLNAME string as 'LastnameFirstname', how can I do:
User.objects.filter(last_name+firstname=FULLNAME)
Another solution is to create a custom User model and create a new field called "full name", but this solution disables me to use other django built-in functions:
class User( models.Model ):
first_name = models.CharField( max_length=64 )
last_name = models.CharField( max_length=64 )
full_name = models.CharField( max_length=128 )
def save( self, *args, **kw ):
self.full_name = '{0}{1}'.format( first_name, last_name )
super( User, self ).save( *args, **kw )
I guess there would be a better solution.
Thanks. :)
You can either link to the User model and create your own class:
class UserProfile(models.Model):
user = models.OneToOneField(settings.AUTH_USER_MODEL)
def full_name(self):
return self.user.last_name + self.user.first_name
....
or you could subclass the AbstractUser model to define your own custom user attributes there:
from django.contrib.auth.models import AbstractUser
class MyUser(AbstractUser):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
def full_name(self):
return self.last_name + self.first_name
Then in your settings set the default user model to your new class:
AUTH_USER_MODEL = 'myapp.MyUser'
The AbstractUser model only has 3 attributes: password,last_login and is_active. You can define the rest yourself.

sqlalchemy query from SQL with multiple where conditions

I have three tables which are defined as:
class User(Base):
__tablename__ = 'users'
id = Column(Integer(10), primary_key=True)
firstname = Column(String(64))
surname = Column(String(64))
class SWMS(Base):
__tablename__ = 'swms'
id = Column(Integer(10), primary_key=True)
owner_id = Column(Integer(10), ForeignKey('users.id', ondelete='CASCADE'))
filename = Column(String(255))
swmowner = relationship('User', backref=backref('users'))
class SWM_perms(Base):
__tablename__ = 'swm_perms'
id = Column(Integer(10), primary_key=True)
swm_id = Column(Integer(10), ForeignKey('swms.id', ondelete='CASCADE'))
user_id = Column(Integer(10), ForeignKey('users.id', ondelete='CASCADE'))
swm = relationship('SWMS', backref=backref('swms'))
swmuser = relationship('User', backref=backref('swmusers'))
Essentially, the SWMS table is a table of document info where the owner_id defines the user who created the document. SWM_perms is a table that has a mapping of document id to user id - to define which users are allowed to see the document.
To produce a table of all documents which are either 1) owned by the user or 2) are viewable by the user, in SQL I would do:
select owner_id, users.firstname, users.surname, filename
from swms, swm_perms, users
where users.id=swms.owner_id and
((swms.id=swm_perms.swm_id and swm_perms.user_id = 27) or (owner_id = 27));
How would you define this query in sqlalchemy? I am familiar with the or_() function but the variants I am trying do not generate the correct objects.
cond1 = and_(SWMS.id==SWM_perms.swm_id,SWM_perms.user_id==27)
swms = DBSession.query(User,SWMS).filter(or_(cond1,SWMS.owner_id==27)).\
filter(User.id==SWMS.owner_id).all()
and then you can do a list comprehension to pull the fields you want:
details = [(u.firstname, s.filename, s.blob_key, s.last_modified) for u,s in swms]
Also worth noting you can use the '&' operator, in place of 'and_', in the body of the query. See the example (second code block) they give here:
http://docs.sqlalchemy.org/en/rel_1_0/core/sqlelement.html#sqlalchemy.sql.expression.and_