Hstore search on values with wildcard - sql

Using PostgreSQL, I want to be able to find all of the key-value pairs in an HStore field where the value matches a given query. So, for example, given a table like the following:
Name Parentage (Hstore)
_____ ___________________
Rover Mother => Yellow Lab, Father => Black Lab
Fido Mother => Black Lab, Father => Rottweiler
Rex Mother => Labrador Retriever, Father => Springer Spaniel
Lassie Mother => Border Collie, Father => Collie
How could I do a query for any dog that has a '%lab%' in its family tree? I.e. the search would bring up Rover, Fido and Rex, but not Lassie. The only examples I've seen are of searches within a single key - I need a search within all values that allows wildcards.
I have looked at this similar question, but it seems to only do a search on a particular key, not on all of the values found in the Hstore field across all keys.
Note that this is a constructed example, I was trying to make it accessible. In my actual database, I have keys for language codes followed by values of translations of the same words in the different language. I need to be able to do a search that could hit any of the values, regardless of what language it is in.

Break out the hstore into rows of name/parent and then select distinct
names of dogs where the parent column matches your search criteria.
Depending on the nature of your actual data, this may want additional
indexes. I wouldn't use hstore for this, but your actual data may
be different.
% psql -qe -f hstore.sql
begin;
create extension hstore;
create temp table dogs (
"name" text,
parentage hstore
);
insert into dogs values
('Rover', 'Mother => "Yellow Lab",Father => "Black Lab"')
,('Fido', 'Mother => "Black Lab",Father => "Rottweiler"')
,('Rex', 'Mother => "Labrador Retriever",Father => "Springer Spaniel"')
,('Lassie', 'Mother => "Border Collie",Father => "Collie"')
;
table dogs;
name | parentage
--------+--------------------------------------------------------------
Rover | "Father"=>"Black Lab", "Mother"=>"Yellow Lab"
Fido | "Father"=>"Rottweiler", "Mother"=>"Black Lab"
Rex | "Father"=>"Springer Spaniel", "Mother"=>"Labrador Retriever"
Lassie | "Father"=>"Collie", "Mother"=>"Border Collie"
(4 rows)
select * from dogs where "name" in
(select distinct "name" from (
select "name", unnest(avals(parentage)) as parent
) as plist
where parent ilike '%lab%'
);
name | parentage
-------+--------------------------------------------------------------
Rover | "Father"=>"Black Lab", "Mother"=>"Yellow Lab"
Fido | "Father"=>"Rottweiler", "Mother"=>"Black Lab"
Rex | "Father"=>"Springer Spaniel", "Mother"=>"Labrador Retriever"
(3 rows)
rollback;

Related

Example of a relation (in math sense/relational algebra) illustrated on SQL table

I have a table with header "ID" (Integer), "Name" (String), "Address" (String).
+---------------------+
| ID | Name | City |
+---------------------+
| 1 | John | London |
| 2 | Mary | Paris |
| 3 | Cris | Tokio |
+---------------------+
Could you please help me and show how that table is a relation?
Binary Relation is a set of ordered pairs, each pair is a mapping from set A to set B
(like A={1,2,3} B={A,B} => {1A, 2A, 2B, 3A} is a binary relation)
I guess table header (set of attribute names) form the first set of pair (A as in example above), but what is the second set in my case? And what is a 3-tuple ("1 | John | London") if I need to have pairs (and tuple is not a pair) - the second part of that pair (A set) is just always implicitly implied?
I think in my case it is a ternary relation, I have three sets (set of ISs, set of Names, set of cities) and table header shall not be considered a set speaking about relations (header is used because all 3-tuples would be ordered according to header). So binary relation is totally unrelated to my table. Am I right?
prooflink
Don't mix binary relation in mathematics with Binary Relation in Database. In DB concept binary relation points the relationship between two different Entities where as binary relation in math is related to Set Theory. See this
Altough they are different there are close relationships between them. You can check Applied Mathematics for Database Professionals
1) For table header:
relation is a set of (ordered) pairs:
{ (ID, Integer), {Name, String}, {City, String) }
2) For table body
relation is a set of (ordered) pairs:
{ (ID, 1), (Name, John), (City, London),
(ID, 2), (Name, Mary), (City, Paris),
(ID, 3), (Name, Cris), (City, Tokio) }
For example these three pairs { (ID, 1), (Name, John), (City, London) } mean a tuple (3-tuple = record = table row). More exactly a tuple shall be a set of (ordered) triples, not set of pairs: { (ID, Integer, 1), (Name, String, John), (City, String, London) }, but whenever we see "ID" we always know it is Integer (domain = data type), so domain/type is omitted and so we speak of pairs (instead of speaking about triplets).
So a (normalized) table can be analagous to an algebraic binary relation and binary relation is defined as a set of ordered pairs.
I am just not 100% sure if for such relaltion=table the set of pairs for the relation shall include union of pair sets in 1 and 2 (sets of pirs for header and body).

Getting join table data into Yii2 gridview

I have 3 tables that i basically need to display in the single gridview.
property
sale (FK property_id)
sale_staff (FK sale_id, staff_id)
staff
Now a staff member can be a lister, a seller, or both, and each sale can have multiple lister or sellers.
So table sale_staff has column 'staff_type' with possible enum values lister',seller`.
So the gridview must have a display something like
property street | sale price | sale date | lister | seller
and the lister/seller fields should be able to display simple initials comma separated like MJ, AB. This initial field comes from the staff table, which as above is linked to sale_staff via id.
I have not seen this type of scenario before so im simply unsure of how to incorporate the sale_type into the search so that only a staff listed as lister or seller is shown in that place.
Also, if i try something like this, just to see if anything is pulled back:
'attribute' => 'saleStaff.staff_id.initials'
or
'attribute' => 'saleStaff.staff'
I get (not set) in the gridview.
I have relations set up as:
Sale:
public function getProperty()
{
return $this->hasOne(Property::className(), ['id' => 'property_id']);
}
public function getSaleStaff()
{
return $this->hasMany(SaleStaff::className(), ['sale_id' => 'id']);
}
Sale_Staff:
public function getStaff()
{
return $this->hasOne(Staff::className(), ['id' => 'staff_id']);
}
public function getSale()
{
return $this->hasOne(Sale::className(), ['id' => 'sale_id']);
}
How do I match up the join table sale_staff with the sale table ? Is the structure ok or is DB incorrect ? I have done it this way to be flexible - any number of staff can be either a lister, seller, or both such as:
property street | sale price | sale date | lister | seller
Parkers Road | 400,000 | 22/06/2016| MJ | MJ, AB
Naturally, key question is how to represent multiple (unknown number of) values in a single gridview column like that.

postgis query for addresses (with osm data)

I want to make queries for addresses to postgis database with data from openstreetmap, check if such address exist in database and if so, get coordinates. Database was filled from .pbf file using osmosis. This is schema for the database http://pastebin.com/Yigjt77f. I have addresses in form of city name, street name and number of street. The most important for me is this table:
CREATE TABLE node_tags (
node_id BIGINT NOT NULL,
k text NOT NULL,
v text NOT NULL
);
k column is in form of tags, one that I'm interested are: addr:housenumber, addr:street, addr:city and v is corresponding value. First I'm searching if name of city matches one in database, than in results set I'm searching for street and than for house number. The problem is that I don't know how to make SQL query that will get this result with asking only once. I can ask first only for city name, get all node_id that match my city and save them in java program, than make queries asking for each found(matching my city) id_number (list from my java program) for the street, and so on. This way is really slow, because asking for more detailed information (city than street than number) I have to make more and more queries and what is more I have to check a lot of addresses. Once I have matching node_id I can easily find coordinates, so that's not a problem.
Example of this table:
node_id | k | v <br>
123 | addr:housenumber | 50
123 | addr:street | Kingsway
123 | addr:city | London
123 | (some other stuff) | .....
100 | addr:housenumber | 121
100 | addr:street | Edmund St
100 | addr:city | London
I hope I explained clearly what is my problem.
This is not as easy as you might think. Addresses in OSM are hierarchically, like in the real world. Not all elements in OSM have a full address attached. Some only have addr:housenumber and simply belong to the nearest street. Some have addr:housenumber and addr:street but no addr:city because they simply belong to the nearest city. Or they are enclosed by a boundary relation which specifies the corresponding city. And instead of addr:housenumber there are sometimes also just address interpolations described by the addr:interpolation key. See the addr key wiki page for more information.
The Karlsruhe Schema page in the OSM wiki explains a lot about addresses in OSM. It also mentions associatedStreet relations which are sometimes used to group house numbers and their corresponding streets.
As you can see a single query in the database probably won't suffice. If you need some inspiration you can take a look at OSM's address search engine Nominatim. But note that Nominatim uses a different data base scheme than the usual one in order to optimize address queries. You can also take a look at one of the many routing applications which all have to do address lookups.

How do I create sql query for searching partial matches?

I have a set of items in db .Each item has a name and a description.I need to implement a search facility which takes a number of keywords and returns distinct items which have at least one of the keywords matching a word in the name or description.
for example
I have in the db ,three items
1.item1 :
name : magic marker
description: a writing device which makes erasable marks on whiteboard
2.item2:
name: pall mall cigarettes
description: cigarette named after a street in london
3.item3:
name: XPigment Liner
description: for writing and drawing
A search using keyword 'writing' should return magic marker and XPigment Liner
A search using keyword 'mall' should return the second item
I tried using the LIKE keyword and IN keyword separately ,..
For IN keyword to work,the query has to be
SELECT DISTINCT FROM mytable WHERE name IN ('pall mall cigarettes')
but
SELECT DISTINCT FROM mytable WHERE name IN ('mall')
will return 0 rows
I couldn't figure out how to make a query that accommodates both the name and description columns and allows partial word match..
Can somebody help?
update:
I created the table through hibernate and for the description field, used javax.persistence #Lob annotation.Using psql when I examined the table,It is shown
...
id | bigint | not null
description | text |
name | character varying(255) |
...
One of the records in the table is like,
id | description | name
21 | 133414 | magic marker
First of all, this approach won't scale in the large, you'll need a separate index from words to item (like an inverted index).
If your data is not large, you can do
SELECT DISTINCT(name) FROM mytable WHERE name LIKE '%mall%' OR description LIKE '%mall%'
using OR if you have multiple keywords.
This may work as well.
SELECT *
FROM myTable
WHERE CHARINDEX('mall', name) > 0
OR CHARINDEX('mall', description) > 0

Creating new table from data of other tables

I'm very new to SQL and I hope someone can help me with some SQL syntax. I have a database with these tables and fields,
DATA: data_id, person_id, attribute_id, date, value
PERSONS: person_id, parent_id, name
ATTRIBUTES: attribute_id, attribute_type
attribute_type can be "Height" or "Weight"
Question 1
Give a person's "Name", I would like to return a table of "Weight" measurements for each children. Ie: if John has 3 children names Alice, Bob and Carol, then I want a table like this
| date | Alice | Bob | Carol |
I know how to get a long list of children's weights like this:
select d.date,
d.value
from data d,
persons child,
persons parent,
attributes a
where parent.name='John'
and child.parent_id = parent.person_id
and d.attribute_id = a.attribute_id
and a.attribute_type = "Weight';
but I don't know how to create a new table that looks like:
| date | Child 1 name | Child 2 name | ... | Child N name |
Question 2
Also, I would like to select the attributes to be between a certain range.
Question 3
What happens if the dates are not consistent across the children? For example, suppose Alice is 3 years older than Bob, then there's no data for Bob during the first 3 years of Alice's life. How does the database handle this if we request all the data?
1) It might not be so easy. MS SQL Server can PIVOT a table on an axis, but dumping the resultset to an array and sorting there (assuming this is tied to some sort of program) might be the simpler way right now if you're new to SQL.
If you can manage to do it in SQL it still won't be enough info to create a new table, just return the data you'd use to fill it in, so some sort of external manipulation will probably be required. But you can probably just use INSERT INTO [new table] SELECT [...] to fill that new table from your select query, at least.
2) You can join on attributes for each unique attribute:
SELECT [...] FROM data AS d
JOIN persons AS p ON d.person_id = p.person_id
JOIN attributes AS weight ON p.attribute_id = weight.attribute_id
HAVING weight.attribute_type = 'Weight'
JOIN attributes AS height ON p.attribute_id = height.attribute_id
HAVING height.attribute_type = 'Height'
[...]
(The way you're joining in the original query is just shorthand for [INNER] JOIN .. ON, same thing except you'll need the HAVING clause in there)
3) It depends on the type of JOIN you use to match parent/child relationships, and any dates you're filtering on in the WHERE, if I'm reading that right (entirely possible I'm not). I'm not sure quite what you're looking for, or what kind of database you're using, so no good answer. If you're new enough to SQL that you don't know the different kinds of JOINs and what they can do, it's very worthwhile to learn them - they put the R in RDBMS.
when you do a select, you need to specify the exact columns you want. In other words you can't return the Nth child's name. Ie this isn't possible:
1/2/2010 | Child_1_name | Child_2_name | Child_3_name
1/3/2010 | Child_1_name
1/4/2010 | Child_1_name | Child_2_name
Each record needs to have the same amount of columns. So you might be able to make a select that does this:
1/2/2010 | Child_1_name
1/2/2010 | Child_2_name
1/2/2010 | Child_3_name
1/3/2010 | Child_1_name
1/4/2010 | Child_1_name
1/4/2010 | Child_2_name
And then in a report remap it to how you want it displayed