Consider a SPARQL aggregation query with a where clause composed of only triple patterns and filter statements.
Does the order of the triple patterns or the order of the filters affect the results (not the performance)? I am quite convinced it doesn't affect the validity of the results. But maybe someone can clarify why? Also, does the order of the variables in the group by affect the result's validity? I don't think so either but maybe someone can clarify why?
The following example is just for clarification; I am asking generally.
The query counts the number of people grouped by their eye color and country and applies a couple of filters. The query contains only these four triple patterns (each line ending by a dot is a triple pattern, i.e., a triple with variables)
?p wdt:P26 ?human.
?p wdt:P27 ?country.
?human wdt:P31 wd:Q5.
?human wdt:P1340 ?eyeColor.
and these two filters:
FILTER (?country != wd:Q142)
FILTER (?eyeColor != wd:Q17122705)
The WHERE clause doesn't contain any OPTIONAL, UNION, MINUS,... or any other constructor; only triple patterns and filters.
SELECT ?eyeColor (COUNT(?human) AS ?count) ?country
WHERE
{
?p wdt:P26 ?human.
?p wdt:P27 ?country.
?human wdt:P31 wd:Q5.
?human wdt:P1340 ?eyeColor.
FILTER (?country != wd:Q142)
FILTER (?eyeColor != wd:Q17122705)
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
GROUP BY ?eyeColor ?country
My question again: if we change the order of the six elements in the WHERE clause in any way, does that affect the result (not only in this query but also for similar ones in terms of structure)? The same question about whether the order of the variables in the GROUP BY might affect the results, for example GROUP BY ?country ?eyeColor instead of GROUP BY ?eyeColor ?country. Again, I am asking in general, not only for this case query.
No, the order does not change the results for tripe patterns and filters only.
All filters happen at the end of the {} block they are in (an optimizer may move them but must not change the result).
A number of triple patterns "subject predicate object" will result in the same results regardless of order.
SERVICE is not a triple pattern.
wikibase:label is special to wikidata, not the SPARQL spec anyway.
Including any of the graph pattern operations like SERVICE or OPTIONAL.
See the SPARQL algebra the spec or sparql.org to see the algebra.
Related
I am using Jena ARQ to write a SPARQL query against a large ontology being read from Jena TDB in order to find the types associated with concepts based on rdfs label:
SELECT DISTINCT ?type WHERE {
?x <http://www.w3.org/2000/01/rdf-schema#label> "aspirin" .
?x <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?type .
}
This works pretty well and is actually quite speedy (<1 second). Unfortunately, for some terms, I need to perform this query in a case-insensitive way. For instance, because the label "Tylenol" is in the ontology, but not "tylenol", the following query comes up empty:
SELECT DISTINCT ?type WHERE {
?x <http://www.w3.org/2000/01/rdf-schema#label> "tylenol" .
?x <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?type .
}
I can write a case-insensitive version of this query using FILTER syntax like so:
SELECT DISTINCT ?type WHERE {
?x <http://www.w3.org/2000/01/rdf-schema#label> ?term .
?x <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?type .
FILTER ( regex (str(?term), "tylenol", "i") )
}
But now the query takes over a minute to complete! Is there any way to write the case-insensitive query in a more efficient manner?
From all the the possible string operators that you can use in SPARQL, regex is probably the most expensive one. Your query might run faster if you avoid regex and you use UCASE or LCASE on both sides of the test instead. Something like:
SELECT DISTINCT ?type WHERE {
?x <http://www.w3.org/2000/01/rdf-schema#label> ?term .
?x <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?type .
FILTER (lcase(str(?term)) = "tylenol")
}
This might be faster but in general do not expect great performance for text search with any triple store. Triple stores are very good at graph matching and not so good at string matching.
The reason the query with the FILTER query runs slower is because ?term is unbound it requires scanning the PSO or POS index to find all statements with the rdfs:label predicate and filter them against the regex. When it was bound to a concrete resource (in your first example), it could use a OPS or POS index to scan over only statements with the rdfs:label predicate and the specified object resource, which would have a much lower cardinality.
The common solution to this type of text searching problem is to use an external text index. In this case, Jena provides a free text index called LARQ, which uses Lucene to perform the search and joins the results with the rest of the query.
I am trying to map DBPedia types to Wikipedia Categories, a simple example would be the following SPARQL query
select distinct ?cat where {
?s a dbpedia-owl:LacrossePlayer; dcterms:subject ?cat . filter(regex(?cat,'players','i') )
} limit 100
SPARQL Result
But this is highly inefficient as it has to first map the DBpedia types to DBpedia Named Entities(resources) and then extract their corresponding Wikipedia categories. I am trying to do this mapping for a lot of other DBpedia types.
Is there a direct or more efficient way to do this?
Improving the filter may help…
As an initial note, you may get some speedup if you remove or improve your filter. You can, of course, just remove it, but you could also make it more efficienct, since you're not really using any special regular expressions. Just do
filter contains(lcase(str(?cat)),'players')
to check whether the URI for ?cat contains the string players. It might even be better (I'm not sure) to grab the English rdfs:label of ?cat and check that, since you wouldn't have to do the case or string conversions.
… but there are lots of results.
But this is highly inefficient as it has to first map the DBpedia
types to DBpedia Named Entities(resources) and then extract their
corresponding Wikipedia categories. I am trying to do this mapping for
a lot of other DBpedia types. Is there a direct or more efficient way
to do this?
I'm not sure exactly what's inefficient in this. The only way that DBpedia types and categories are associated is that resources have types (via rdf:type) and have categories (via dcterms:subject). If you want to find the connections, then you'll need to find the instances of the type and the categories to which they belong. There may be some possibility that you can look into whether any particular infoboxes provide categories to articles and are used in the infobox mapping to provide DBpedia types. That's the only way to get category/DBpedia-types directly, without going through instances that I can think of, and I don't know whether the current dataset has that kind of information.
In general, since Wikipedia categories are not a type hierarchy, there will be lots of categories with which instances of any particular type are associated. For instance, we can count the number of categories associated with the types Fish and LacrossePlayer with a query like this:
select ?type (count(distinct ?category) as ?nCategories) where {
values ?type { dbpedia-owl:Fish dbpedia-owl:LacrossePlayer }
?type ^a/dcterms:subject ?category
}
group by ?type
SPARQL results
type nCategories
http://dbpedia.org/ontology/LacrossePlayer 346
http://dbpedia.org/ontology/Fish 2375
That query responds pretty quickly, and you can even get those categories pretty easily, too:
select distinct ?type ?category where {
values ?type { dbpedia-owl:Fish dbpedia-owl:LacrossePlayer }
?type ^a/dcterms:subject ?category
}
order by ?type
limit 4000
SPARQL results
When you start using types that have many more instances, though, these counts get big, and the queries take a while to return. E.g., a very common type like Place:
select ?type (count(distinct ?category) as ?nCategories) where {
values ?type { dbpedia-owl:Place }
?type ^a/dcterms:subject ?category
}
group by ?type
type nCategories
http://dbpedia.org/ontology/Place 191172
I wouldn't suggest trying to pull all that data down from the remote server. If you want to extract it, you should load the data locally.
I have a burning question concerning DBpedia. Namely, I was wondering how I could search for all the properties in DBpedia per page. The URI http://nl.dbpedia.org/property/einde concerns the property "einde". I would like to get all existing property/ pages. This does not seem too hard, but I don't know anything about SPARQL, so that's why I want to ask for some help. Perhaps there is some kind of dump of it, but I honestly don't know.
Rather than asking for pages whose URLs begin with, e.g., http://nl.dbpedia.org/property/, we can express the query by asking “for which values of ?x is there a triple ?x rdf:type rdf:Property in DBpedia?” This is a pretty simple SPARQL query to write. Because I expected that there would be lots of properties in DBPedia, I first wrote a query to count how many there are, and afterward wrote a query to actually list them.
There are 48292 things in DBpedia declared to be of rdf:type rdf:Property, as reported by this SPARQL query, run against one of DBpedia's SPARQL endpoints:
select COUNT( ?property ) where {
?property a rdf:Property
}
SPARQL Results
You can get the list by selecting ?property instead of COUNT( ?property ):
select ?property where {
?property a rdf:Property
}
SPARQL Results
I second Joshua Taylor's answer, however if you want to limit the properties to the Dutch DBpedia, you need to change the default-graph-uri query parameter to nl.dbpedia.org and set the SPARQL endpoint to nl.dbpedia.org/sparql, as in the following query. You will get a result-set of just above 8000 elements.
SELECT DISTINCT ?pred WHERE {
?pred a rdf:Property
}
ORDER BY ?pred
run query
These are the Dutch translations of the properties that have been mapped from Wikipedia so far. The full English list is also available. According to mappings.dbpedia.org, there are ~1700 properties with missing Dutch translations.
I want to get data (movie title, director name, actor name and the wikipedia link) of all movies present on dbpedia.
I tried this query on http://dbpedia.org/snorql/.
SELECT ?film_title ?star_name ?nameDirector ?link WHERE {
{
SELECT DISTINCT ?movies ?film_title
WHERE {
?movies rdf:type <http://dbpedia.org/ontology/Film>;
rdfs:label ?film_title.
}
}.
?movies dbpedia-owl:starring ?star;
foaf:isPrimaryTopicOf ?link;
dbpedia-owl:director ?director.
?director foaf:name ?nameDirector.
?star foaf:name ?star_name.
FILTER LANGMATCHES( LANG(?film_title), 'en')
} LIMIT 100
Responses seems correct, but the response time are slow, so I'm wondering if I can improve my query for get a faster response.
There are a couple of things you could change in your query that might make it faster.
Firstly what is the point of your SELECT DISTINCT subquery? Is that merely trying to eliminate duplicate film titles? Removing this may make things faster if you can live with a few duplicates.
Secondly the FILTER clauses requires the database to scan over all the possible matches and evaluate the expression on each possible match to determine whether to keep it or throw it away. Again if you can live with getting some duplicate data and don't mind non-English language tags removing the FILTER may make the query run faster.
I'm trying to use a SPARQL query against DBpedia to retrieve a list of musicals and some associated properties. However, despite using the appropriate filters (as far as I can tell), the results include many of the musicals more than once. Here is my query:
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX dbpprop: <http://dbpedia.org/property/>
SELECT ?label ?abstract ?book ?music ?lyrics
WHERE {
?play <http://purl.org/dc/terms/subject> <http://dbpedia.org/resource/Category:Broadway_musicals> ;
rdfs:label ?label ;
dbo:abstract ?abstract ;
dbpprop:book ?book ;
dbpprop:lyrics ?lyrics ;
dbpprop:music ?music .
FILTER (LANG(?label) = 'en')
FILTER (LANG(?abstract) = 'en')
FILTER (LANG(?book) = 'en')
FILTER (LANG(?lyrics) = 'en')
FILTER (LANG(?music) = 'en')
}
The resulting list has many duplicate entries. Pasting the query here:
DBpedia SPARQL Explorer, you'll see that starting with 'Mama Mia!' there are a lot of duplicates in the list.
Any idea what I'm missing to get unique results with no duplicates? Thanks!
[Edited by glenn mcdonald to clarify that it's musicals which are "duplicated" here, not triples.]
SPARQL returns variable-bindings. Your "duplicates" are cartesian products of multiples in your projected properties. Mamma Mia has multiple music writers and multiple lyricists, so you get every possible combination of them that could produce a row in your table.
What a pain, huh? The "solution" is to use CONSTRUCT instead of SELECT, and deal with getting back a graph instead of a table. Maybe like this:
http://dbpedia.org/snorql/?query=PREFIX+rdfs%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23%3E%0D%0A++++PREFIX+dbo%3A+%3Chttp%3A%2F%2Fdbpedia.org%2Fontology%2F%3E%0D%0A++++PREFIX+dbpprop%3A+%3Chttp%3A%2F%2Fdbpedia.org%2Fproperty%2F%3E%0D%0A++++CONSTRUCT+%7B%0D%0A++++++++%3Fplay+rdfs%3Alabel+%3Flabel+%3B%0D%0A++++++++++++dbo%3Aabstract+%3Fabstract+%3B%0D%0A++++++++++++dbpprop%3Abook+%3Fbook+%3B%0D%0A++++++++++++dbpprop%3Alyrics+%3Flyrics+%3B%0D%0A++++++++++++dbpprop%3Amusic+%3Fmusic+.%0D%0A++++%7D%0D%0A++++WHERE+%7B+%0D%0A++++++++%3Fplay+%3Chttp%3A%2F%2Fpurl.org%2Fdc%2Fterms%2Fsubject%3E+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FCategory%3ABroadway_musicals%3E+%3B%0D%0A++++++++++++rdfs%3Alabel+%3Flabel+%3B%0D%0A++++++++++++dbo%3Aabstract+%3Fabstract+%3B%0D%0A++++++++++++dbpprop%3Abook+%3Fbook+%3B%0D%0A++++++++++++dbpprop%3Alyrics+%3Flyrics+%3B%0D%0A++++++++++++dbpprop%3Amusic+%3Fmusic+.%0D%0A++++++++FILTER+%28LANG%28%3Flabel%29+%3D+%27en%27%29++++%0D%0A++++++++FILTER+%28LANG%28%3Fabstract%29+%3D+%27en%27%29%0D%0A++++++++FILTER+%28LANG%28%3Fbook%29+%3D+%27en%27%29%0D%0A++++++++FILTER+%28LANG%28%3Flyrics%29+%3D+%27en%27%29%0D%0A++++++++FILTER+%28LANG%28%3Fmusic%29+%3D+%27en%27%29%0D%0A++++%7D
Are the duplicates exact duplicates? i.e. every value for every variable of each duplicate result is identical
If so then add the DISTINCT keyword after SELECT to force the SPARQL engine to discard duplicates solutions.
If not then Glenn is entirely correct that because there are multiple values given for the various properties so you will get multiple results. There are complex workarounds you can do with subqueries, GROUP BY etc. but they would tend to lead to less efficient queries. Sometimes you just have to deal with the duplicates on the client side.