I want to get data (movie title, director name, actor name and the wikipedia link) of all movies present on dbpedia.
I tried this query on http://dbpedia.org/snorql/.
SELECT ?film_title ?star_name ?nameDirector ?link WHERE {
{
SELECT DISTINCT ?movies ?film_title
WHERE {
?movies rdf:type <http://dbpedia.org/ontology/Film>;
rdfs:label ?film_title.
}
}.
?movies dbpedia-owl:starring ?star;
foaf:isPrimaryTopicOf ?link;
dbpedia-owl:director ?director.
?director foaf:name ?nameDirector.
?star foaf:name ?star_name.
FILTER LANGMATCHES( LANG(?film_title), 'en')
} LIMIT 100
Responses seems correct, but the response time are slow, so I'm wondering if I can improve my query for get a faster response.
There are a couple of things you could change in your query that might make it faster.
Firstly what is the point of your SELECT DISTINCT subquery? Is that merely trying to eliminate duplicate film titles? Removing this may make things faster if you can live with a few duplicates.
Secondly the FILTER clauses requires the database to scan over all the possible matches and evaluate the expression on each possible match to determine whether to keep it or throw it away. Again if you can live with getting some duplicate data and don't mind non-English language tags removing the FILTER may make the query run faster.
Related
I am trying to map DBPedia types to Wikipedia Categories, a simple example would be the following SPARQL query
select distinct ?cat where {
?s a dbpedia-owl:LacrossePlayer; dcterms:subject ?cat . filter(regex(?cat,'players','i') )
} limit 100
SPARQL Result
But this is highly inefficient as it has to first map the DBpedia types to DBpedia Named Entities(resources) and then extract their corresponding Wikipedia categories. I am trying to do this mapping for a lot of other DBpedia types.
Is there a direct or more efficient way to do this?
Improving the filter may help…
As an initial note, you may get some speedup if you remove or improve your filter. You can, of course, just remove it, but you could also make it more efficienct, since you're not really using any special regular expressions. Just do
filter contains(lcase(str(?cat)),'players')
to check whether the URI for ?cat contains the string players. It might even be better (I'm not sure) to grab the English rdfs:label of ?cat and check that, since you wouldn't have to do the case or string conversions.
… but there are lots of results.
But this is highly inefficient as it has to first map the DBpedia
types to DBpedia Named Entities(resources) and then extract their
corresponding Wikipedia categories. I am trying to do this mapping for
a lot of other DBpedia types. Is there a direct or more efficient way
to do this?
I'm not sure exactly what's inefficient in this. The only way that DBpedia types and categories are associated is that resources have types (via rdf:type) and have categories (via dcterms:subject). If you want to find the connections, then you'll need to find the instances of the type and the categories to which they belong. There may be some possibility that you can look into whether any particular infoboxes provide categories to articles and are used in the infobox mapping to provide DBpedia types. That's the only way to get category/DBpedia-types directly, without going through instances that I can think of, and I don't know whether the current dataset has that kind of information.
In general, since Wikipedia categories are not a type hierarchy, there will be lots of categories with which instances of any particular type are associated. For instance, we can count the number of categories associated with the types Fish and LacrossePlayer with a query like this:
select ?type (count(distinct ?category) as ?nCategories) where {
values ?type { dbpedia-owl:Fish dbpedia-owl:LacrossePlayer }
?type ^a/dcterms:subject ?category
}
group by ?type
SPARQL results
type nCategories
http://dbpedia.org/ontology/LacrossePlayer 346
http://dbpedia.org/ontology/Fish 2375
That query responds pretty quickly, and you can even get those categories pretty easily, too:
select distinct ?type ?category where {
values ?type { dbpedia-owl:Fish dbpedia-owl:LacrossePlayer }
?type ^a/dcterms:subject ?category
}
order by ?type
limit 4000
SPARQL results
When you start using types that have many more instances, though, these counts get big, and the queries take a while to return. E.g., a very common type like Place:
select ?type (count(distinct ?category) as ?nCategories) where {
values ?type { dbpedia-owl:Place }
?type ^a/dcterms:subject ?category
}
group by ?type
type nCategories
http://dbpedia.org/ontology/Place 191172
I wouldn't suggest trying to pull all that data down from the remote server. If you want to extract it, you should load the data locally.
I have a burning question concerning DBpedia. Namely, I was wondering how I could search for all the properties in DBpedia per page. The URI http://nl.dbpedia.org/property/einde concerns the property "einde". I would like to get all existing property/ pages. This does not seem too hard, but I don't know anything about SPARQL, so that's why I want to ask for some help. Perhaps there is some kind of dump of it, but I honestly don't know.
Rather than asking for pages whose URLs begin with, e.g., http://nl.dbpedia.org/property/, we can express the query by asking “for which values of ?x is there a triple ?x rdf:type rdf:Property in DBpedia?” This is a pretty simple SPARQL query to write. Because I expected that there would be lots of properties in DBPedia, I first wrote a query to count how many there are, and afterward wrote a query to actually list them.
There are 48292 things in DBpedia declared to be of rdf:type rdf:Property, as reported by this SPARQL query, run against one of DBpedia's SPARQL endpoints:
select COUNT( ?property ) where {
?property a rdf:Property
}
SPARQL Results
You can get the list by selecting ?property instead of COUNT( ?property ):
select ?property where {
?property a rdf:Property
}
SPARQL Results
I second Joshua Taylor's answer, however if you want to limit the properties to the Dutch DBpedia, you need to change the default-graph-uri query parameter to nl.dbpedia.org and set the SPARQL endpoint to nl.dbpedia.org/sparql, as in the following query. You will get a result-set of just above 8000 elements.
SELECT DISTINCT ?pred WHERE {
?pred a rdf:Property
}
ORDER BY ?pred
run query
These are the Dutch translations of the properties that have been mapped from Wikipedia so far. The full English list is also available. According to mappings.dbpedia.org, there are ~1700 properties with missing Dutch translations.
I'm trying to query for all Countries in DBpedia and get their human development index.
The query I am trying is:
SELECT *
WHERE {
?Country a <http://dbpedia.org/ontology/Country> .
?Country <http://dbpedia.org/ontology/humanDevelopmentIndex> ?humanDevelopmentIndex .
}
LIMIT 1000
Would anyone be able to explain why this query isn't returning any results? It seems straightforward to me.
You're not getting anything back because apparently, none of the countries in DBpedia actually have a humanDevelopmentIndex property associated with them.
You can verify this for yourself. If you simplify your query to just get back countries:
SELECT *
WHERE {
?Country a <http://dbpedia.org/ontology/Country> .
}
LIMIT 1000
You will get back a list of countries, so clearly it is the addition of the other property pattern that causes the query to not match any results. Also, if you take a look at the data for, for example, Austrialia in DBPedia, you will not find the property you want there.
The reason it doesn't appear is that the data you want is probably located in the ontology_infobox_properties or the ontology_infobox_properties_specific dataset. These are not exposed in the public endpoint, but you can download them.
I have a sparql query that returns duplicates, and I want it to clean them up on one of the values only (subjectID). Unlike DISTINCT that seems to find a unique value for the combination of values selected, rather than for only one of the parameters.
I saw someone here propose group by, but that only seems applicable if I list all the parameters after group by (my sparql endpoint complains, e.g. Non-group key variable in SELECT: ?occupation).
I tried running an internal select, but it doesn't seem to work for this specific query. So might be an issue with the query itself ( the values of the livedIn optional seem to be causing the duplicate) ?
While happy enough with relational DBs early in the learning curve with SPARQL, so feel free to explain the obvious for the otherwise uninitiated! :)
select distinct
?subjectID ?englishName ?sex ?locatedIn15Name
?dob ?dod ?dom ?bornLocationName ?occupation
where {
?person a hc:Person ;
hc:englishName ?englishName ;
hc:sex ?sex;
hc:subjectID ?subjectID;
optional { ?person hc:livedIn11 ?livedIn11 .
?livedIn11 hc:englishName ?lived11LocationName .
?livedIn11 hc:locatedIn11 ?locatedIn11 .
?locatedIn11 hc:englishName ?locatedIn11Name .
?locatedIn11 hc:locatedIn15 ?locatedIn15 .
?locatedIn15 hc:englishName ?locatedIn15Name .
} .
optional {?person hc:born ?dob } .
optional {?person hc:dateOfDeath ?dod } .
optional {?person hc:dateOfMarriage ?dom } .
optional { ?person hc:bornIn ?bornIn .
?bornIn hc:englishName ?bornLocationName .
?bornIn hc:easting ?easting .
?bornIn hc:northing ?northing } .
optional { ?person hc:occupation ?occupation }
FILTER regex(?englishName, "^FirstName LastName")
}
GROUP BY
?subjectID ?englishName ?sex
?locatedIn15Name ?dob ?dod ?dom
?bornLocationName ?occupation
Re the error message:
Non-group key variable in SELECT: ?occupation
You can avoid this by using the SAMPLE() aggregate - this will allow you to just group on ?subjectID but still select values for the rest of the variables provided you only care about getting one value for those other variables.
Here's a simple example of this:
SELECT ?subjectID (SAMPLE(?dob) AS ?dateOfBirth)
WHERE
{
?person a hc:Person ;
hc:subjectID ?subjectID .
OPTIONAL { ?person hc:born ?dob }
}
GROUP BY ?subjectID
First thing to note is that there is no such thing as a key, really, in RDF/SPARQL. You're querying a graph, and ?subjectID may simply have several possible combinations of values for the other variables you are selecting. This is caused by the shape of the graph you're querying: perhaps your person has more than one english name, or indeed the other way around: the same english name can be shared by more than one person.
A SPARQL SELECT query is a strange beast: it queries a graph structure but presents the result as a flat table (technically, it's a sequence of sets of variable bindings, but it amounts to the same thing). Duplicates occur because different combinations of values for your variables can be found by basically following different paths in the graph.
The fact that you get duplicate values for ?subjectID in your result is therefore unavoidable, simply because these are, from the point of view of the RDF graph, unique solutions to your query. You can not filter out results without actually losing information, so in general it's hard to give you a solution without knowing more about exactly which 'duplicates' you want to discard: do you only want one possible english name for each subject, or one possible date of birth (even though there may be more than one in your data)?
However, here are some tips for handling/procesing such results more easily:
First of all, you could choose to use an ORDER BY clause on your ?subjectID variable. This will still give you several rows with the same value for ?subjectID, but they'll all be in order, so you can process your result more efficiently.
Another solution is to split your query in two: do a first query that only selects all unique subjects (and possibly all other values for which you know, in advance, that they will be unique given the subject), then iterate over the result and do a separate query to get the other values you're interested in, for each individual subjectID value. This solution may sound like heresy (especially if you're from an SQL background), but it might actually be quicker and easier than trying to do everything in one huge query.
Yet another solution is the one suggested by RobV: using a SAMPLE aggregate on a particular variable to just select one (random) unique value. A variation on that is to use the GROUP_CONCAT aggregate, which creates a single value by concatenating all possible values into a single string.
I'm trying to use a SPARQL query against DBpedia to retrieve a list of musicals and some associated properties. However, despite using the appropriate filters (as far as I can tell), the results include many of the musicals more than once. Here is my query:
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX dbpprop: <http://dbpedia.org/property/>
SELECT ?label ?abstract ?book ?music ?lyrics
WHERE {
?play <http://purl.org/dc/terms/subject> <http://dbpedia.org/resource/Category:Broadway_musicals> ;
rdfs:label ?label ;
dbo:abstract ?abstract ;
dbpprop:book ?book ;
dbpprop:lyrics ?lyrics ;
dbpprop:music ?music .
FILTER (LANG(?label) = 'en')
FILTER (LANG(?abstract) = 'en')
FILTER (LANG(?book) = 'en')
FILTER (LANG(?lyrics) = 'en')
FILTER (LANG(?music) = 'en')
}
The resulting list has many duplicate entries. Pasting the query here:
DBpedia SPARQL Explorer, you'll see that starting with 'Mama Mia!' there are a lot of duplicates in the list.
Any idea what I'm missing to get unique results with no duplicates? Thanks!
[Edited by glenn mcdonald to clarify that it's musicals which are "duplicated" here, not triples.]
SPARQL returns variable-bindings. Your "duplicates" are cartesian products of multiples in your projected properties. Mamma Mia has multiple music writers and multiple lyricists, so you get every possible combination of them that could produce a row in your table.
What a pain, huh? The "solution" is to use CONSTRUCT instead of SELECT, and deal with getting back a graph instead of a table. Maybe like this:
http://dbpedia.org/snorql/?query=PREFIX+rdfs%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23%3E%0D%0A++++PREFIX+dbo%3A+%3Chttp%3A%2F%2Fdbpedia.org%2Fontology%2F%3E%0D%0A++++PREFIX+dbpprop%3A+%3Chttp%3A%2F%2Fdbpedia.org%2Fproperty%2F%3E%0D%0A++++CONSTRUCT+%7B%0D%0A++++++++%3Fplay+rdfs%3Alabel+%3Flabel+%3B%0D%0A++++++++++++dbo%3Aabstract+%3Fabstract+%3B%0D%0A++++++++++++dbpprop%3Abook+%3Fbook+%3B%0D%0A++++++++++++dbpprop%3Alyrics+%3Flyrics+%3B%0D%0A++++++++++++dbpprop%3Amusic+%3Fmusic+.%0D%0A++++%7D%0D%0A++++WHERE+%7B+%0D%0A++++++++%3Fplay+%3Chttp%3A%2F%2Fpurl.org%2Fdc%2Fterms%2Fsubject%3E+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FCategory%3ABroadway_musicals%3E+%3B%0D%0A++++++++++++rdfs%3Alabel+%3Flabel+%3B%0D%0A++++++++++++dbo%3Aabstract+%3Fabstract+%3B%0D%0A++++++++++++dbpprop%3Abook+%3Fbook+%3B%0D%0A++++++++++++dbpprop%3Alyrics+%3Flyrics+%3B%0D%0A++++++++++++dbpprop%3Amusic+%3Fmusic+.%0D%0A++++++++FILTER+%28LANG%28%3Flabel%29+%3D+%27en%27%29++++%0D%0A++++++++FILTER+%28LANG%28%3Fabstract%29+%3D+%27en%27%29%0D%0A++++++++FILTER+%28LANG%28%3Fbook%29+%3D+%27en%27%29%0D%0A++++++++FILTER+%28LANG%28%3Flyrics%29+%3D+%27en%27%29%0D%0A++++++++FILTER+%28LANG%28%3Fmusic%29+%3D+%27en%27%29%0D%0A++++%7D
Are the duplicates exact duplicates? i.e. every value for every variable of each duplicate result is identical
If so then add the DISTINCT keyword after SELECT to force the SPARQL engine to discard duplicates solutions.
If not then Glenn is entirely correct that because there are multiple values given for the various properties so you will get multiple results. There are complex workarounds you can do with subqueries, GROUP BY etc. but they would tend to lead to less efficient queries. Sometimes you just have to deal with the duplicates on the client side.