Searching semantically tagged documents in MarkLogic - semantics

Can any one please point me to some simple examples of semantic tagging and querying semantically tagged documents in MarkLogic?
I am fairly new in this area,so some beginner level examples will do.

When you say "semantically tagged" do you mean regular XML documents that happen to have some triples in them? The discussion and examples at http://docs.marklogic.com/guide/semantics/embedded are pretty good for that.
Start by enabling the triple index in your database. Then insert a test doc. This is just XML, but the sem:triple element represents a semantic fact.
xdmp:document-insert(
'test.xml',
<test>
<source>AP Newswire</source>
<sem:triple date="1972-02-21" confidence="100">
<sem:subject>http://example.org/news/Nixon</sem:subject>
<sem:predicate>http://example.org/wentTo</sem:predicate>
<sem:object>China</sem:object>
</sem:triple>
</test>)
Then query it. The example query is pretty complicated. To understand what's going on I'd insert variations on that sample document, using different URIs instead of just test.xml, and see how the various query terms match up. Try using just the SPARQL component, without the extra cts query. Try cts:search with no SPARQL, just the cts:query.
xquery version "1.0-ml";
import module namespace sem = "http://marklogic.com/semantics"
at "/MarkLogic/semantics.xqy";
sem:sparql('
SELECT ?country
WHERE {
<http://example.org/news/Nixon> <http://example.org/wentTo> ?country
}
',
(),
(),
cts:and-query((
cts:path-range-query( "//sem:triple/#confidence", ">", 80) ,
cts:path-range-query( "//sem:triple/#date", "<", xs:date("1974-01-01")),
cts:or-query((
cts:element-value-query( xs:QName("source"), "AP Newswire"),
cts:element-value-query( xs:QName("source"), "BBC"))))))

In case you are talking about enriching your content using semantic technology, that is not directly provided by MarkLogic.
You can enrich your content externally, for instance by calling a public service like the one provided by OpenCalais, and then insert the enrichments to the content before insert.
You can also build lists of lookup values, and then using cts:highlight to mark such terms within your content. That could be as simple as:
let $labels := ("MarkLogic", "StackOverflow")
return
cts:highlight($doc, cts:word-query($labels), <b>{$cts:text}</b>)
Or with a more dynamic replacement using spraql:
let $labels := map:new()
let $_ :=
for $result in sem:sparql('
PREFIX demo: <http://www.marklogic.com/ontologies/demo#>
SELECT DISTINCT ?label
WHERE {
?s a demo:person.
{
?s demo:fullName ?label
} UNION {
?s demo:initialsName ?label
} UNION {
?s demo:email ?label
}
}
')
return
map:put($labels, map:get($result, 'label'), 'person')
return
cts:highlight($doc, cts:word-query(map:keys($labels)),
let $result := sem:sparql(concat('
PREFIX demo: <http://www.marklogic.com/ontologies/demo#>
SELECT DISTINCT ?s ?p
{
?s a demo:', map:get($labels, $cts:text), ' .
?s ?p "', $cts:text, '" .
}
'))
return
if (map:contains($labels, $cts:text))
then
element { xs:QName(fn:concat("demo:", map:get($labels, $cts:text))) } {
attribute subject { map:get($result, 's') },
attribute predicate { map:get($result, 'p') },
$cts:text
}
else ()
)
HTH!

Related

How to filter distinct regex matches with SPARQL?

I am querying a (kind of) bibliographic database and would like to find all the distinct matches of a certain regex (matching the signature of typescripts (TS) and manuscripts (MS)); i.e. I would like to return all documents that are currently in the database.
I came up with
SELECT ?document
WHERE
{
{
?documentURI a witt:MS;
rdfs:label ?document.
}
UNION
{
?documentURI a witt:TS;
rdfs:label ?document.
}
FILTER (regex(?document, "(Ms|Ts)\\-((1|2|3)\\d{2}\\w?\\d?)"))
}
(endpoint); this returns all the signatures but I would like to filter the result for the distinct regex matches, i.e. the distinct signatures up to and excluding the comma.
How can this be achieved?
Ok, I think I found a solution with strbefore:
SELECT DISTINCT ?document
WHERE
{
{
?documentURI a witt:MS;
rdfs:label ?documentFull.
}
UNION
{
?documentURI a witt:TS;
rdfs:label ?documentFull.
}
BIND (strbefore(?documentFull, ",") AS ?document)
}
Try it.
Would appreciate opinions on the query though, is this effective/good style?

Why are some of the classes in my ontolgoy not being returned in my SPARQL query?

I'm rather confused about why I'm receiving too little results (256 expected, 224 returned). When I run the code below, everything returns exactly as I want it, except that I miss all the classes in my ontology which lie at the highest level, or one below the highest level. I don't understand where I'm being too "strict" in my query so that these classes are not being returned in the table. They also all have parent classes (whether that be the topmost class "top concept" of the ontology, or something like an enumeration type class, either way, the leaf should still be found. Would be thankful for tips or pointers where my code might be inadvertently filtering out those classes.
SELECT DISTINCT ?leaf ?parentclasses
WHERE {
GRAPH <>
#a leaf is the lowest level class: A class that does not have a subclass
{
{
{
#I want anything which is a class
{
?leaf rdf:type owl:Class.
}
#i also want the subclass of any superclass if that exists
{
?leaf rdfs:subClassOf ?superclass .
}
#squeezed to specific section of OTL.
filter strstarts(str(?leaf), "URIgoeshere")
#Only keep the results that do not have a preflabel
OPTIONAL {
?leaf skos:prefLabel ?subclasslabel.
}
#make sure the subclasslabel is in dutch
#filter( langMatches(lang(?subclasslabel),"nl") )
#give me the label of the superclass
OPTIONAL {
?superclass skos:prefLabel ?superclasslabel.
}
#make sure it's in dutch
FILTER (lang(?superclasslabel) = "nl")
#if it exists, give me also the superclass of the superclass creating a supersuperclass
{
?superclass rdfs:subClassOf ?supersuperclass.
#give me the label of the supersuperclass
OPTIONAL {
?supersuperclass skos:prefLabel ?supersuperclasslabel.
}
#make sure it's in dutch
FILTER (lang(?supersuperclasslabel) = "nl")
#keep the leafs that are NOT The values whereby the subclass is not empty. (double negative for removing leafs where the subclass has a subclass below it)
FILTER NOT EXISTS {
?subclass rdfs:subClassOf ?leaf
FILTER (?subclass != owl:Nothing )
}
#concatenate the two parentclass variables into one
BIND(concat(str(?superclasslabel), str("-"), str(?supersuperclasslabel) ) as ?parentclasses)
}
}
}
}
}
Here is a ttl file with the same structure as my database: https://file.io/jjwkAWbK4jrF
Below is my end solution to my problem. It was more complicated than I expected, but it works.
The problem was that these classes that didn't have a parent class were not being accepted by the query. With some union cases this could be covered for 0 parent classes, 1 parent class or 2 parent classes.
SELECT DISTINCT ?leaf ?parentclasses
WHERE {
GRAPH <>
#a leaf is the lowest level class: A class that does not have a subclass
{{{{{
#I want anything which is a class
{?leaf rdf:type owl:Class.}
#squeezed
filter strstarts(str(?leaf), "graph")
#keep the leafs that are NOT The values whereby the subclass is not empty.
(double negative for removing leafs where the subclass has a subclass below it)
FILTER NOT EXISTS {?subclass rdfs:subClassOf ?leaf
FILTER (?subclass != owl:Nothing ) }
}
{
{?leaf rdfs:subClassOf ?superclass .}
#grab dutch label if available
optional {
?superclass skos:prefLabel ?superclassnllabel .
filter( langMatches(lang(?superclassnllabel),"nl") )
}
# take either as the label, but dutch over empty
bind( coalesce( ?superclassnllabel, replace(str(?
superclass),"^[^#]*#", "" ) ) as ?superclasslabel )
{
{?superclass rdfs:subClassOf ?supersuperclass.}
#grab dutch label if available
?supersuperclass skos:prefLabel ?supersuperclassnllabel .
filter( langMatches(lang(?supersuperclassnllabel),"nl") )
# take either as the label, but dutch over empty
bind( coalesce( ?supersuperclassnllabel, replace(str(?
supersuperclass),"^[^#]*#", "" ) ) as ?supersuperclasslabel )
BIND(concat(str(?superclasslabel), str(" - "), str(?
supersuperclasslabel) ) as ?parentclasses)
}
union
{
{?superclass ?p ?o.filter(!isblank(?superclass))}
FILTER NOT EXISTS {?superclass rdfs:subClassOf ?supersuperclass}
BIND(concat(str(?superclasslabel), str(" - "), str("Top
Concept") ) as ?parentclasses)
#concatenate the two parentclass variables into one
}
}
}
union
{
#figure this out, WHY IS IT HERE?
{?leaf rdf:type owl:Class .filter(!isblank(?leaf))}
FILTER strstarts(str(?leaf), "graph")
FILTER NOT EXISTS {?leaf rdfs:subClassOf ?superclass}
FILTER NOT EXISTS {?subclass rdfs:subClassOf ?leaf
FILTER (?subclass != owl:Nothing ) }
BIND (str("Top Class") as ?parentclasses )
}
}}}}

Simple SPARQL query does not return any results

I am just getting up and running with Blazegraph in embedded mode. I load a few sample triples and am able to retrieve them with a "select all" query:
SELECT * WHERE { ?s ?p ?o }
This query returns all my sample triples:
[s=<<<http://github.com/jschmidt10#person_Thomas>, <http://github.com/jschmidt10#hasAge>, "30"^^<http://www.w3.org/2001/XMLSchema#int>>>;p=blaze:history:added;o="2017-01-15T16:11:15.909Z"^^<http://www.w3.org/2001/XMLSchema#dateTime>]
[s=<<<http://github.com/jschmidt10#person_Tommy>, <http://github.com/jschmidt10#hasLastName>, "Test">>;p=blaze:history:added;o="2017-01-15T16:11:15.909Z"^^<http://www.w3.org/2001/XMLSchema#dateTime>]
[s=<<<http://github.com/jschmidt10#person_Tommy>, <http://www.w3.org/2002/07/owl#sameAs>, <http://github.com/jschmidt10#person_Thomas>>>;p=blaze:history:added;o="2017-01-15T16:11:15.909Z"^^<http://www.w3.org/2001/XMLSchema#dateTime>]
[s=<http://github.com/jschmidt10#person_Thomas>;p=<http://github.com/jschmidt10#hasAge>;o="30"^^<http://www.w3.org/2001/XMLSchema#int>]
[s=<http://github.com/jschmidt10#person_Tommy>;p=<http://github.com/jschmidt10#hasLastName>;o="Test"]
[s=<http://github.com/jschmidt10#person_Tommy>;p=<http://www.w3.org/2002/07/owl#sameAs>;o=<http://github.com/jschmidt10#person_Thomas>]
Next I try a simple query for a particular subject:
SELECT * WHERE { <http://github.com/jschmidt10#person_Thomas> ?p ?o }
This query yields no results. It seems that none of my queries for a URI are working. I am able to get results when I query for a literal (e.g. ?s ?p "Test").
The API I am using to create my query is BigdataSailRepositoryConnection.prepareQuery().
Code snippet (Scala) that executes and generates the query:
val props = BasicRepositoryProvider.getProperties("./graph.jnl")
val sail = new BigdataSail(props)
val repo = new BigdataSailRepository(sail)
repo.initialize()
val query = "SELECT ?p ?o WHERE { <http://github.com/jschmidt10#person_Thomas> ?p ?o }"
val cxn = repo.getConnection
cxn.begin()
var res = cxn.
prepareTupleQuery(QueryLanguage.SPARQL, query).
evaluate()
while (res.hasNext) println(res.next)
cxn.close()
repo.shutDown()
Have you checked the way you filled the database? You might have characters that are getting encoded strangely, or it looks like you might have excess brackets in your objects.
From the print statement, your URI's are printing extra angled brackets. You are likely using:
val subject = valueFactory.createURI("<http://some.url/some/entity>")
when you should be doing this (without angled brackets):
val subject = valueFactory.createURI("http://some.url/some/entity")

How to parse elements from Sparql algebra

I can get the algebra form from sparql query string using ARQ algebra (com.hp.hpl.jena.sparql.algebra):
String queryStr =
"PREFIX foaf: <http://xmlns.com/foaf/0.1/>" +
"SELECT DISTINCT ?name ?nick" +
"{?x foaf:mbox <mailt:person#server> ." +
"?x foaf:name ?name" +
"OPTIONAL { ?x foaf:nick ?nick }}";
Query query = QueryFactory.create(queryStr);
Op op = Algebra.compile(query);
Print the returned value of op:
(distinct
(project (?name ?nick)
(join
(bgp
(triple ?x <http://xmlns.com/foaf/0.1/mbox> <mailt:person#server>)
(triple ?x <http://xmlns.com/foaf/0.1/name> ?nameOPTIONAL)
)
(bgp (triple ?x <http://xmlns.com/foaf/0.1/nick> ?nick)))))
Returned value is an Op type, but I can't find any direct methods that can parse the op into elements, e.g., basic graph patterns of s, p, o, and the relations between these graph patterns.
Any hint is appreciated, thanks.
Why serialise out the algebra at all?
If your aim is to walk the algebra tree and extract the BGPs then you can do this using the OpVisitor interface of which there are various implementations of that will get you started. The particular method you would care about is visit(OpBgp opBgp) since then you can access the methods of the OpBgp class to extract the pattern information
It might be too late but still as I am not seeing final answer so I am writing the code for printing all BGP.
For that create the class as follows that extends OpVisitorBase and override the public void visit(final OpBGP) function as given below in the code.
And from your code simply call the function:
MyOpVisitorBase.myOpVisitorWalker(op);
public class MyOpVisitorBase extends OpVisitorBase
{
public static void myOpVisitorWalker(Op op)
{
OpWalker.walk(op, this);
}
#Override
public void visit(final OpBGP opBGP) {
final List<Triple> triples = opBGP.getPattern().getList();
int i = 0;
for (final Triple triple : triples) {
System.out.println("Triple: "+triple.toString());
}
}
}

ARQ Making a Query from Scratch

I have a problem in building the query from scratch syntactically or in algebra, based on
https://jena.apache.org/documentation/query/manipulating_sparql_using_arq.html
For example I have the below query
SELECT (count(?instance) AS ?count)
WHERE
{ ?instance <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
<http://data.linkedmdb.org/resource/movie/film> }
(project (?count)
(extend ((?count ?.0))
(group () ((?.0 (count ?instance)))
(bgp (triple ?instance <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://data.linkedmdb.org/resource/movie/film>)))))
Can any one direct me with a sample code of how to build the above query from scratch?
I have tried to build it syntactically but failing to know about how to alias the aggregation above.
If anybody can at least guide me in including aggregation with its aliasing name in projection it will be very great.
I don't typically construct queries through code, since I can just parse a query string, or use a parameterized SPARQL query, but here's a reconstruction of your query using the API. Most of the methods I used here I found by exploring the autocomplete options in Eclipse, and by looking at the Javadoc.
import com.hp.hpl.jena.graph.Node;
import com.hp.hpl.jena.graph.Triple;
import com.hp.hpl.jena.query.Query;
import com.hp.hpl.jena.query.QueryFactory;
import com.hp.hpl.jena.sparql.core.Var;
import com.hp.hpl.jena.sparql.expr.ExprAggregator;
import com.hp.hpl.jena.sparql.expr.ExprVar;
import com.hp.hpl.jena.sparql.expr.aggregate.AggCountVar;
import com.hp.hpl.jena.sparql.syntax.ElementTriplesBlock;
import com.hp.hpl.jena.vocabulary.RDF;
public class QueryBuilding {
public static void main(String[] args) {
// Create the query and make it a SELECT query.
final Query query = QueryFactory.create();
query.setQuerySelectType();
// Set the projection expression.
final ExprVar instance = new ExprVar( "instance" );
query.getProject().add( Var.alloc( "count" ), new ExprAggregator( instance.asVar(), new AggCountVar( instance )));
// Construct the triples pattern and add it.
final ElementTriplesBlock triples = new ElementTriplesBlock();
final Node film = Node.createURI( "http://data.linkedmdb.org/resource/movie/film" );
triples.addTriple( new Triple( instance.getAsNode(), RDF.type.asNode(), film ));
query.setQueryPattern( triples );
// Show the query
System.out.println( query );
}
}
The output (i.e., the printed query) follows. It's the same as your query, modulo some whitespace location and newlines.
SELECT (count(?instance) AS ?count)
WHERE
{ ?instance <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://data.linkedmdb.org/resource/movie/film> .}
Although the solution proposed by Joshua was very helpful to me and produces the correct String output, I have found that it contains a problem; the line :
query.getProject().add( Var.alloc( "count" ), new ExprAggregator( instance.asVar(), new AggCountVar( instance )));
Should be replaced by :
query.getProject().add( Var.alloc( "count" ), query.allocAggregate( new AggCountVar( instance ) ));
Otherwise if you execute the query against a Model you will get an exception "NotAVariableException: Node_variable (not a Var) found"