RDF, SPARQL and entities

Toni, 2017

Join the Conversation #WolframTechConf

Topics

topicGraph=Graph[
{
Labeled["RDF",Column[{
Style["RDF",Bold],
IRI,
RDFLiteral,
RDFLangString,
RDFBlankNode,
"triples, graphs, datasets"
}]],
Labeled["graph matching",Column[{
Style["graph matching",Bold],
GraphCases,
GraphMatches,
GraphMemberQ,
GraphMatchQ
}]],
Labeled["SPARQL query",Column[{
Style["SPARQL query",Bold],
SPARQLSelect,
SPARQLAsk,
SPARQLConstruct
}]],
Labeled["SPARQL update",Column[{
Style["SPARQL update",Bold],
"insert, delete, load, clear",
"create, drop, copy, move, clear"
}]],
Labeled["entities",Column[{
Style["entities",Bold],
"relational data",
Entity,
EntityProperty
}]],
Labeled["common graph patterns",Column[{
Style["common graph patterns",Bold],
"basic graph patterns",
"group graph patterns",
"filter / condition",
"optional",
"union",
"property paths",
"negation"
}]],
Labeled["SPARQL features",Column[{
Style["SPARQL features",Bold],
"subqueries",
"aggregation",
"assignment",
"graph selection",
"values",
"federation"
}]],
Labeled["RDF entity store",Column[{
Style["RDF entity store",Bold],
"entities backed by RDF data",
"in memory or SPARQL endpoint"
}]],
Labeled["in memory",Column[{
Style["in memory",Bold],
GraphStore,
TripleGraph,
TripleEdge
}]],
Labeled["in database",Column[{
Style["in database",Bold],
"remote or local SPARQL endpoint",
SPARQLExecute
}]],
Labeled["import, export",Column[{
Style["import, export",Bold],
"Turtle",
"RDF/XML",
"N-Triples",
"N-Quads"
}]],
Labeled["property paths",Column[{
Style["property paths",Bold],
"sequence",
"alternatives",
"inverse",
"variable length",
"negated"
}]]
},
{
"SPARQL query"\[UndirectedEdge]"RDF",
"SPARQL query"\[UndirectedEdge]"entities",
"SPARQL query"\[UndirectedEdge]"graph matching",
"RDF"\[UndirectedEdge]"SPARQL update",
"RDF"\[UndirectedEdge]"graph matching",
"SPARQL query"\[UndirectedEdge]"common graph patterns",
"graph matching"\[UndirectedEdge]"common graph patterns",
"RDF entity store"\[UndirectedEdge]"RDF",
"RDF entity store"\[UndirectedEdge]"entities",
"RDF"\[UndirectedEdge]"in memory",
"RDF"\[UndirectedEdge]"in database",
"SPARQL query"\[UndirectedEdge]"SPARQL features",
"RDF"\[UndirectedEdge]"import, export",
"common graph patterns"\[UndirectedEdge]"property paths"
},
ImageSize->Full
]

RDF

Resource Description Framework. W3C standard for describing resources.

Resources

Identifier for “Adrenaline:

IRI["http://www.wikidata.org/entity/Q132621"];

Identifier for the property “label”:

IRI["http://www.w3.org/2000/01/rdf-schema#label"];

Literals:

{
15,
"hello",
RDFLangString["Handtuch","de"],
RDFLiteral["Point(86.925277777 27.988055555)","http://www.opengis.net/ont/geosparql#wktLiteral"]
};

Blank node:

RDFBlankNode["a"];

Statements

Albert Einstein was born in Ulm:

TripleEdge[IRI["http://www.wikidata.org/entity/Q937"],IRI["http://www.wikidata.org/entity/P19"],IRI["http://www.wikidata.org/entity/Q3012"]];

Graphs

Collect statements into a graph:

TripleGraph[{
TripleEdge[IRI["http://www.wikidata.org/entity/Q132621"],IRI["http://www.w3.org/2000/01/rdf-schema#label"],"adrenalin"],
TripleEdge[IRI["http://www.wikidata.org/entity/Q132621"],IRI["http://www.w3.org/1999/02/22-rdf-syntax-ns#type"],IRI["http://www.wikidata.org/entity/Q162657"]]
}]

Datasets

GraphStore[
TripleGraph[{
TripleEdge[IRI["http://example.org/Germany"],IRI["http://example.org/population"],80500000]
}],
<|
IRI["http://example.org/dataset/1986"]->TripleGraph[{
TripleEdge[IRI["http://example.org/Germany"],IRI["http://example.org/population"],78000000]
}]
|>
]

RDF: Recipes

Formulate temporal relations between steps of a food recipe.

Vocabularies

Vocabularies used to describe recipes.

prefixes=<|
"rdf"->"http://www.w3.org/1999/02/22-rdf-syntax-ns#",
"rdfs"->"http://www.w3.org/2000/01/rdf-schema#",
"time"->"http://www.w3.org/2006/time#",
"ex"->"http://example.org/"
|>;

Function[sym,
sym[s_String]:=IRI[prefixes[ToString[sym]]<>s]
]/@{rdf,rdfs,time,ex};

Graph

Each recipe has a number of steps.
Steps can have relations to other steps and a duration.

recipeGraph=TripleGraph[{
TripleEdge[ex["riceWithMushrooms"],rdf["type"],ex["recipe"]],
TripleEdge[ex["riceWithMushrooms"],rdfs["label"],"rice with mushrooms"],

(* steps *)
TripleEdge[ex["riceWithMushrooms"],ex["step"],ex["heatOil"]],
TripleEdge[ex["riceWithMushrooms"],ex["step"],ex["fryMushroom"]],
TripleEdge[ex["riceWithMushrooms"],ex["step"],ex["cookRice"]],
TripleEdge[ex["riceWithMushrooms"],ex["step"],ex["mixRiceAndMushroom"]],

(* temporal relations *)
TripleEdge[ex["fryMushroom"],time["after"],ex["heatOil"]],
TripleEdge[ex["mixRiceAndMushroom"],time["after"],ex["fryMushroom"]],
TripleEdge[ex["mixRiceAndMushroom"],time["after"],ex["cookRice"]],

(* times *)
TripleEdge[ex["cookRice"],time["minutes"],10],
TripleEdge[ex["fryMushroom"],time["minutes"],7]
}]

Export

Export the graph to publish it on the web.

ExportString[recipeGraph,"Turtle","Prefixes"->prefixes]

Verify

ImportString[%,"Turtle"]

IsomorphicGraphQ[%,recipeGraph]

Homework

Build a temporal reasoner that verifies that all time:after and time:before relations in a graph can be satisfied.

SPARQL

W3C standard for querying RDF data.

Select

Returns a list of variable bindings.

Fruit data:

ex[s_]:=IRI["http://example.org/"<>s];

fruitGraph=TripleGraph[{
TripleEdge[ex["banana"],ex["color"],ex["yellow"]],
TripleEdge[ex["strawberry"],ex["color"],ex["red"]],
TripleEdge[ex["strawberry"],ex["shape"],ex["round"]]
}];

Find colors and optionally the shape of fruits:

fruitGraph//SPARQLSelect[{
TripleEdge[fruit,ex["color"],color],
SPARQLOptional[TripleEdge[fruit,ex["shape"],shape]]
}]

Construct

Make a graph from a template.

fruitGraph//SPARQLConstruct[TripleEdge[fruit,ex["color"],color]]

Ask

Check whether the pattern has a match.

Are there yellow fruits?

fruitGraph//SPARQLAsk[TripleEdge[fruit,ex["color"],ex["yellow"]]]

Topics

HighlightGraph[topicGraph,Style[{"RDF","in memory","import, export","SPARQL query"},Green]]

Intermission: Wikidata

What’s that?

wikidata.org

Central data repository

Data of infoboxes in Wikipedia?

Before

Update info boxes in each language edition of Wikipedia.
Different data for population density of Stuttgart in German and Hungarian Wikipedia:

Now

Update data in Wikidata. New data shows up in participating infoboxes.

Stable identifiers

Reference concepts?

Before

Link to potentially unstable Wikipedia pages (Astor Piazzolla or Ástor Piazzolla? Note the thing over the A).
Link to some internal database ID.
Each Wikipedia page stored likes to all other language versions of the same concept. One page rename could required hundreds of edits to restore consistency.

Now

Wikidata ID: Won’t ever change.

Access to data

How to use “the sum of all human knowledge”?

Before

Parse infoboxes and text?
Which page (remember the ID problem)?

Slovak Wikipedia “Kravany nad Dunajom”:

Does that mean “between 106 and 111 meters above see level”?

Now

SPARQL.

For humans: https://query.wikidata.org
For machines: https://query.wikidata.org/sparql

Which data?
- links to Wikipedia and other Wikimedia pages in all languages
- actual data (numbers, formulas, links to images, masses, locations, lengths, ...)
- external identifiers

SPARQL execute

Query a SPARQL endpoint.

Query string

Count the triples in Wikidata:

SPARQLExecute[
"https://query.wikidata.org/sparql",
"select (count(*) as ?count) where {?x ?y ?z}"
]

Symbolic query

Find neurotransmitters and their formula:

SPARQLExecute[
"https://query.wikidata.org/sparql",
SPARQLSelect[{
(* instance of (a subclass of) neurotransmitters *)
PropertyPathPattern[item,{IRI[{"wdt","P31"}],IRI[{"wdt","P279"}]...},IRI[{"wd","Q162657"}]],
(* formula *)
SPARQLOptional[TripleEdge[item,IRI[{"wdt","P274"}],formula]],
(* label *)
TripleEdge[item,IRI[{"rdfs","label"}],label]/;SPARQLEvaluation["lang"][label]=="en"
}->{label,formula}]
]/.RDFLangString[l_,_]:>l

Example: Taxonomy

Whale (Q42196).
Parent taxon (P171).

With[{whaleTaxons=SPARQLExecute[
"https://query.wikidata.org/sparql",
SPARQLSelect[{
TripleEdge[IRI[{"wd","Q42196"}],IRI[{"wdt","P171"}]...,parent],
SPARQLOptional[TripleEdge[parent,IRI[{"wdt","P171"}],linkTo]],
SPARQLService[IRI[{"wikibase","label"}],TripleEdge[IRI[{"bd","serviceParam"}],IRI[{"wikibase","language"}],"en"]]
}->{parent,parentLabel,linkTo}]
]},
Graph[
Labeled[#[parent],First[#[parentLabel]]]&/@whaleTaxons,
If[KeyExistsQ[#,linkTo],#[parent]->#[linkTo],Nothing]&/@whaleTaxons
]
]

Example: Events

Which university was Albert Einstein at when Halley’s Comet passed near earth?

SPARQLExecute[
"https://query.wikidata.org/sparql",
SPARQLSelect[{
(* Albert Einstein's times at universities *)
TripleEdge[IRI[{"wd","Q937"}],IRI[{"p","P108"}],employerStatement],
TripleEdge[employerStatement,IRI[{"ps","P108"}],university],
TripleEdge[university,IRI[{"wdt","P31"}],IRI[{"wd","Q3918"}]],
TripleEdge[employerStatement,IRI[{"pq","P580"}],start],
TripleEdge[employerStatement,IRI[{"pq","P582"}],end],

(* Halley's Comet passes near earth *)
PropertyPathPattern[IRI[{"wd","Q23054"}],{IRI[{"p","P793"}],IRI[{"pq","P585"}]},halleyDate],

SPARQLFilter[start<halleyDate&&halleyDate<end],

SPARQLService[IRI[{"wikibase","label"}],TripleEdge[IRI[{"bd","serviceParam"}],IRI[{"wikibase","language"}],"en"]]
}->{universityLabel,start,end,halleyDate}]
]

SPARQL for built-in entities

Query entities using SPARQL.

Basic

Data retrieval: For all countries retrieve population and area.

Entity["Country"]//SPARQLSelect[{
TripleEdge[country,"Population",population],
TripleEdge[country,"Area",area]
},"Limit"->3]

Reverse lookup: Movies starring Bud Spencer

Entity["Movie"]//SPARQLSelect[TripleEdge[movie,(EntityProperty["cast", EntityProperty["Movie","Cast"], "EntityProperty[\"Movie\", \"Cast\"]"]),(Entity["Bud Spencer", Entity["Person","BudSpencer::hg42g"], "Entity[\"Person\", \"BudSpencer::hg42g\"]", "person"])]]

Filter: High mountains in Italy.

Entity["Mountain"]//SPARQLSelect[{
TripleEdge[mountain,"Countries",(Entity["Italy", Entity["Country","Italy"], "Entity[\"Country\", \"Italy\"]", "country"])],
TripleEdge[mountain,"Elevation",elevation]/;elevation>(Quantity[4.6, "km", kilometers, "Kilometers"])
}]

Optional: Images.

Entity["Movie"]//SPARQLSelect[{
TripleEdge[movie,(EntityProperty["cast", EntityProperty["Movie","Cast"], "EntityProperty[\"Movie\", \"Cast\"]"]),(Entity["Bud Spencer", Entity["Person","BudSpencer::hg42g"], "Entity[\"Person\", \"BudSpencer::hg42g\"]", "person"])],
SPARQLOptional[TripleEdge[movie,(EntityProperty["image", EntityProperty["Movie","Image"], "EntityProperty[\"Movie\", \"Image\"]"]),image]]
}]

Not so basic

Property chain across multiple domains: Populations of capitals of countries bordering Germany.

{Entity["Country"],Entity["City"]}//SPARQLSelect[{
TripleEdge[(Entity["Germany", Entity["Country","Germany"], "Entity[\"Country\", \"Germany\"]", "country"]),"BorderingCountries",country],
TripleEdge[country,"CapitalCity",city],
TripleEdge[city,"Population",population]
}]

Aggregation: Largest countries in Europe by number of bordering countries.

(EntityClass["Europe", EntityClass["Country","Europe"], "EntityClass[\"Country\", \"Europe\"]", "countries"])//SPARQLSelect[
TripleEdge[country,"BorderingCountries",borderingCountry]->{
country,
SPARQLEvaluation["COUNT"][]->count
},
"GroupBy"->country,
"OrderBy"->SPARQLEvaluation["COUNT"][]->"Descending",
"Limit"->5
]

Aggregation and filter: Buildings in more than one country.

Entity["Building"]//SPARQLSelect[
TripleEdge[building,"Countries",country]->{
building,
SPARQLEvaluation["COUNT"][]->count
},
"GroupBy"->building,
"Having"->SPARQLEvaluation["COUNT"][]>1,
"Limit"->3
]

Crazy

Subqueries: Longest rivers by geographic region.

{Entity["GeographicRegion"],Entity["River"]}//SPARQLSelect[{
SPARQLSelect[
{
TripleEdge[continent,(EntityProperty["countries", EntityProperty["GeographicRegion","Countries"], "EntityProperty[\"GeographicRegion\", \"Countries\"]"]),country],
TripleEdge[river,(EntityProperty["countries", EntityProperty["River","Countries"], "EntityProperty[\"River\", \"Countries\"]"]),country],
TripleEdge[river,(EntityProperty["length", EntityProperty["River","Length"], "EntityProperty[\"River\", \"Length\"]"]),length]
}->{
continent,
SPARQLEvaluation["MAX"][length]->length
},
"GroupBy"->continent
],
TripleEdge[river,(EntityProperty["length", EntityProperty["River","Length"], "EntityProperty[\"River\", \"Length\"]"]),length]
},"OrderBy"->length->"Descending"]//N

Can you think of a simpler version of this query?

Topics

HighlightGraph[topicGraph,Style[{"RDF","in memory","import, export","SPARQL query","in database","entities"},Green]]

Graph matching

Inspired by SPARQL.

GraphCases

//GraphCases[Graph[{x_->y_,y_->z_,z_->x_}]/;x<y<z]//Map[Graph[#,VertexLabels->"Name"]&]

//GraphCases[Graph[{x_->y_,y_->z_,z_->x_}]/;x<y<z:>x+y+z]

Find all path connecting 1 and 200:

TripleGraph[{
TripleEdge[1,2,3],
TripleEdge[3,4,5],
TripleEdge[3,8,9],
TripleEdge[9,100,200],
TripleEdge[1,77,200]
}]//GraphCases[TripleGraph[{PropertyPathPattern[1,path_,200]}]:>path]

GraphMatches

//GraphMatches[Graph[{x_->y_,y_->z_,z_->x_}]/;x<y<z]

Other functions

{GraphMemberQ,GraphMatchQ};

SPARQL update

W3C standard for updating graph stores.

Insert data

Insert explicit data.

Fruit data:

ex[s_]:=IRI["http://example.org/"<>s];

fruitGraph=TripleGraph[{
TripleEdge[ex["banana"],ex["color"],"yellow"],
TripleEdge[ex["strawberry"],ex["color"],"red"]
}];

Insert additional data:

fruitGraph//SPARQLInsertData[{
TripleEdge[ex["kiwi"],ex["color"],"green"],
TripleEdge[ex["apricot"],ex["color"],"orange"]
}]

Insert

Insert data based on a templated.

Specify a graph containing data about geometric objects:

geoGraph=TripleGraph[{
TripleEdge[ex["square"],ex["edgeLength"],1],
TripleEdge[ex["square"],ex["area"],1],
TripleEdge[ex["disk"],ex["radius"],1],
TripleEdge[ex["disk"],ex["area"],3.14]
}];

Add the diameter to object that have a radius:

geoGraph//SPARQLInsert[{
TripleEdge[obj,ex["radius"],r],
2r->d
}->TripleEdge[obj,ex["diameter"],d]]

Look up diameters:

%//SPARQLSelect[TripleEdge[obj,ex["diameter"],d]]

Local SPARQL endpoint: Project Rephetio

Study relationships between drugs and diseases.
From “Systematic integration of biomedical knowledge prioritizes drugs for repurposing”, Daniel Scott Himmelstein et al.

Setup

Run this in a terminal to start the server:

CopyToClipboard[StringRiffle[{
"cd "<>ToString[NotebookDirectory[],InputForm],
"java -server -Xmx4g -jar blazegraph.jar"
},"\n"]]

bg="http://localhost:9999/blazegraph/sparql";

Clear the database:

SPARQLExecute[bg,SPARQLClear[All]];

Load data, combined from 29 public resources:

SPARQLExecute[bg,SPARQLLoad[File[FileNameJoin[{NotebookDirectory[],"data.nt"}]]]];

SPARQLExecute[bg,SPARQLSelect[TripleEdge[x,y,z]->SPARQLEvaluation["count"][]->count]]

Prefixes used for queries:

prefixes=<|
"rdf"->"http://www.w3.org/1999/02/22-rdf-syntax-ns#",
"rdfs"->"http://www.w3.org/2000/01/rdf-schema#",
"ex"->"http://example.org/"
|>;

Look up known treatments

Some examples of known treatments:

SPARQLExecute[bg,SPARQLSelect[
{
TripleEdge[compound,IRI[{"ex","treats"}],disease],
TripleEdge[compound,IRI[{"rdfs","label"}],compoundLabel],
TripleEdge[disease,IRI[{"rdfs","label"}],diseaseLabel]
}->{compoundLabel,diseaseLabel},
"Prefixes"->prefixes,
"Limit"->3
]]

Total number of treatments:

SPARQLExecute[bg,SPARQLSelect[
TripleEdge[compound,IRI[{"ex","treats"}],disease]->SPARQLEvaluation["count"][]->count,
"Prefixes"->prefixes
]]

Total number of compound-disease pairs:

SPARQLExecute[bg,SPARQLSelect[
{
TripleEdge[compound,IRI[{"rdf","type"}],IRI[{"ex","Compound"}]],
TripleEdge[disease,IRI[{"rdf","type"}],IRI[{"ex","Disease"}]]
}->SPARQLEvaluation["count"][]->count,
"Prefixes"->prefixes
]]

Paths

Find possible paths of type upregulates-interacts-associates between a given compound-disease pair:

SPARQLExecute[bg,SPARQLSelect[
{
TripleEdge[IRI[{"ex","UBERON:0000178"}],IRI[{"ex","upregulates"}],gene1],
TripleEdge[gene1,IRI[{"ex","interacts"}],gene2],
TripleEdge[IRI[{"ex","DOID:2531"}],IRI[{"ex","associates"}],gene2]
},
"Limit"->3,
"Prefixes"->prefixes
]]

Path counts

For all compound-disease pairs, count the number of path of this particular type:

SPARQLExecute[bg,SPARQLSelect[
PropertyPathPattern[compound,{
IRI[{"ex","upregulates"}],(* a gene *)
IRI[{"ex","interacts"}], (* with another gene *)
InverseProperty[IRI[{"ex","associates"}]]
},disease]->{compound,disease,SPARQLEvaluation["count"][]->count},
"GroupBy"->{compound,disease},
"OrderBy"->count->"Descending",
"Limit"->4,
"Prefixes"->prefixes
]]

Predict

Homework 1: Generate all possible path types of a given length between compounds and diseases.
Homework 2: Based on counts for different path types, build a classifier that predicts whether a given compound might cure some disease.

Example: Chemical structure

Terms

type=IRI["http://www.w3.org/1999/02/22-rdf-syntax-ns#type"];
subClassOf=IRI["http://www.w3.org/2000/01/rdf-schema#subClassOf"];
hasPart=IRI["http://purl.org/dc/terms/hasPart"];
hasBond=IRI["http://www.example.org/chemistry/hasBond"];

Vocabulary

vocabulary=TripleGraph[{
(* top concepts *)
TripleEdge[IRI["Atom"],type,IRI["ChemicalSubstructure"]],
TripleEdge[IRI["Bond"],type,IRI["ChemicalSubstructure"]],

(* atoms *)
TripleEdge[IRI["Carbon"],type,IRI["Atom"]],
TripleEdge[IRI["Hydrogen"],type,IRI["Atom"]],
TripleEdge[IRI["Oxygen"],type,IRI["Atom"]],

(* bonds *)
TripleEdge[IRI["SingleBond"],subClassOf,IRI["Bond"]],
TripleEdge[IRI["DoubleBond"],subClassOf,IRI["Bond"]],
TripleEdge[IRI["CovalentBond"],subClassOf,IRI["Bond"]]
}]

Data

data=TripleGraph[{
(* --- water --- *)
TripleEdge[IRI["Water"],type,IRI["ChemicalStructure"]],

(* atoms *)
TripleEdge[IRI["Water"],hasPart,RDFBlankNode["1"]],
TripleEdge[RDFBlankNode["1"],type,IRI["Hydrogen"]],
TripleEdge[IRI["Water"],hasPart,RDFBlankNode["2"]],
TripleEdge[RDFBlankNode["2"],type,IRI["Oxygen"]],
TripleEdge[IRI["Water"],hasPart,RDFBlankNode["3"]],
TripleEdge[RDFBlankNode["3"],type,IRI["Hydrogen"]],

(* bonds *)
TripleEdge[IRI["Water"],hasPart,RDFBlankNode["4"]],
TripleEdge[RDFBlankNode["4"],type,IRI["SingleBond"]],
TripleEdge[RDFBlankNode["4"],type,IRI["CovalentBond"]],
TripleEdge[IRI["Water"],hasPart,RDFBlankNode["5"]],
TripleEdge[RDFBlankNode["5"],type,IRI["SingleBond"]],
TripleEdge[RDFBlankNode["5"],type,IRI["CovalentBond"]],

(* graph *)
TripleEdge[RDFBlankNode["1"],hasBond,RDFBlankNode["4"]],
TripleEdge[RDFBlankNode["2"],hasBond,RDFBlankNode["4"]],
TripleEdge[RDFBlankNode["2"],hasBond,RDFBlankNode["5"]],
TripleEdge[RDFBlankNode["3"],hasBond,RDFBlankNode["5"]],


(* --- carbon dioxide --- *)
TripleEdge[IRI["CarbonDioxide"],type,IRI["ChemicalStructure"]],

(* atoms *)
TripleEdge[IRI["CarbonDioxide"],hasPart,RDFBlankNode["6"]],
TripleEdge[RDFBlankNode["6"],type,IRI["Oxygen"]],
TripleEdge[IRI["CarbonDioxide"],hasPart,RDFBlankNode["7"]],
TripleEdge[RDFBlankNode["7"],type,IRI["Carbon"]],
TripleEdge[IRI["CarbonDioxide"],hasPart,RDFBlankNode["8"]],
TripleEdge[RDFBlankNode["8"],type,IRI["Oxygen"]],

(* bonds *)
TripleEdge[IRI["CarbonDioxide"],hasPart,RDFBlankNode["9"]],
TripleEdge[RDFBlankNode["9"],type,IRI["DoubleBond"]],
TripleEdge[RDFBlankNode["9"],type,IRI["CovalentBond"]],
TripleEdge[IRI["CarbonDioxide"],hasPart,RDFBlankNode["10"]],
TripleEdge[RDFBlankNode["10"],type,IRI["DoubleBond"]],
TripleEdge[RDFBlankNode["10"],type,IRI["CovalentBond"]],

(* graph *)
TripleEdge[RDFBlankNode["6"],hasBond,RDFBlankNode["9"]],
TripleEdge[RDFBlankNode["7"],hasBond,RDFBlankNode["9"]],
TripleEdge[RDFBlankNode["7"],hasBond,RDFBlankNode["10"]],
TripleEdge[RDFBlankNode["8"],hasBond,RDFBlankNode["10"]]
}]

Merge

allData=RDFMerge[{vocabulary,data}]

Queries

All chemical structures:

allData//SPARQLSelect[TripleEdge[chem,type,IRI["ChemicalStructure"]]]

Chemical structures containing carbon:

allData//SPARQLSelect[{
TripleEdge[chem,type,IRI["ChemicalStructure"]],
TripleEdge[chem,hasPart,atom],
TripleEdge[atom,type,IRI["Carbon"]]
}->chem]

Chemical structures containing a single bond:

allData//SPARQLSelect[{
TripleEdge[chem,type,IRI["ChemicalStructure"]],
TripleEdge[chem,hasPart,bond],
TripleEdge[bond,type,IRI["SingleBond"]]
}->chem,"Distinct"->True]

Chemical structures containing any type of bond between a carbon and an oxygen atom:

copatt={
TripleEdge[chem,type,IRI["ChemicalStructure"]],
TripleEdge[chem,hasPart,c],
TripleEdge[c,type,IRI["Carbon"]],
TripleEdge[chem,hasPart,o],
TripleEdge[o,type,IRI["Oxygen"]],
TripleEdge[c,hasBond,b],
TripleEdge[o,hasBond,b],
PropertyPathPattern[b,{type,subClassOf...},IRI["Bond"]]
};

allData//SPARQLSelect[copatt->chem,"Distinct"->True]

Visualize the graph pattern:

UndirectedEdge@@@Subsets[copatt,{2}]//Select[Map[Cases[_Symbol]]/*Apply[IntersectingQ]]//Graph[#,VertexLabels->"Name"]&

SPARQL for custom entities

Each property links to a list of statements.
Each statement links to value and a source.

Entity store

store = EntityStore[{
	"researcher" -> <|
		"Entities" -> {"albert", "celsius", "marie", "gru", "werner"}
	|>,
	"article" -> <|
		"Entities" -> <|
			"a1" -> <|
				"Label" -> "Definition of a temperature scale",
				"Authors" -> {Entity["researcher", "celsius"]},
				"Date" -> 1840
			|>,
			"a2" -> <|
				"Label" -> "On the art of boiling water",
				"Authors" -> {Entity["researcher", "gru"]},
				"Date" -> 2013
			|>,
			"a3" -> <|
				"Label" -> "Temperature",
				"Authors" -> {Entity["researcher", "albert"], Entity["researcher", "marie"]},
				"Date" -> 1950
			|>
		|>
	|>,
	
	"statement" -> <|
		"Entities" -> <|
			"s1" -> <|"Value" -> (Quantity[100, "°C", degrees Celsius, "DegreesCelsius"]), "Reference" -> Entity["article", "a1"]|>,
			"s2" -> <|"Value" -> (Quantity[77, "°C", degrees Celsius, "DegreesCelsius"]), "Reference" -> Entity["article", "a2"]|>,
			"s3" -> <|"Value" -> (Quantity[100, "°C", degrees Celsius, "DegreesCelsius"]), "Reference" -> Entity["article", "a3"]|>,
			"s4" -> <|"Value" -> (Quantity[4, "km""/""h", kilometers per hour, (("Kilometers")/("Hours"))])|>,
			"s5" -> <|"Value" -> (Quantity[3, "km""/""h", kilometers per hour, (("Kilometers")/("Hours"))]), "Reference" -> Entity["article", "a2"]|>
		|>
	|>,
	
	"chemical" -> <|
		"Entities" -> <|
			"water" -> <|
				"BoilingTemperature" -> {Entity["statement", "s1"], Entity["statement", "s2"], Entity["statement", "s3"]}
			|>
		|>
	|>,
	"animal" -> <|
		"Entities" -> <|
			"dog" -> <|
				"WalkingSpeed" -> {Entity["statement", "s4"], Entity["statement", "s5"]}
			|>
		|>
	|>
}];

$EntityStores={store};

Entity queries

Tricky to navigate the entity -> statement -> value chain.

Dog

Entity["animal","dog"]["WalkingSpeed"]

EntityValue[%,{"Value","Reference"}]

Water

Entity["chemical","water"]["BoilingTemperature"]

EntityValue[%,"Value"]

EntityValue[%%,"Reference"]

EntityValue[%,"Authors"]

SPARQL queries

allTypes=Entity/@store[]

Dog

allTypes//SPARQLSelect[{
TripleEdge[(Entity["dog", Entity["animal","dog"], "Entity[\"animal\", \"dog\"]", "animal"]),"WalkingSpeed",statement],
TripleEdge[statement,"Value",value]
}->value]

Only sourced values:

allTypes//SPARQLSelect[{
TripleEdge[(Entity["dog", Entity["animal","dog"], "Entity[\"animal\", \"dog\"]", "animal"]),"WalkingSpeed",statement],
TripleEdge[statement,"Value",value],
SPARQLFilter[SPARQLExists[TripleEdge[statement,"Reference",source]]]
}->value]

Gru

All values reported by Gru:

allTypes//SPARQLSelect[{
TripleEdge[entity,property,statement],
TripleEdge[statement,"Value",value],
TripleEdge[statement,"Reference",article],
TripleEdge[article,"Authors",Entity["researcher","gru"]]
}->{entity,property,value}]

Average values, part 1

For all entity-property pairs, find the average value:

allTypes//SPARQLSelect[
{
TripleEdge[entity,property,statement],
TripleEdge[statement,"Value",value]
}->{
entity,
property,
SPARQLEvaluation[Flatten/*Mean/*N][value]->value
},
"GroupBy"->{entity,property}
]

Latest value

Latest reported boiling temperature of water:

allTypes//SPARQLSelect[{
TripleEdge[(Entity["water", Entity["chemical","water"], "Entity[\"chemical\", \"water\"]", "chemical"]),"BoilingTemperature",statement],
TripleEdge[statement,"Value",value],
PropertyPathPattern[statement,{"Reference","Date"},date]
}->value,"OrderBy"->{date->"Descending"},"Limit"->1]

Outlier

Find authors who report values that deviate from the trimmed mean by more than (QuantityPostfix[20, "%", percent, "Percent"]):

allTypes//SPARQLSelect[{
SPARQLSelect[
{
TripleEdge[entity,property,statement],
TripleEdge[statement,"Value",value]
}->{
entity,
property,
SPARQLEvaluation[Flatten/*(TrimmedMean[#,0.4]&)][value]->mean
},
"GroupBy"->{entity,property}
],

TripleEdge[entity,property,statement],
TripleEdge[statement,"Value",value]/;Abs[(mean-value)/mean]>(QuantityPostfix[20, "%", percent, "Percent"]),
PropertyPathPattern[statement,{"Reference","Authors"},author]
}->{entity,property,mean,value,author}]

Average values, part 2

Query for suspicious authors:

suspiciousAuthorQuery=SPARQLSelect[{
SPARQLSelect[
{
TripleEdge[entity,property,statement],
TripleEdge[statement,"Value",value]
}->{
entity,
property,
SPARQLEvaluation[Flatten/*(TrimmedMean[#,0.4]&)][value]->mean
},
"GroupBy"->{entity,property}
],

TripleEdge[entity,property,statement],
TripleEdge[statement,"Value",value]/;Abs[(mean-value)/mean]>(QuantityPostfix[20, "%", percent, "Percent"]),
PropertyPathPattern[statement,{"Reference","Authors"},suspiciousAuthor]
}->suspiciousAuthor];

allTypes//suspiciousAuthorQuery

The mean, excluding values by authors who have reported deviating values:

allTypes//SPARQLSelect[
{
suspiciousAuthorQuery,

TripleEdge[entity,property,statement],
TripleEdge[statement,"Value",value],
SPARQLFilter[Not[SPARQLExists[PropertyPathPattern[statement,{"Reference","Authors"},suspiciousAuthor]]]]
}->{
entity,
property,
SPARQLEvaluation[Flatten/*Mean][value]->value
},
"GroupBy"->{entity,property}
]

Topics

HighlightGraph[topicGraph,Style[{"RDF","in memory","import, export","SPARQL query","in database","entities","graph matching","SPARQL features","property paths","SPARQL update","common graph patterns"},Green]]

RDF entity store

Rivers and lakes in Spain.
All data provided by Wikidata on the fly.

Setup

Create an RDF entity store:
- for each type declare an “EntityFilter”
- for properties specify how to retrieve and process data
- for properties specify how to handle implicit entity class queries

Common properties:

$commonProperties = <|
	(* label *)
	"Label" -> <|
		"Label" -> "label",
		"Template" -> Function[evar, SPARQLSelect[{
			TripleEdge[evar, IRI[{"rdfs", "label"}], label],
			SPARQLFilter[SPARQLEvaluation["langMatches"][SPARQLEvaluation["lang"][label], "es"]]
		}]],
		"ValueProcessingFunction" -> Replace[{
			{___, KeyValuePattern[label -> RDFLangString[l_, "es"]], ___} :> l,
			_ :> Missing["NotAvailable"]
		}],
		"FilterHandler" -> Function[{evar, f}, f // Replace[{
			StringStartsQ[s_String] :> SPARQLSelect[{
				TripleEdge[evar, IRI[{"rdfs", "label"}], label],
				SPARQLFilter[SPARQLEvaluation["langMatches"][SPARQLEvaluation["lang"][label], "es"]],
				SPARQLFilter[SPARQLEvaluation["STRSTARTS"][label, s]]
			} -> evar, "Distinct" -> True]
		}]]
	|>,
	(* image *)
	"Image" -> <|
		"Label" -> "image",
		"Template" -> Function[evar, TripleEdge[evar, IRI[{"wdt", "P18"}], image]],
		"ValueProcessingFunction" -> Replace[{
			{___, KeyValuePattern[image -> IRI[i_]], ___} :> Thumbnail[i],
			_ :> Missing["NotAvailable"]
		}]
	|>,
	(* position *)
	"Position" -> <|
		"Label" -> "position",
		"Template" -> Function[evar, TripleEdge[evar, IRI[{"wdt", "P625"}], loc]],
		"ValueProcessingFunction" -> Replace[{
			{___, KeyValuePattern[loc -> RDFLiteral[l_, "http://www.opengis.net/ont/geosparql#wktLiteral"]], ___} :> GeoPosition[Reverse[Interpreter["Number"][StringSplit[StringTake[l, {7, -2}]]]]],
			_ :> Missing["NotAvailable"]
		}],
		"FilterHandler" -> Function[{evar, f}, f // Replace[{
			GeoDisk[GeoPosition[{lat_, lon_}], r_?QuantityQ] :> SPARQLSelect[{
				(* https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual#Search_around_point *)
				SPARQLService[IRI[{"wikibase", "around"}], {
					TripleEdge[evar, IRI[{"wdt", "P625"}], location],
					(* https://www.mediawiki.org/wiki/Wikibase/Indexing/RDF_Dump_Format#Globe_coordinate *)
					TripleEdge[IRI[{"bd", "serviceParam"}], IRI[{"wikibase", "center"}], RDFLiteral[StringTemplate["Point(`` ``)"][lon, lat], IRI[{"geo", "wktLiteral"}]]],
					TripleEdge[IRI[{"bd", "serviceParam"}], IRI[{"wikibase", "radius"}], QuantityMagnitude[r, "Kilometers"]]
				}]
			} -> evar, "Distinct" -> True]
		}]]
	|>
|>;

The store:

store = RDFEntityStore[
	{
		(* lake *)
		"lake" -> <|
			"EntityFilter" -> Function[evar, SPARQLSelect[{
				PropertyPathPattern[evar, {IRI[{"wdt", "P279"}] ..., IRI[{"wdt", "P31"}]}, IRI[{"wd", "Q23397"}]], (* lake *)
				TripleEdge[evar, IRI[{"wdt", "P17"}], IRI[{"wd", "Q29"}]] (* Spain *)
			}]],
			"Properties" -> Join[$commonProperties, <|
				(* area *)
				"Area" -> <|
					"Label" -> "area",
					"Template" -> Function[evar, PropertyPathPattern[evar, {IRI[{"p", "P2046"}], IRI[{"psn", "P2046"}], IRI[{"wikibase", "quantityAmount"}]}, areaSquareMeters]],
					"ValueProcessingFunction" -> Replace[{
						{___, KeyValuePattern[areaSquareMeters -> area_], ___} :> Quantity[area, "Meters"^2],
						_ :> Missing["NotAvailable"]
					}]
				|>
			|>]
		|>,
		(* river *)
		"river" -> <|
			"EntityFilter" -> Function[evar, SPARQLSelect[{
				PropertyPathPattern[evar, {IRI[{"wdt", "P279"}] ..., IRI[{"wdt", "P31"}]}, IRI[{"wd", "Q4022"}]], (* river *)
				TripleEdge[evar, IRI[{"wdt", "P17"}], IRI[{"wd", "Q29"}]] (* Spain *)
			}]],
			"Properties" -> Join[$commonProperties, <|
				(* length *)
				"Length" -> <|
					"Label" -> "length",
					"Template" -> Function[evar, PropertyPathPattern[evar, {IRI[{"p", "P2043"}], IRI[{"psn", "P2043"}], IRI[{"wikibase", "quantityAmount"}]}, lengthMeters]],
					"ValueProcessingFunction" -> Replace[{
						{___, KeyValuePattern[lengthMeters -> l_], ___} :> Quantity[l, "Meters"],
						_ :> Missing["NotAvailable"]
					}],
					"FilterHandler" -> Function[{evar, f}, f // Replace[{
						op_[q_?QuantityQ] :> SPARQLSelect[{
							PropertyPathPattern[evar, {IRI[{"p", "P2043"}], IRI[{"psn", "P2043"}], IRI[{"wikibase", "quantityAmount"}]}, lengthMeters],
							SPARQLFilter[op[QuantityMagnitude[q, "Meters"]][lengthMeters]]
						} -> evar, "Distinct" -> True]
					}]]
				|>
			|>]
		|>
	},
	"https://query.wikidata.org/sparql"
];

Register the store:

$EntityStores={store};

Examples

Discover entities:

EntityList["lake"]//Length

Basic data:

(Entity["Tajo", Entity["river","http://www.wikidata.org/entity/Q14294"], "Entity[\"river\", \"http://www.wikidata.org/entity/Q14294\"]", "river"])["Length"]

(Entity["Tajo", Entity["river","http://www.wikidata.org/entity/Q14294"], "Entity[\"river\", \"http://www.wikidata.org/entity/Q14294\"]", "river"])["Image"]

Long rivers:

EntityList[EntityClass["river","Length"->GreaterThan[(Quantity[900, "km", kilometers, "Kilometers"])]]]

River in some geo disc:

GeoGraphics[GeoDisk[GeoPosition[{39.778,-5.613}],(Quantity[30, "km", kilometers, "Kilometers"])]]

EntityList[EntityClass["river","Position"->GeoDisk[GeoPosition[{39.778,-5.613}],(Quantity[30, "km", kilometers, "Kilometers"])]]]

Find and highlight lakes in some region:

EntityList[EntityClass["lake","Position"->GeoDisk[GeoPosition[{40,-5.1}],(Quantity[190, "km", kilometers, "Kilometers"])]]]

GeoGraphics[GeoMarker[%]]

A dataset for a lake:

(Entity["Mar Menor", Entity["lake","http://www.wikidata.org/entity/Q1553876"], "Entity[\"lake\", \"http://www.wikidata.org/entity/Q1553876\"]", "lake"])["Dataset"]

All data about long rivers:

EntityClass["river","Length"->GreaterThan[(Quantity[850, "km", kilometers, "Kilometers"])]]["Dataset"]

Thanks.

HighlightGraph[topicGraph,Style[{"RDF","in memory","import, export","SPARQL query","in database","entities","graph matching","SPARQL features","property paths","SPARQL update","common graph patterns","RDF entity store"},Green]]