Using the Wolfram Data Repository for Beginners and Experts

Bob Sandheinrich, Wolfram Research

Join the Conversation #WolframTechConf

Why do we have the Data Repository?

Using data from other repositories is not easy

No standards across datasets

File Formats

Data Structure

No examples

Endless chain of links leading to a pdf

Not designed for the data to ever actually be used, just to meet publication requirements.

The Wolfram Data Repository makes data both available and useful

All data is in WDF ((HyperlinkURL["wolfram.com/data-framework", http://www.wolfram.com/data-framework/]))

Data is easily imported into Wolfram Language

What is the Data Repository

The Wolfram Data Repository is a system for publishing data and making it available in the Wolfram Language for immediate computation.
It includes:

A website

Wolfram Language functions

A Wolfram Cloud-based backend hosting data and APIs

Content! Always increasing

Ways to use the Data Repository

This talk will demonstrate the following escalating use cases:

Browsing the website

Accessing public resources in the Wolfram Language

Creating custom data resources for private use

Submitting data for publication

The Data Repository Website

Data Resources in the Wolfram Language

Getting Data Directly with ResourceData

Copy an example from the website:

Step-by-step analysis example:

rain=ResourceData["Rain Frequency by WMO Station"];
RandomSample[rain,5]

rain[TakeLargestBy["Jan",1]]

rain[TakeLargestBy["Jan",1],"Coordinates"]

months=Keys[rain[1]][[6;;17]]

Normal[rain[TakeLargestBy[#,1],"Coordinates"]]&/@months

Tooltip[GeoMarker[First@Normal[rain[TakeLargestBy[#,1],"Coordinates"]]],#]&/@months

GeoGraphics[Tooltip[GeoMarker[First@Normal[ResourceData["Rain Frequency by WMO Station"][TakeLargestBy[#,1],"Coordinates"]]],#]&/@months]

Using the ResourceObject

resource=ResourceObject["Rain Frequency by WMO Station"]

resource["Properties"]

resource["Categories"]

resource["SourceMetadata"]

resource["ContentElements"]

Get additional data elements

ResourceData[resource,"ColumnDescriptions"]//Dataset

Searching for resources

sampleResources=ResourceSearch["Sample"]

randomSample=RandomChoice[sampleResources]

randomSample["Description"]

data=ResourceData[randomSample]

Creating a data resource

Programmatically

myResource=ResourceObject[<|
"ResourceType"->"Data",
"Name"->"2017 Wolfram Tech Conference Joy",
"Description"->"A precise sequence of minute-by-minute total happiness levels at the Wolfram Technology Conference",
"Content"->RandomReal[100,Round[Quantity[3.5,"Days"]/Quantity[1,"Minutes"]]]
|>]

ListPlot[MovingAverage[ResourceData[myResource],20],Joined->True,Filling->Axis]

With a template notebook

CreateNotebook["DataResource"]

A previously created definition notebook

NotebookOpen[CloudObject["user:bobs/WTCExample/BabeRuth"]]

What to submit to the Data Repository?

Data from academic publications

Well-vetted, generally interesting data

New kinds of sample data, useful for computation

What to include in the submission

Attribution

Rights information

Description

Preparing data

Data should use the Wolfram Data Framework

Quantity

Entity

DateObject

GeoPosition

Aaron Enright’s three part

Next Talk and Q&A

Stay right here for the next talk on the resource system, the technology that enables the data repository

bobs@wolfram.com