Data Management for Machine Learning

Meghan Rieu-Werden, Wolfram Research

Join the Conversation #WolframTechConf

Outline

In this talk, you will:

Get a high level view of the data management solutions chosen for the machine learning team

Hear the rationale for such choices

Tour the built in Wolfram Language functionalities that allow you to easily interface with external data storage systems:

MongoLink

ExternalStorage

AWSLink

Data Storage: MongoDB

Flat file system vs. Database system

Queryable

Large datasets will not fit into RAM

RDMS vs. NoSQL

Schema less

Number of fields, content and size of the document can differ from one document to another

MongoCursorToArray@MongoCollectionAggregate[coll,<|"$sample"-><|"size"-> 2|>|>]

Validation rules can enforce a schema

Document structure is compatible with that of a nested Association in WL

{ "_id" : ObjectId("56b88a74d682ee08887278e8"), "Class" : "1st", "Age" : 71, "Sex" : "male", "Survived" : false }

Document Oriented Storage

Data is stored in the form of JSON style documents

Ease of scale-out

Document structure

Replica sets

Sharding

MongoDB

Popularity

https://db-engines.com/en/ranking_trend/document+store

Robust

Easy to get help

File Storage: AWS S3

Database file system vs. Object storage system

Flexibility

Database file systems exits, both in RDMS and MongoDB, however, these solutions require more capacity planning and monitoring

Dataset and File size

Size of all datasets combined could be on the scale of tens of terabytes

Size of individual files makes data transfer more difficult: https://smerity.com/articles/2015/the_small_file _problem.html

Data transfer

Solution for transferring large volumes of small files

Data transfer to EC2 is especially fast, and its free

Distribution

Geographic distribution for greater availability and redundancy

Duplication

Naming files by a hash of their contents eliminates duplication

Training, Data Ingestion, Benchmarking, etc: AWS EC2

Own equipment vs. Cloud service

Training Demand

Highly variable

Multiple Users

Configuration

Each user can control their own machine

Hardware Trends

MongoLink

Available in versions 11.3+

<<MongoLink`

Prerequisite: 
A MongoDB instance must be installed and running. For more information, see https://docs.mongodb.com/manual/installation/

Create a client connection:

mongoclient=MongoConnect[<|"Host"->"mldatadb.wolfram.com",
      "Username"->"wtc2018", "Password"-> "$Prompt",
       "CAFile"->$MongoDefaultCAFile|>]

View all databases available through the client:

Sort@Keys@mongoclient

Connect to a database:

db=mongoclient["MachineLearningData"]

View all collections within that database:

Sort@Keys@db

Connect to the metadata collection:

metadatacoll=db["_metadata"]

View the metadata for the CIFAR10 collection

Dataset@MongoCollectionFindOne[metadatacoll,<|"DatasetName"->"CIFAR10"|>]

Connect to the CIFAR10 collection:

coll=db["CIFAR10"]

View a random sample of documents:

MongoCursorToArray@MongoCollectionAggregate[coll,<|"$sample"-><|"size"-> 5|>|>]

Connect to the files collection:

filescoll=db["_files"]

View file metadata for the above file:

MongoCollectionFindOne[filescoll,<|"_id"->"bbc1545b02dfdbef440acaec1bb159a3.png"|>]

Write query for your script:

data=MongoCursorToArray@MongoCollectionFind[coll,<|"ValidationRole"->"training"|>,<|"_id"->False,"ValidationRole"->False|>]

ExternalStorage

Multithreaded downloader using the Amazon C++ SDK

Available in versions 12.0+

Currently internal functionality:

no official documentation available yet

subject to change

to be exposed via ServiceConnect in future

Quiet[<<ExternalStorage`]

Prerequisite: 
An AWS account is required, and AWS credentials must be configured. For more information, see https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html

Create a client connection:

s3client=S3Connect[]

Using the query we created for a script, get the filenames:

data=MongoCursorToArray[MongoCollectionFind[coll,<|"ValidationRole"->"training"|>,<|"_id"->False,"ValidationRole"->False|>]]

filenames=RandomSample[data[[All,"File"]],5]

Download the selected files:

files=S3Download["arg-mldata",filenames];

View the files:

Import/@files

AWSLink

Available in versions 11.3+

Currently internal functionality:

no official documentation available yet

subject to change

Quiet[<<AWSLink`]

Prerequisite: 
An AWS account is required, and AWS credentials must be configured. For more information, see https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html

Create a client connection:

ec2client = AWSClientConnect["ec2"]

View available images (AMI’s):

Dataset@EC2DescribeImages[ec2client]

View available security groups:

EC2DescribeSecurityGroups[ec2client]

View available key pairs:

Dataset@EC2DescribeKeyPairs[ec2client]

View the names of available instance types. For more detailed information about the specs of instance types, see https://aws.amazon.com/ec2/instance-types/:

EC2InstanceNames[]

Launch an EC2 instance:

inst=EC2CreateInstance[EC2Image[ec2client, "ami-08e83eaccb4f62657"],"InstanceType" -> "m5.xlarge","KeyPair" ->"meghanrkey","SecurityGroups" ->{"sg-a07163d9"}]

Tag an EC2 instance:

EC2SetTag[inst,<|"Owner"-> "meghanr"|>]

Get shell command for connecting to instance:

$SSHUserName = "ubuntu";
$SSHPrivateKeyPath = "/Users/meghanr/Desktop/AWSLink/";
sshCommand = "ssh -i "<>$SSHPrivateKeyPath<>inst["KeyName"]<>".pem "<>$SSHUserName<>"@"<>inst["PublicDnsName"]

Terminate the instance:

EC2InstanceTerminate[inst]

Putting it all together

Determine what data to use using MongoLink

Create a script to:

Query MongoDB for appropriate data using MongoLink

Download the accompanying files from S3 using ExternalStorage

Restructure data

Train

Launch EC2 instance using AWSLink

Copy script to EC2 instance

Run script

Resources

MongoLink documentation

https://reference.wolfram.com/language/MongoLink/guide/MongoLinkOperations.html

ExternalStorage documentation

?ExternalStorage`*

AWSLink documentation

?AWSLink`*

AWS documentation

https://docs.aws.amazon.com/index.html#lang/en_us