Students 
Building Learning Tools 
via Augment™ Training 
in Mathematica

Presenters:
Augment LLC -- Minneapolis, MN startup -- Training & Software Dev
	Eric Nelson
Students at Torrey Pines High School in San Diego, CA
	Carine You
	Jodie Hoh
	Seojin Jung

Essential Question:

How can the computer be better employed to grow, express, & assess human understanding?

Spotlight: Grant Wiggins (Dec. 2015) -- Author, UbD -- Founder, Authentic Education

Row[{,"    ",
Graphics[Text[Style["Enduring Understandings
Backwards Design
Authentic Assessment",30]], ImageSize-> Medium]}]

Wolfram language as the tool for interfacing human mind and computation

For Augmented Cognition (cyborg thinking), Knowledge-making

Wolfram Summer Camp

For training cognitive disciplines & habits

Computer-Based Math (CBM)

The Need: Scaling up + Assessment

The Twist: Assessment may be the key to scale-up

Thought Experiment: Imagine Year-Round Wolfram Summer Camp

Math class as continuous knowledge-making exercise.

i.e. Use Math as “tool” for making sense of the world.
& Use Wolfram language to “drive” the math tool.

Paraphrasing Jon McLoone (Co-Founder CBM): 
A lot more people learn to drive cars than to design them.

Same true in Math.

How to scale facility with Math concepts and Wolfram language?

Challenges

Policy, PR, and Infrastructure -- Leapfrogging years of stunted-growth inertia in K-12 Mathematics

Ensuring Efficacy -- Often only one chance to deploy a new idea to the marketplace...

Better forms of assessment can address both

Visualized Assessment of Understanding

A few models...

Topographic Map (of conceptual understanding & accomplishments)
 Network Graph (of conceptual relationships)
Affinity Matrix (for Concept Transfer)

Many more such models...

A spectrum of tooling for assessment

1. Basic: Typing Tutor to train automaticity
Drill is the same as performance
No mentorship required
Data is easy to gather and process

TypingAIPro.com (New Dawn Technologies)

Other skill sets add complexity & require different assessment approaches...

E.g. Piano...
Coordinated technical automaticity (piano fingering)
Conceptual understanding (piano composition)
Habits of Mind (empathic study of different piano style)

Assessment requires Inference from behaviors

I.e. Predicting future performance of “trained judgment”

Same for Math, Language Learning, ... and Programming!

Opportunity: Make real-time use of present & prior inferences

Scalable training and Scalable collaborative teams

Tutor systems (e.g. Piano Tutor -- more in a moment)

Capture, Aggregate, and Feedback practice and performance data

The key is Signal-to-Noise Ratio -- i.e. Back to Wiggins... Enduring Understandings, Essential Questions...

What are the indicators of comprehension & automaticity? 
Co-develop domain-specific assessment tools with SMEs.

Recognizing patterns which indicate Edge of Comprehension -- likely best use of neural nets...

Behaviorally-Anchored Self-Assessment

Expert-mediated assessment based on presence of behavioral anchors

Originated in Academia (e.g. Texas A&M), BARS in Commercial Aviation certification

Commercialized by Gary Jedynak (Augment VP Development), improving practical usage

Based on Dreyfus & Dreyfus model of Developmental Stages of Expertise, based on Benner US Navy grant

Presented by Jedynak in 2003 to Agile Software Development Conference as a way to assess programmers, then to 2005 U of MN Twin SPIN (Software Process & Quality group)

Vetted for legal defensibility -- published in Employee Rights Quarterly -- legal journal, 1999

Published in regard to assessment of distance learning -- 2004 AHRD Conference

Multiple commercial implementations

Back to Year-Round Wolfram Summer Camp & Scale-up

And our question...

How can the computer be better employed to grow, express, & assess human understanding?

The following, made possible via assessment tooling

Jigsawed Teams -- i.e. Early-stage learners -> Narrow & Deep study -> Shared Competencies

Collaboration with SMEs on subjects of consequence (Mentorship, Apprenticeship)

Distributed work

End result:

Students...
Grow skills, 
Contribute to meaningful accomplishment, and 
Merge into professional work

Example:

Abby Brown’s Advanced Topics class, and their work on a tool to assess piano skills

Piano Skills Visualization

Student input > Computations > Feedback

Working Principles of Project Development

Jigsaw Teams

Documentation: Design in English                                ■ Concept Domain
	- Designer’s Intent
	- Pseudo-code

Grid[{{,}},Spacings->10]

Strategies for Project Development

Simple to Complex

Instances to General

Simple to Complex

Timing Function

timingFunction[inputAudio_(*the track to be modified*), 
timingVariation_(*desired ammount of error, real value 0 -> 1*),
lessonNumber_(*integer value ≥0*),
improvementFactor_(*degree of improvement lesson by lesson, real value 0 -> 1*)]:= Apply[SoundNote,{inputAudio[[#,1]],
{Abs[inputAudio[[#,2,1]]+RandomReal[{-timingVariation,timingVariation}/(1+(improvementFactor*lessonNumber))(*find a random value between the timingVariation divided by the improvementFactor multiplied by the lesson number*)](*Add this value to the timing of the notes and take the absolute value*)],Abs[inputAudio[[#,2,2]]+RandomReal[{-timingVariation,timingVariation}/(1+(improvementFactor*lessonNumber))(*find a random value between the timingVariation divided by the improvementFactor multiplied by the lesson number*)](*Add this value to the timing of the notes and take the absolute value*)]},inputAudio[[#,3]],inputAudio[[#,4]]}]&/@Range[Length[inputAudio]]; (*Apply this over every note in the piece*)

(*Manipulate creates a compact form for sliders and localizes variables*)
Manipulate[
practiceMaryLambRecording1 := timingFunction[perfectMaryLambMIDI,variation,lessons,improvement];
Sound[practiceMaryLambRecording1],
{{variation,0.05,"Variation Range:"}, 0,.2,Appearance->"Labeled"},
{{lessons,2,"Number of Lessons: "}, 0,30,1,Appearance->"Labeled"},
{{improvement,0.075,"Improvement Factor: "}, 0,.2,Appearance->"Labeled"}
]

Data Generator

Manipulate[
Column[{
practiceMaryLambRecording := dataGen[perfectMaryLambMIDI,timingVariation,noteVariation,volumeVariation,lessons,improvement]; (*generate a practice recording with errors*)
Row[{Sound[perfectMaryLambMIDI],Spacer[{20,1}],
Sound[practiceMaryLambRecording]}], (*display perfect and practice recordings side by side*)
Spacer[{1,20}],
TableForm[Prepend[
data= Table[{ (*an array of variations in timing, note value, and volume as lesson number increases*)
lessons,
Total[Abs[Flatten[practiceMaryLambRecording[[All,2]]]-Flatten[perfectMaryLambMIDI[[All,2]]]]], (*total timing variation*)
Total[Abs[(Flatten[practiceMaryLambRecording[[All,1]]]/.noteNameToNumber)-(Flatten[perfectMaryLambMIDI[[All,1]]]/.noteNameToNumber)]]/N@Length[perfectMaryLambMIDI], (*total SCALED note variation*)
Total[Abs[Flatten[practiceMaryLambRecording[[All,4,2]]]-Flatten[perfectMaryLambMIDI[[All,4,2]]]]] (*total volume variation*)
},{lessons,0,30,2}],
{"Lesson Number","Timing Variation","Note Variation","Volume Variation"}]],
Spacer[20],
ListLinePlot[Table[data[[All,{1,n}]],{n,2,4}],PlotRange->{{0,30},{0,Automatic}},ImageSize->500,PlotLabel->"Variation vs Lesson Number",PlotLegends->{"Timing","Note","Volume"}] (*plot the variation data*)
}],
(*variables being manipulated*)
{{timingVariation,0.05,"Timing Variation Range:"}, 0,.2,Appearance->"Labeled"},
{{noteVariation,0.2,"Note Variation Range:"}, 0,.5,Appearance->"Labeled"},
{{volumeVariation,0.08,"Volume Variation Range:"}, 0,.1,Appearance->"Labeled"},
{{improvement,0.075,"Improvement Factor: "}, 0,.2,Appearance->"Labeled"},
{{lessons,5,"Lessons"},0,30}
]

Instances to General

Mary had a Little Lamb

(*Manipulate creates a compact form for sliders and localizes variables*)
Manipulate[
practiceMaryLambRecording1 := timingFunction[perfectMaryLambMIDI,variation,lessons,improvement];
Sound[practiceMaryLambRecording1],
{{variation,0.05,"Variation Range:"}, 0,.2,Appearance->"Labeled"},
{{lessons,2,"Number of Lessons: "}, 0,30,1,Appearance->"Labeled"},
{{improvement,0.075,"Improvement Factor: "}, 0,.2,Appearance->"Labeled"}
]

visualLongTermGen[perfectEntertainerSlashedMIDI]

Visualization

Timing Function

Note Graph

Future

Long-term Analysis

Complex technique signatures

Enhanced user interface

Initialization

noteGen[start_(*beginning octave number*),stop_(*ending octave number*),array_(*True/False: true to give an array, false to give a flattened list*)]:=If[array,Partition[#,12],#](*if statement to selectively apply Partition*)&@Flatten@{
(*letters generator*)letters = Flatten[Table[{"C","C#","D","D#","E","F","F#","G","G#","A","A#","B"},{stop-start+1}]];
(*numbers generator*)
numbers=Flatten[Table[ToString[#],{12}(*repeat each number 12 times*)]&/@Range[start,stop]];
(*Assemble letters and numbers*)
Table[letters[[n]]<>numbers[[n]],{n,1,Length[letters]}]}

keyGen[chromatic_,key_,majorMinor_,start_(*beginning octave number*),stop_(*ending octave number*)]:=(major={2,2,1,2,2,2,1}; naturalMinor={2,1,2,2,1,2,2};harmonicMinor={2,1,2,2,1,3,1};(*lists of interval patterns based on type of scale*)
indexOfKey=First@Flatten@Position[StringStartsQ[#,key<>ToString@start],True]&@Flatten[chromatic];(*index of key note in the chromatic scale list*)
Flatten[chromatic][[indexOfKey+Prepend[Accumulate@Which[majorMinor=="major",Flatten@Table[major,stop-start+1],majorMinor=="natural minor",Flatten@Table[naturalMinor,stop-start+1],majorMinor=="harmonic minor",Flatten@Table[harmonicMinor,stop-start+1]],0]]]);

(*Uses Table to create a SoundNote object for each note in the input scale*)
midiFormat[scale_,duration_,volume_]:=Table[SoundNote[scale[[n]],{duration*(n-1),duration*n},"Piano",SoundVolume->volume],{n,1,Length[scale]}]

(*Generating a list of SoundNote objects from octave range, key, duration, and volume*)
scaleGenMIDI[start_,stop_,key_,majorMinor_,duration_,volume_]:=midiFormat[keyGen[noteGen[start,stop+1,False],key,majorMinor,start,stop],duration,volume];

midiToList[track_]:=track/.SoundNote->List;

(*Perfect recording of "Mary Had a Little Lamb" with note names*)
perfectMaryLambMIDI={SoundNote["E4",{0,0.25},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{0.25,0.5},"BrightPiano",SoundVolume-> 1],SoundNote["C4",{0.5,0.75},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{0.75,1},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{1,1.25},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{1.25,1.5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{1.5,2},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{2,2.25},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{2.25,2.5},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{2.5,3},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{3,3.25},"BrightPiano",SoundVolume-> 1],SoundNote["G4",{3.25,3.5},"BrightPiano",SoundVolume-> 1],SoundNote["G4",{3.5,4},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{4,4.25},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{4.25,4.5},"BrightPiano",SoundVolume-> 1],SoundNote["C4",{4.5,4.75},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{4.75,5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{5,5.25},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{5.25,5.5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{5.5,5.75},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{5.75,6},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{6,6.25},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{6.25,6.5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{6.5,6.75},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{6.75,7},"BrightPiano",SoundVolume-> 1],SoundNote["C4",{7,8},"BrightPiano",SoundVolume-> 1]};
perfectMaryLambMIDI/.noteNameToNumber ;(*Perfect recording of "Mary Had a Little Lamb" with note numbers*)
Sound[Apply[SoundNote,perfectMaryLambMIDI,1]];

timingFunction[inputAudio_(*the track to be modified*), 
timingVariation_(*desired ammount of error, real value 0 -> 1*),
lessonNumber_(*integer value ≥0*),
improvementFactor_(*degree of improvement lesson by lesson, real value 0 -> 1*)]:= Apply[SoundNote,{inputAudio[[#,1]],
{Abs[inputAudio[[#,2,1]]+RandomReal[{-timingVariation,timingVariation}/(1+(improvementFactor*lessonNumber))(*find a random value between the timingVariation divided by the improvementFactor multiplied by the lesson number*)](*Add this value to the timing of the notes and take the absolute value*)],Abs[inputAudio[[#,2,2]]+RandomReal[{-timingVariation,timingVariation}/(1+(improvementFactor*lessonNumber))(*find a random value between the timingVariation divided by the improvementFactor multiplied by the lesson number*)](*Add this value to the timing of the notes and take the absolute value*)]},inputAudio[[#,3]],inputAudio[[#,4]]}]&/@Range[Length[inputAudio]]; (*Apply this over every note in the piece*)

noteValueFunction[inputAudio_(*the track to be modified*), 
noteVariation_(*desired ammount of error, real value 0 -> 1*),
lessonNumber_(*integer value ≥0*),
improvementFactor_(*degree of improvement lesson by lesson, real value 0 -> 1*)]:=
Apply[SoundNote,{((inputAudio[[#,1]]/.noteNameToNumber(*convert to note numbers for easy adding*)
)+RandomChoice[{#/4,#/4,1-#,#/4,#/4}(*probabilities of choosing each distance from expected note*)
&@(noteVariation/(1+(improvementFactor*lessonNumber))^2)(*total note variation*)
->{-2,-1,0,1,2}(*distances from expected note*)])/.noteNumberToName (*convert back to note names*),inputAudio[[#,2]],inputAudio[[#,3]],inputAudio[[#,4]]}]&/@Range[Length[inputAudio]];

volumeFunction[inputAudio_(*the track to be modified*), 
volumeVariation_(*desired ammount of error, real value 0 -> 1*),
lessonNumber_(*integer value ≥0*),
improvementFactor_(*degree of improvement lesson by lesson, real value 0 -> 1*)]:= Apply[SoundNote,{inputAudio[[#,1]],inputAudio[[#,2]],inputAudio[[#,3]],inputAudio[[#,4,1]](*"Volume"*)->If[RandomReal[]<(volumeVariation/(1+(improvementFactor*lessonNumber))),(inputAudio[[#,4,2]]-volumeVariation/(1+(improvementFactor*lessonNumber))),inputAudio[[#,4,2]]]}](*total amount of variation*)&/@Range[Length[inputAudio]];

(*Function that alters timing, note value, and volume*)
dataGen[inputAudio_, timingVariation_,noteVariation_,volumeVariation_,lessonNumber_,improvementFactor_]:= 
volumeFunction[noteValueFunction[timingFunction[inputAudio,  timingVariation,lessonNumber, improvementFactor],  noteVariation,lessonNumber, improvementFactor],  volumeVariation,lessonNumber, improvementFactor] ;

practiceScaleGenerator[{start_,stop_,key_,majorMinor_,duration_,volume_},{timingVariation_,noteVariation_,volumeVariation_,lessonNumber_,improvementFactor_}]:=dataGen[scaleGenMIDI[start,stop,key,majorMinor,duration,volume],timingVariation,noteVariation,volumeVariation,lessonNumber,improvementFactor];

(*Perfect recording of "Mary Had a Little Lamb" with note names*)
perfectMaryLambMIDI={SoundNote["E4",{0,0.25},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{0.25,0.5},"BrightPiano",SoundVolume-> 1],SoundNote["C4",{0.5,0.75},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{0.75,1},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{1,1.25},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{1.25,1.5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{1.5,2},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{2,2.25},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{2.25,2.5},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{2.5,3},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{3,3.25},"BrightPiano",SoundVolume-> 1],SoundNote["G4",{3.25,3.5},"BrightPiano",SoundVolume-> 1],SoundNote["G4",{3.5,4},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{4,4.25},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{4.25,4.5},"BrightPiano",SoundVolume-> 1],SoundNote["C4",{4.5,4.75},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{4.75,5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{5,5.25},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{5.25,5.5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{5.5,5.75},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{5.75,6},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{6,6.25},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{6.25,6.5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{6.5,6.75},"BrightPiano",SoundVolume-> 1],SoundNote["D4",{6.75,7},"BrightPiano",SoundVolume-> 1],SoundNote["C4",{7,8},"BrightPiano",SoundVolume-> 1]};
perfectMaryLambMIDI/.noteNameToNumber ;(*Perfect recording of "Mary Had a Little Lamb" with note numbers*)
Sound[Apply[SoundNote,perfectMaryLambMIDI,1]];

perfectEntertainerSlashedMIDI={SoundNote["D4",{0,0.25},"BrightPiano",SoundVolume-> 1],SoundNote["D#4",{0.25,0.5},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{0.5,0.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{0.75,1.25},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{1.25,1.5},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{0.5,1.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{1.5,2},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{2,2.25},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{1.5,2.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{1.5,2.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{2.25,3.5},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{2.5,3},"BrightPiano",SoundVolume-> 0.65],SoundNote["A3",{3,3.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["F3",{3,3.5},"BrightPiano",SoundVolume-> 0.65],
SoundNote["C5",{3.75,4},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{4,4.25},"BrightPiano",SoundVolume-> 1],SoundNote["D#5",{4.25,4.5},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{3.5,3.75},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{3.5,3.75},"BrightPiano",SoundVolume-> 0.65],SoundNote["E5",{4.5,4.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{4.75,5},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{5,5.25},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{4.5,5.25},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{4.5,5.25},"BrightPiano",SoundVolume-> 0.65],SoundNote["C3",{4.5,5.25},"BrightPiano",SoundVolume-> 0.65],SoundNote["E5",{5.25,5.75},"BrightPiano",SoundVolume-> 1],SoundNote["B4",{5.75,6},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{6,6.5},"BrightPiano",SoundVolume-> 1],SoundNote["F3",{5.25,6.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["G3",{5.25,6.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["B2",{5.25,6.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{6.5,7.5},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{6.5,7},"BrightPiano",SoundVolume-> 0.65],SoundNote["G3",{7,7.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{7,7.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["D4",{8,8.25},"BrightPiano",SoundVolume-> 1],SoundNote["D#4",{8.25,8.5},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{8,8.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{8,8.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E4",{8.5,8.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{8.75,9.25},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{9.25,9.5},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{8.5,9.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{9.5,10},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{10,10.25},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{9.5,10.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{9.5,10.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{10.25,11.5},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{10.5,11},"BrightPiano",SoundVolume-> 0.65],SoundNote["A3",{11,11.5},"BrightPiano",SoundVolume->0.65],SoundNote["F3",{11,11.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["A4",{12,12.25},"BrightPiano",SoundVolume-> 1],SoundNote["G4",{12.25,12.5},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{11.5,11.75},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{11.5,11.75},"BrightPiano",SoundVolume-> 0.65],SoundNote["F#4",{12.5,12.75},"BrightPiano",SoundVolume-> 1],SoundNote["A4",{12.75,13},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{13,13.25},"BrightPiano",SoundVolume-> 1],SoundNote["E5",{13.25,13.75},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{13.75,14},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{14,14.25},"BrightPiano",SoundVolume-> 1],SoundNote["A4",{14.25,14.5},"BrightPiano",SoundVolume-> 1],SoundNote["D3",{12.5,14.5},"BrightPiano",SoundVolume->0.65],SoundNote["D5",{14.5,15.5},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{14.5,15},"BrightPiano",SoundVolume-> 0.65],SoundNote["F3",{15,15.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["D4",{16,16.25},"BrightPiano",SoundVolume-> 1],SoundNote["D#4",{16.25,16.5},"BrightPiano",SoundVolume-> 1],SoundNote["E3",{15.5,16},"BrightPiano",SoundVolume-> 0.65],SoundNote["D3",{16,16.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E4",{16.5,16.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{16.75,17.25},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{17.25,17.5},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{16.5,17.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{17.5,18},"BrightPiano",SoundVolume-> 1],SoundNote["E4",{18,18.25},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{17.5,18.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{17.5,18.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{18.25,19.5},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{18.5,19},"BrightPiano",SoundVolume-> 0.65],SoundNote["A3",{19,19.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["F3",{19,19.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{19.75,20},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{20,20.25},"BrightPiano",SoundVolume-> 1],SoundNote["D#5",{20.25,20.5},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{19.5,19.75},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{19.5,19.75},"BrightPiano",SoundVolume-> 0.65],SoundNote["E5",{20.5,20.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{20.75,21},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{21,21.25},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{20.5,21.25},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{20.5,21.25},"BrightPiano",SoundVolume-> 0.65],SoundNote["C3",{20.5,21.25},"BrightPiano",SoundVolume-> 0.65],SoundNote["E5",{21.25,21.75},"BrightPiano",SoundVolume-> 1],SoundNote["B4",{21.75,22},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{22,22.5},"BrightPiano",SoundVolume-> 1],SoundNote["F3",{21.25,22.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["G3",{21.25,22.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["B2",{21.25,22.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{22.5,23.5},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{22.5,23},"BrightPiano",SoundVolume-> 0.65],SoundNote["G3",{23,23.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{23,23.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{24,24.25},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{24.25,24.5},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{23.5,24},"BrightPiano",SoundVolume-> 0.65],SoundNote["E3",{23.5,24},"BrightPiano",SoundVolume-> 0.65],SoundNote["E5",{24.5,24.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{24.75,25},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{25,25.25},"BrightPiano",SoundVolume-> 1],SoundNote["C4",{24.5,25.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E5",{25.25,25.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{25.75,26},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{26,26.25},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{26.25,26.5},"BrightPiano",SoundVolume-> 1],SoundNote["A#3",{25.5,26.5},"BrightPiano",SoundVolume->0.65],SoundNote["E5",{26.5,26.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{26.75,27},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{27,27.25},"BrightPiano",SoundVolume-> 1],SoundNote["A3",{26.5,27.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E5",{27.25,27.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{27.75,28},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{28,28.25},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{28.25,28.5},"BrightPiano",SoundVolume-> 1],SoundNote["G#3",{27.5,28.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["E5",{28.5,28.75},"BrightPiano",SoundVolume-> 1],SoundNote["C5",{28.75,29},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{29,29.25},"BrightPiano",SoundVolume-> 1],SoundNote["E5",{29.25,29.75},"BrightPiano",SoundVolume-> 1],SoundNote["B4",{29.75,30},"BrightPiano",SoundVolume-> 1],SoundNote["D5",{30,30.5},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{28.5,30},"BrightPiano",SoundVolume-> 0.65],SoundNote["F3",{30,30.5},"BrightPiano",SoundVolume-> 0.65],SoundNote["C5",{30.5,31},"BrightPiano",SoundVolume-> 1],SoundNote["E3",{30.5,31},"BrightPiano",SoundVolume-> 1],SoundNote["G3",{31,31.5},"BrightPiano",SoundVolume->0.55],SoundNote["C6",{31.5,31.75},"BrightPiano",SoundVolume-> 1],SoundNote["E5",{31.5,31.75},"BrightPiano",SoundVolume-> 1],SoundNote["C3",{31.5,31.75},"BrightPiano",SoundVolume-> 0.65]};

(*Library of all letter notes on a standard full-sized piano*)
noteNameInventory=Append[noteGen[0,8,False],"C8"] ;

(*Rule list to convert note names to number values*)
noteNameToNumber=Table[noteNameInventory[[n+49]]-> n,{n,-48,Length@noteNameInventory-49}];

(*Rule list to convert number values to note names*)
noteNumberToName=Table[ n->noteNameInventory[[n+49]],{n,-48,Length@noteNameInventory-49}];

(*A filter that deletes notes not found in the note library*)
noteFilter[track_,noteLib_]:=track[[DeleteCases[If[MemberQ[noteLib,track[[All,1]][[#]]],#]&/@Range[Length[track]],Null]]];  

(*A filter that deletes notes with volume less than .43*)
volumeFilter[track_]:= DeleteCases[If[track[[#,4,2]]>.43,track[[#]]]&/@Range[Length[track]],Null];

(*A filter that shifts the time of the entire track to remove silence at the beginning*)
silenceFilter[track_]:={track[[#,1]],track[[#,2]]-track[[1,2,1]](*start time of track*),track[[#,3]],track[[#,4]]}&/@Range[Length[track]];

(*Apply previous filters to track*)
filterTrack [track_,noteNumList_]:= silenceFilter@volumeFilter@noteFilter[track,noteNumList];

(*Extracts each note number from a track*)
noteExtract[recording_]:=(recording[[#,1]]&/@Range[Length[recording]])/.noteNameToNumber;

(*extract the time intervals from a list of MIDI format notes*)
timingExtract[recording_]  := recording[[#,2]] &/@Range[Length[recording]];

(*find the duration of each note*)
timingDuration [recording_] := recording[[#,2]] - recording[[#,1]]&/@Range[Length[recording]];

noteListArray[music_] :=music[[All, 1]];
absNoteList = {"C","C#","D","D#","E","F","F#","G","G#","A","A#","B"};
ruleList=Table[absNoteList[[n]]-> n,{n,1,12}];

n=normalizationCoefficient[perfectTitle_,recording_]:=Apply[Plus,Duration[perfectTitle]]/Apply[Plus,Duration[recording]];
startTimingExtract[piece_]:=piece[[#,2,1]]&/@Range[Length[piece]];
normalizePractice[perfectTitle_,recording_]:=normalizationCoefficient[perfectTitle,recording] startTimingExtract[recording];
difference[perfectTitle_,recording_]:=Abs[startTimingExtract[perfectTitle]-normalizePractice[recording]];
range1[piece_]:=Range[0,Length[piece],2];
intervalPositions1[piece_]:=Append[Cases[FirstPosition[Cases[startTimingExtract[piece],a_->a>=#],True]&/@range1[piece],{_}],{range1[piece][[-1]]}];
timeError1[perfectTitle_,recording_]:=Abs[startTimingExtract[perfectTitle]-normalizePractice[perfectTitle,recording]];
all[perfectTitle_,recording_]:=Table[Apply[Plus,timeError1[perfectTitle,recording][[intervalPositions1[perfectTitle][[z,1]];;(intervalPositions1[perfectTitle][[z+1,1]])-1]]],{z,1,Length[intervalPositions1[perfectTitle]]-1}];
timeErrorBar[perfectTitle_,recording_]:=BarChart[all[perfectTitle,recording],BarSpacing->None,ChartLabels->{2*Range[Length[intervalPositions1[perfectTitle]]-1]-1},Ticks->Automatic]

visualLongTermGen[perfect_]:=Manipulate[
Column[{
practice:= dataGen[perfect,timingVariation,noteVariation,volumeVariation,lessons,improvement]; (*generate a practice recording with errors*)
Row[{Sound[Apply[SoundNote,perfect,1]],Spacer[{20,1}],
Sound[Apply[SoundNote,practice,1]]}], (*display perfect and practice recordings side by side*)
Spacer[{1,20}],
TableForm[Prepend[
data= Table[{ (*an array of variations in timing, note value, and volume as lesson number increases*)
lessons,
Total[Abs[Flatten[practice[[All,2]]]-Flatten[perfect[[All,2]]]]], (*total timing variation*)
Total[Abs[(Flatten[practice[[All,1]]]/.noteNameToNumber)-(Flatten[perfect[[All,1]]]/.noteNameToNumber)]]/N@Length[perfect], (*total SCALED note variation*)
Total[Abs[Flatten[practice[[All,4,2]]]-Flatten[perfect[[All,4,2]]]]] (*total volume variation*)
},{lessons,0,30,2}],
{"Lesson Number","Timing Variation","Note Variation","Volume Variation"}]],
Spacer[20],
ListLinePlot[Table[data[[All,{1,n}]],{n,2,4}],PlotRange->{{0,30},{0,Automatic}},ImageSize->500,PlotLabel->"Variation vs Lesson Number",PlotLegends->{"Timing","Note","Volume"}] (*plot the variation data*)
}],
(*variables being manipulated*)
{{timingVariation,0.1,"Timing Variation Range:"}, 0,.2,Appearance->"Labeled"},
{{noteVariation,0.4,"Note Variation Range:"}, 0,.5,Appearance->"Labeled"},
{{volumeVariation,0.3,"Volume Variation Range:"}, 0,.3,Appearance->"Labeled"},
{{improvement,0.075,"Improvement Factor: "}, 0,.2,Appearance->"Labeled"},
{{lessons,5,"Lessons"},0,30}
]

perfectTiming=timingExtract[perfectMaryLambMIDI];(*time intervals of perfect recording*)
practiceTiming=timingExtract[practiceMaryLambRecording];(*time intervals of practice recording*)
durationDifferences=timingDuration[perfectTiming]- timingDuration[practiceTiming];

noteListArray[music_] :=music[[All, 1]];
absNoteList = {"C","C#","D","D#","E","F","F#","G","G#","A","A#","B"};
ruleList=Table[absNoteList[[n]]-> n,{n,1,12}];

(*perfectSongNotes=StringDrop[noteListArray[perfectEntertainerSlashedMIDI],-1];
perfectSongNoteArray=Table[{perfectSongNotes[[i]],perfectSongNotes[[i+1]]},{i,1,Length[perfectSongNotes]-1}];
SongNotes=StringDrop[noteListArray[testEnt],-1];
SongNoteArray=Table[{SongNotes[[i]],SongNotes[[i+1]]},{i,1,Length[SongNotes]-1}];
songDupNoteCompare=DeleteCases[Table[If[Table[UnsameQ[SongNoteArray[[i]],perfectSongNoteArray[[i]]],{i,1,Length[perfectSongNoteArray]}][[i]],perfectSongNoteArray[[i]]],{i,1,Length[perfectSongNoteArray]}],Null];
songNoteCompare=DeleteDuplicates[songDupNoteCompare];
songNoteCount=Partition[Flatten[Table[{songNoteCompare[[i]],Count[songDupNoteCompare,songNoteCompare[[i]]]},{i,1,Length[songNoteCompare]}],3],3];
songDupNoteCompare=DeleteCases[Table[If[Table[UnsameQ[SongNoteArray[[i]],perfectSongNoteArray[[i]]],{i,1,Length[perfectSongNoteArray]}][[i]],perfectSongNoteArray[[i]]],{i,1,Length[perfectSongNoteArray]}],Null];
songNoteCompare=DeleteDuplicates[songDupNoteCompare];
songNoteCount=Partition[Flatten[Table[{songNoteCompare[[i]],Count[songDupNoteCompare,songNoteCompare[[i]]]},{i,1,Length[songNoteCompare]}],3],3];
songNoteArray=songNoteCount/.ruleList;*)