ModLex: Modelling Early Lexical Development

Francisco Lacerda
Department of Linguistics
Stockholm University
Stockholm, Sweden

francisco.lacerda@ling.su.se

Join the Conversation #WolframTechConf

Abstract

A stochastic model of lexical development and change is presented as an example of how Mathematica can be used to address linguistically relevant issues. The model simulates spontaneous stochastic interactions between “mathematical agents” endowed with a range of initial lexical abilities and characterized by different individual parameters like propensity for imitation, memory capacity or misunderstanding probability.
In the simulated interaction process, an agent is randomly selected to produce an utterance. The length of the utterance is determined by a stochastic variable varying between zero (silence, no utterance) to the maximum reasonable utterance length. Non-silent utterances are assumed to have implicit, though a priori unknown, communicative meaning. Another agent in the immediate neighborhood of the speaking agent is assumed to hear the utterance and responds with its own version of the utterance (ranging from perfect imitation to a completely new utterance or just a silence).
By varying the agents’ parameters controlling imitation probability, probability of adding new sounds, memory span, etc., the model provides relevant empirical bases for the study of linguistic processes - like the early stages of the language acquisition or the social and areal spreading of linguistic trends - and the subsequent implementation of theoretically sound mathematical models describing such processes.

References
Pierre - Yves Oudeyer (2006), Self - organization in the evolution of speech, Oxford UP
Lacerda, F. (2009). On the emergence of early linguistic fucntions: A biological and interactional perspective. In K. Alter, M. Horne, M. Lindgren, M. Roll, & J. von Koss Torkildsen (Eds.), Brain Talk: Discourse with and in the brain (Vol. 1, pp. 207-230). Lund: Media-Tryck.
Lim, S. J., Lacerda, F., & Holt, L. L. (2015). Discovering functional units in continuous speech. Journal of Experimental Psychology: Human Perception and Performance, 41(4), 1139-1152.

Natural languages?

What is a natural language?

How are they structured?

What are the constraints?

How stable are languages?

How are they learned?

Linguistics and Mathematica

Basic Examples

Retrieve the resource:

ResourceObject["7ee344c1-361f-4f1e-8d67-d9bae308f204"]

Retrieve the default content:

ResourceData["7ee344c1-361f-4f1e-8d67-d9bae308f204"]

Find properties related to verbs:

Select[ResourceData["7ee344c1-361f-4f1e-8d67-d9bae308f204","ColumnNames"],StringContainsQ["Verb"]]

Create a function to generate maps of specific property values:

walsMap[property_,title_]:=With[{data=KeySort[KeyDrop[GroupBy[ResourceData["7ee344c1-361f-4f1e-8d67-d9bae308f204"],property][[All,All,"GeoPosition"]],Missing["NotAvailable"]]]},GeoListPlot[Values[data],PlotLegends->Placed[Normal[Keys[data]],Below],GeoCenter->{0,0},ImageSize->Full,PlotLabel->title]];
walsMap["81A",ResourceData["7ee344c1-361f-4f1e-8d67-d9bae308f204","ColumnNames"]["81A"]]

Find properties related to articles:

Select[ResourceData["7ee344c1-361f-4f1e-8d67-d9bae308f204","ColumnNames"],StringContainsQ["Article"]]

Compare language families by features related to the use of definite articles:

Sort[#,Greater]&/@(Counts/@KeySort[KeyDrop[GroupBy[ResourceData["7ee344c1-361f-4f1e-8d67-d9bae308f204"],Key["37A"]],Missing["NotAvailable"]]][All,All,"Family"])

What are the possible sounds for a language?

What are the possible sounds for a language?

What are the possible sounds for a language?

LinguisticAssistant["Graphics3D"]

An overview of the sound inventories in world’s languages (vowels)

An overview of the sound inventories in world’s languages (consonants)

An overview of the sound inventories in world’s languages (non-pulmonic sounds)

Acoustic theory of speech production

Hearing domain:
How many different stationary sounds would be discriminable?

Assuming about 17 discriminable frequency bands and 17 discriminable levels within each give (17^17), which is about (10^20)
This corresponds roughly to the number of cells in the bodies of about (10^6) individuals, weighing 100 kg each
How can the infant navigate in this domain and become a competent speaker of the ambient language?

Possible lexical inferences/utterance as a function of the listener’s age

Current implementation of the model:
Two agents in a lexical development situation

Two agents with different initial linguistic experiences interact with each other. One of the agents has established lexical categories and produces conventional words; the other agent has no established lexical categories and produces no conventional words.


The agents have limited memory capacity, may try to replicate heard utterances and always hear both the other agent’s and own utterances.

Initialization steps

Load the Statistical plots (necessary for Paretoplots)

Define the set of vowels and consonants that can be used to create spontaneous random sequences

Define the utterance generation function

Define the agent' s individual characteristics

Needs["StatisticalPlots`"] (* Necessary for the Paretoplot *)

(* Provide an orthographic definition of what is meant by vowels and consonants *)
vowels={"a","e","i","o","u","y"};
consonants=Complement[Flatten[CharacterRange["a","z"]],vowels];

Define the utterance generation function

(* Produce a babbled utterance *)
spokenUtterance[p1_,p2_,p3_,p4_]:=
(   (* Parenthesis "(" must be used instead of "{" in order to get a string output *)
(* 
p1 - Probability of a new sound in an utterance,
p2 - Probability of vowel as first sound,
p3 - Probability of repeating a vowel after another vowel,
p4 - Probability of repeating a consonant after another consonant
*)

(* Make sure that p1, p2, p3 and p4 are probabilies within the open interval ]0;1[ *)
(*If[p1≤0,p1=(10^(-3)),If[p1≥1,p1=1-(10^(-3))]];
If[p2≤0,p2=(10^(-3)),If[p2≥1,p2=1-(10^(-3))]];
If[p3≤0,p3=(10^(-3)),If[p3≥1,p3=1-(10^(-3))]];
If[p4≤0,p4=(10^(-3)),If[p4≥1,p4=1-(10^(-3))]];
*)

utterance=""; (* Initialize the babbled utterance generator *)
While[RandomReal[]≤ (p1^(StringLength[utterance]+1)), (* Add a new sound to the utterance with probability (p1^(StringLength[utterance]+1)) *)

If[StringLength[utterance]<1, (* If there are no sounds in the utterance, prepare to produce the first sound *)
If[RandomReal[]<p2,  (* Pick a vowel with probability p2, or otherwise a consonant, and store it in utterance *)
utterance=StringJoin[utterance,RandomChoice[vowels]],
utterance=StringJoin[utterance,RandomChoice[consonants]]
],

utterance=  (* Continue babbling by adding a new sound to the utterance *)
StringJoin[utterance,
If[MemberQ[vowels,StringTake[utterance,-1]],
If[RandomReal[]≤p3,
RandomChoice[vowels],RandomChoice[consonants]
],
If[RandomReal[]≤p4,
RandomChoice[consonants],RandomChoice[vowels]
]
]
]
]
];
utterance
)  (* Parenthesis "(" must be used instead of "{" in order to get a string output *)

Define the agent’ s individual characteristics

Agent[n_,justHeardUtterance_,p0_,p1_,p2_,p3_,p4_,maxLexicalMemorySize_,memoryType_]:=
(  (* Parenthesis "(" must be used instead of "{" in order to get a string output *)
(* 
n - Agent's reference number,
justHeardUtterance - Utterance heard by the agent,
p0 - Probability of imitating the heard utterance,
p1 - Probability of a new sound in a utterance,
p2 - Probability of vowel as first sound,
p3 - Probability of repeating a vowel after another vowel,
p4 - Probability of repeating a consonant after another consonant,
maxLexicalMemorySize - Maximum number of items that can be held in memory,
memoryType - Primacy "P", Recency "R" (default) or Any "A"

*)

(* Make sure that p1, p2, p3 and p4 are probabilies within the open interval ]0;1[ *)
If[p1≤0,p1=(10^(-3)),If[p1≥1,p1=1-(10^(-3))]];
If[p2≤0,p2=(10^(-3)),If[p2≥1,p2=1-(10^(-3))]];
If[p3≤0,p3=(10^(-3)),If[p3≥1,p3=1-(10^(-3))]];
If[p4≤0,p4=(10^(-3)),If[p4≥1,p4=1-(10^(-3))]];

δ=0.05; (* Increment of the imitation probability p0 if the just heard utterance has been heard before *)
verboise=False; (* Prints out partial results for debugging, if True *)

If[MemberQ[heardUtterances[n],justHeardUtterance],imitationProbability=p0+δ,imitationProbability=p0];
(* Make sure that imitationProbability still falls within the open interval ]0;1[ *)
If[imitationProbability≤0,imitationProbability=(10^(-3)),If[imitationProbability≥1,imitationProbability=1-(10^(-3))]];

utterance=  (* Check if the just heard utterance qualifies to be repeated by the agent *)
If[(RandomReal[]≤imitationProbability && StringLength[justHeardUtterance]>0),

If[verboise,Print["This is the just heard utterance: ",justHeardUtterance]];

(* Scan for possible lexical candidates by comparing the just heard utterance with previous utterances that are still stored in the memory *)

For[i=1,i≤Length[heardUtterances[n]],

If[ (* Search for a primacy, recency or any match between the just heard utterance and any of those heard before or stored in the long-term lexicon *)
(
memoryType=="P" && StringStartsQ[heardUtterances[n][[i]],justHeardUtterance]
||memoryType=="A" && StringContainsQ[heardUtterances[n][[i]],justHeardUtterance]
||memoryType=="R" && StringEndsQ[heardUtterances[n][[i]],justHeardUtterance] 
),
(*
	Potential lexical candidate detected
	Find its position
	Add the candidate to the lexical candidates list
	Add the string before the found candidate also to the lexical candidates list
	Add also the remaining string to the lexical candidates list
*)
lexicalCandidatePosition=StringPosition[heardUtterances[n][[i]],justHeardUtterance];
(* Find out if there is a preceeding and a following string and how many matches there were within the previously heard utterance *)

newLexicalCandidates=StringTake[heardUtterances[n][[i]],lexicalCandidatePosition[[1]]]; (* It is enough to find the first occurrence *)
lexicalCandidates[n]=Flatten[Append[lexicalCandidates[n],{newLexicalCandidates,StringSplit[heardUtterances[n][[i]],newLexicalCandidates]}]];

If[verboise,Print["Item \"",justHeardUtterance,"\" found in \"",heardUtterances[n][[i]],"\" on position ",lexicalCandidatePosition,"\t-> ",lexicalCandidates[n]]]

];
i++
];

heardUtterances[n]=Append[heardUtterances[n],justHeardUtterance];
If[Length[heardUtterances[n]]>maxLexicalMemorySize,Drop[heardUtterances[n],Length[heardUtterances[n]]-maxLexicalMemorySize]];
RandomChoice[Flatten[Union[heardUtterances[n],lexicalCandidates[n]]]], (* Make a random selection among the n items recently heard or the lexical candidates stored in long-term memory*)

utterance=spokenUtterance[p1,p2,p3,p4]
];
If[StringLength[utterance]>0,producedUtterances[n]=Append[producedUtterances[n],utterance]];
utterance
)   (* Parenthesis "(" must be used instead of "{" in order to get a string output *)

Simulating the mother’s speech directed to the infant

Infant-directed speech consists of utterances containing concatenated words. The infant hears utterances with embedded words and derives potential lexical items from the recurrent sound patterns (combined with recurrent visual patterns).
The infant is assumed not to have knowledge of what a word is and searches simply for recurrent substrings in the mother’s utterances.

ids="jahejhej kommerduihågdomhär kocka kommerduihågkocka kommerduihågkocka kommerduihågkocka jaskadukänna skadukännalite ochtittalite jadetvarkocka ja jatitta javilldubekantadiglitemedkocka skadukänna skadukännalite jagördet ja taupparmendååhej jaa åheja tittahärdetvarmedöronenförragångenkundeviviftapådom kommerduihågdet kommerduihågdenvillduhellretittapåden hm skaduhälsapåkocka hejhej beribomberibomberibombip beribomberibomberibombip kommerduihågdenmedlillasvansen viftviftviftvift viftviftviftvift hejjagheterkockavaheterdu va kandutadenja såkanvitasåkommerkockaigen hejjagäruteågårjagäruteågårjagskagåutå tittaettträd prositdante titta skadukännakanmanocksåklappaså jakanmanocksåklappaskadukänna jaså jadukankänna ja härkännsåkännså såkanmankänna oj ojsan nösdunuigenaa tittaojblevdurädd kockakomuppsåhärvom åååvom åååvom hejhej hejhej ja detärkocka kocka hardusettblåttochröttochsvartochgröntochgult ochsåenlitenrödsvans viftviftkännpådendu skadukännapådenså asåkandukänna adärförsökakännaigen gogogogo kännapånosenockså kännapåkockasnosså jaganskasötvaochmjuk tänktedukundekännasåsåja kanhållasjälv deteheltjätte ja edukoncentreradva edukoncentreraddå åsåvardetjudappaockså";

Simulate each agent’s produced and heard utterances and linguistic knowledge

(* This cell initializes the specific knowledge/experience that of each of the agents has at the onset of the computations *)
heardUtterances[1]=StringSplit[ids," "]

producedUtterances[1]=StringSplit[ids," "]

lexicalCandidates[1]=StringSplit[(*"hej kocka titta känna lampa bord"*)ids," "]

heardUtterances[2]=StringSplit["aaaiiiia uuiiiiaaa oooouu aaaae dappaisababy hhedadsd"," "];
producedUtterances[2]=StringSplit["aaaiiiia uuiiiiaaa oooouu aaaae dappaisababy"," "];
lexicalCandidates[2]=StringSplit["mmmm agagag hihdxd rrrrrr pppiiiiff denincdf"," "];

Simulate interaction between the agents

(*
spokenUtterance[p1_,p2_,p3_,p4_],
p1 - Probability of a new sound in a word,
p2 - Probability of vowel as first sound,
p3 - Probability of repeating a vowel after another vowel,
p4 - Probability of repeating a consonant after another consonant
*)

w=spokenUtterance[0.9,0.1,0.5,0.1];
agentLexicon[1]={};
agentLexicon[2]={};
nCandidatesToDisplay=5;

For[j=1,j≤100,
(*
Agent[n_,justHeardUtterance_,p0_,p1_,p2_,p3_,p4_,maxLexicalMemorySize_],
n - Agent's reference number,
justHeardUtterance - Utterance heard by the agent,
p0 - Probability of imitating the heard word,
p1 - Probability of a new sound in a word,
p2 - Probability of vowel as first sound,
p3 - Probability of repeating a vowel after another vowel,
p4 - Probability of repeating a consonant after another consonant,
maxLexicalMemorySize - Maximum number of items that can be held in memory,
memoryType - Primacy "P", Recency "R" (default) or Any "A"
*)

w=Agent[1,w,0.1,0.1,0.5,0.1,0.1,100,"R"];
w=Agent[2,w,0.9,0.5,0.5,0.1,0.1,30,"R"];

For[k=1,k≤2,
top5pu=Take[SortBy[Tally[producedUtterances[k]],Last],Max[-Length[Tally[producedUtterances[k]]],-nCandidatesToDisplay]];
pu[k]=Column[{ParetoPlot[top5pu,ChartLabels->Placed[{""},Bottom],ImageSize->Medium,PlotLabel-> "Agent"<>ToString[k]<>", epoch "<>ToString[j]<>"\n Top "<>ToString[nCandidatesToDisplay]<>" produced utterances",BarSpacing->Large],top5pu[[5,1]],top5pu[[4,1]],top5pu[[3,1]],top5pu[[2,1]],top5pu[[1,1]]},Center];

top5hu=Take[SortBy[Tally[heardUtterances[k]],Last],Max[-Length[Tally[heardUtterances[k]]],-nCandidatesToDisplay]];
hu[k]=Column[{ParetoPlot[top5hu,ChartLabels->Placed[{""},Bottom],ImageSize->Medium,PlotLabel-> "Agent"<>ToString[k]<>", epoch "<>ToString[j]<>"\n Top "<>ToString[nCandidatesToDisplay]<>" heard utterances",BarSpacing->Large],top5hu[[5,1]],top5hu[[4,1]],top5hu[[3,1]],top5hu[[2,1]],top5hu[[1,1]]},Center];

top5lc=Take[SortBy[Tally[lexicalCandidates[k]],Last],Max[-Length[Tally[lexicalCandidates[k]]],-nCandidatesToDisplay]];
lc[k]=Column[{ParetoPlot[top5lc,ChartLabels->Placed[{""},Bottom],ImageSize->Medium,PlotLabel-> "Agent"<>ToString[k]<>", epoch "<>ToString[j]<>"\n Top "<>ToString[nCandidatesToDisplay]<>" lexical candidates",BarSpacing->Large],top5lc[[5,1]],top5lc[[4,1]],top5lc[[3,1]],top5lc[[2,1]],top5lc[[1,1]]},Center];

agentLexicon[k]=Append[agentLexicon[k],{pu[k],hu[k],lc[k]}];
k++
];
j++
]

Show what is going on in the agents’ memories as the number of interactions increases

ListAnimate[agentLexicon[1],Alignment->{Left,Bottom}]

ListAnimate[agentLexicon[2],Alignment->{Left,Bottom}]