On Feb. 24, 2008, The New York Times Magazine featured a device called LENA (Language ENvironment Analysis) in its "Idea Lab" column. LENA is being marketed to families who have children between birth and 3 years of age. According to the product's Web site, LENA enables parents to monitor the number of words their child hears and the number of conversational turns involving their child for up to 16 hours at a time through a digital language processor about the size of a cell phone. This device attaches to a child's clothing or fits into the pocket of a child-sized jumper that comes with the LENA system. After 10 or more hours of continuous recording, parents are able to download their results to a computer, obtain the number of words they spoke to their child and the number of conversational turns involving their child, and compare these data to those of others.
We were intrigued by the Times article. The device doesn't require audiotapes or videotapes, nor does it require hours of language-sample transcription and coding. For researchers who study the language development of children and clinicians who conduct mother-child training programs, the recording, transcribing, and coding of language samples has always been part of the daily grind. In March 2008, we purchased a LENA (processor version V2.12.00 and parent software version V2.3) to explore this device's potential for our research and clinical services. We share our thoughts about this device and offer what we believe are some positive—but not perfect—findings related to the use of LENA for some research and clinical purposes.
Development of LENA began in 2004 by Terrance and Judi Paul, founders of Infoture, the Colorado-based company that markets and provides technical support for the device. Scientists nationally recognized for their work in child language serve on Infoture's advisory board, and a significant amount of data has been used to support the development of the LENA system. These data include language samples collected from 42 families by Hart and Risley (1995) and from 314 families by Infoture.
We examined LENA's ability to help us measure mother-child interaction by using archival language samples from 17 mothers and their children. These samples were collected as part of a caregiver training study (for preliminary findings, see Oetting, Pruitt, & Roy, 2006; for findings related to the mothers' use of African American English, see Oetting & Pruitt, 2005). All of the mothers were low-income, African American, single and working on their general equivalency degree (GED), and 13 of them reported that they were teens when they had their first child.
At the time of the study, the mothers' average age was 23; their average educational level was 10th grade. Their participating children, ranging in age from 2 to 5 years, were attending a child care center affiliated with the GED program. All of the children were typically developing according to parent and teacher reports; we confirmed the developmental status of each child using the Ages and Stages Questionnaire (Squires, Potter, & Bricker, 1999).
Our goal was to evaluate the LENA system by comparing the number of words and conversational turns generated by the device for each of the 17 mother-child dyads to counts that we had from hours of transcribing and coding language samples using the Systematic Analysis of Language Samples software (SALT; Miller & Chapman, 2001). For each 30-minute sample, the same set of toys and books were provided to the caregivers to facilitate their interactions with their children. The same room—equipped with a high-quality audio system and separate video system—was used to record all of the samples.
From our earlier analyses, we knew that our manual transcription of the language samples using SALT was reliable. Indeed, a second set of examiners independently transcribed 20% of the samples and inter-rater agreement was greater than 90% at the word and utterance boundary level. We also knew that our mothers' average amount of talking to their children was lower than what has been reported for older mothers with more education, but our mothers also showed a wide range of within-group variability. For example, the mother who talked the most to her child produced, on average, 20 utterances per minute; the two mothers who talked the least produced, on average, 10 utterances per minute. This amount of within-group variability is consistent with previous mother-child studies.
In the current work, we examined how well the LENA system captured the variability of these mothers' amounts of talking to their children as well as the accuracy with which it estimated the mothers' spoken words and conversational turns.
We were not capturing live samples, as intended by the LENA system; instead, we played an audio recording of each language sample into the LENA processor. After each recording, we downloaded the data from the processor into a computer, and extracted the number of words and conversational turns generated by the LENA software. This process was repeated for each of the 17 samples. Finally, to check the test-retest reliability of the LENA system, we ran the 17 samples through the device a second time.
Table 1 [PDF] shows that the word counts generated from our manual transcription methods and the two LENA passes were comparable. The LENA estimates from the first and second passes also resulted in similar group averages; however, estimates of conversational turns were less consistent across the two methods and two LENA passes.
Next we completed Pearson correlation analyses. For total word counts spoken by the adult, the
correlation between the two methods was moderate to high in magnitude and statistically significant; LENA first pass vs. transcription, r= 85, p<.001; LENA second pass vs. transcription, r=.71, p=.01. The correlation was also in the moderate range when we examined the relation between the first and second passes using the device; LENA first recording vs. LENA second recording, r=.76, p<.001.
These correlations indicate that the LENA system does a fairly good job of estimating a caregiver's word productions. Correlations in the .71–.85 range, however, also mean that the system has a fair amount (~40%) of unexplained variance or error. Although many of the word count estimates generated by LENA and our manual transcription methods were similar, estimates from these two methods differed by more than 500 words for four of our samples. This outcome reflected a 25% error rate for the LENA system for these four samples, and an average error rate of 18% for all of the samples as compared to the transcription method.
Results were less positive for LENA's estimates of conversational turns. For this measure, correlations between the two methods and between our first and second LENA recordings were much lower; LENA first pass vs. transcription, r=.14, p>.05; LENA second pass vs. transcription, r=.08, p>.05; LENA first pass vs. LENA second pass, r=.25, p>.05. It is important to note, however, that for conversational turns, standard counts using SALT are reported in turn lengths for each speaker (e.g., number of turns involving one, two, and three-plus utterances), and these data must be added together to estimate the total number of conversational turns of the participants.
Moreover, the criteria used to end a speaker's turn vary slightly from what is used by the LENA system (i.e., SALT allows a greater number of events to end a turn). Given this difference, it is perhaps not too surprising that conversational turn counts generated from the two methods were weakly correlated to each other. This explanation, however, does not address the low correlation that was found between the first and second LENA estimates for conversational turns.
Technical reports on the LENA Web site also show that LENA's word counts are more accurate than its conversational turn counts and that the device performs best when more than five hours of conversation for a child are analyzed. This assessment of the device is consistent with our findings.
These findings lead us to conclude that LENA may be a promising tool for researchers and clinicians who want a quick—albeit not perfect—estimate of a caregiver's spoken words to a child. For example, this information could be used for initial screening of caregivers for either a research study or a parent training program. For both researchers and clinicians, LENA word counts could also be used as one type of pre- or post-measure of treatment. Finally, LENA could be explored as part of a treatment program to determine if caregivers change their interactions with their children as a result of using the LENA system.
If the device is used for any of these purposes, however, we recommend that samples be as long as possible and that a secondary recording be made so that reliability can be evaluated. Our recommendations are also limited to the word count estimates of the LENA system; they do not apply to LENA's estimates of conversational turns. Our findings show that more work is needed to improve LENA's ability to estimate conversational turns between mothers and their children.
Our final comment relates to the value of focusing on the quantity of an adult's talking within a research project or parent training program. Numerous studies have shown that caregivers who talk frequently to their children are also more likely to use a facilitative language style—one that includes following a child's lead through the use of affirmatives, repetitions, expansions, and revisions, and that limits the use of directives and prohibitions. These findings are based on correlations; therefore, changes in one type of caregiver behavior may not necessarily lead to changes in others.
Our data can be used to illustrate this finding. Compared to other caregivers, our 17 mothers' amount of talking was described as low. Nevertheless, the five mothers who talked the most were also the mothers who showed extremely high rates of directives and prohibitions (up to 44% of their utterances) and extremely low rates of affirmatives, repetitions, revisions, and expansions (1%–6% of their utterances). Moreover, changes in the quantity of these mothers' talking during a later session were not accompanied by changes in their types of utterances.
Given this finding, researchers and clinicians interested in a caregiver's amount of talking should also consider other types of caregiver behaviors that are important to a child's language development. Our training programs, for example, seek to help caregivers increase their children's daily language activities while also encouraging them to follow their children's leads during these activities. LENA has the potential to facilitate research and clinical practice related to these types of training programs.
Disclaimer: This independent review was not prompted or supported by Infoture. As noted, the parent version of LENA was used in this study. A professional/research version of LENA was released by Infoture in April 2008; the two versions use the same processor and algorithms. Therefore, estimates of word counts and conversational turns are generated in the same way for the parent and professional versions of the device.