8.6 C
New York
Monday, May 10, 2021

Predicting Words’ Grammatical Properties Helps Us Read Faster | Nutrition Fit

Summary: When reading, people are not only able to predict the next word, but also the words’ grammatical properties. This allows us to read faster. The findings could help with the development of new neural networks focused on natural language processing.

Source: HSE

Psycholinguists from the HSE Centre for Language and Brain found that when reading, people are not only able to predict specific words, but also words’ grammatical properties, which helps them to read faster. Researchers have also discovered that predictability of words and grammatical features can be successfully modelled with the use of neural networks.

The study was published in the journal PLOS ONE.

The ability to predict the next word in another person’s speech or in reading has been described by many psycho- and neurolinguistic studies over the last 40 years. It is assumed that this ability allows us to process the information faster. Some recent publications on the English language have demonstrated evidence that while reading, people can not only predict specific words, but also their properties (e.g., the part of speech or the semantic group). Such partial prediction also helps us to read faster.

In order to access predictability of a certain word in a context, researchers usually use cloze tasks, such as The cause of the accident was a mobile phone, which distracted the ______. In this phrase, different nouns are possible, but driver is the most probable, which is also the real ending of the sentence. The probability of the word driver in the context is calculated as the number of people who correctly guessed this word over the total number of people who completed the task.

READ  Memory May Be Preserved in Condition With Brain Changes Similar to Alzheimer's Disease | Nutrition Fit

The other approach for predicting word probability in context is the use of language models that offer word probabilities relying on a big corpus of texts. However, there are virtually no studies that would compare the probabilities received from the cloze task to those from the language model.

READ  First-in-Human Clinical Trial to Assess Gene Therapy for Alzheimer’s Disease | Nutrition Fit

Additionally, no one has tried to model the understudied grammatical predictability of words. The authors of the paper decided to learn whether native Russian speakers would predict grammatical properties of words and whether the language model probabilities could become a reliable substitution to probabilities from cloze tasks.

The researchers analysed responses of 605 native Russian speakers in the cloze task in 144 sentences and found out that people can precisely predict the specific word in about 18% of cases. Precision of prediction of parts of speech and morphological features of words (gender, number and case of nouns; tense, number, person and gender of verbs) varied from 63% to 78%.

They discovered that the neural network model, which was trained on the Russian National Corpus, predicts specific words and grammatical properties with precision that is comparable to people’s answers in the experiment. An important observation was that the neural network predicts low-probability words better than humans and predicts high-probability words worse than humans.

The second step in the study was to determine how experimental and corpus-based probabilities impact reading speed. To look into this, the researchers analysed data on eye movement in 96 people who were reading the same 144 sentences. The results showed that first, the higher the probability of guessing the part of speech, gender and number of nouns, as well as the tense of verbs, the faster the person read words with these features.

READ  Higher Education Levels Linked to Greater Well-Being for Women | Nutrition Fit

The researchers say that this proves that for languages with rich morphology, such as Russian, prediction is largely related to guessing words’ grammatical properties.

This shows a woman reading a book
The other approach for predicting word probability in context is the use of language models that offer word probabilities relying on a big corpus of texts. Image is in the public domain

Second, probabilities of grammatical features obtained from the neural network model explained reading speed as correctly as experimental probabilities. ‘This means that for further studies, we will be able to use corpus-based probabilities from the language model without conducting new cloze task-based experiments,’ commented Anastasiya Lopukhina, author of the paper and Research Fellow at the HSE Centre for Language and Brain.

READ  Memory May Be Preserved in Condition With Brain Changes Similar to Alzheimer's Disease | Nutrition Fit

Third, the probabilities of specific words received from the language model explained reading speed in a different way as compared to experiment-based probabilities. The authors assume that such a result may be related to different sources for corpus-based and experimental probabilities: corpus-based methods are better for low-probability words, and experimental ones are better for high-probability ones.

‘Two things have been important for us in this work. First, we found out that reading native speakers of languages with rich morphology actively involve grammatical predicting,’ Anastasiya Lopukhina said. ‘Second, our colleagues, linguists and psychologists who study prediction got an opportunity to assess word probability with the use of language model: http://lm.ll-cl.org/. This will allow them to simplify the research process considerably’.

About this language research news

Source: HSE
Contact: Liudmila Mezentseva – HSE
Image: The image is in the public domain

Original Research: Open access.
Morphosyntactic but not lexical corpus-based probabilities can substitute for cloze probabilities in reading experiments” by Anastasiya Lopukhina et al. PLOS ONE


READ  Effective Anxiety Therapy Changes Personality | Nutrition Fit

See also

This shows a brain slice of the hippocampus

Morphosyntactic but not lexical corpus-based probabilities can substitute for cloze probabilities in reading experiments

During reading or listening, people can generate predictions about the lexical and morphosyntactic properties of upcoming input based on available context. Psycholinguistic experiments that study predictability or control for it conventionally rely on a human-based approach and estimate predictability via the cloze task.

Our study investigated an alternative corpus-based approach for estimating predictability via language predictability models. We obtained cloze and corpus-based probabilities for all words in 144 Russian sentences, correlated the two measures, and found a strong correlation between them. Importantly, we estimated how much variance in eye movements registered while reading the same sentences was explained by each of the two probabilities and whether the two probabilities explain the same variance.

READ  Researchers Identify Brain Ion Channel as New Approach to Treating Depression | Nutrition Fit

Along with lexical predictability (the activation of a particular word form), we analyzed morphosyntactic predictability (the activation of morphological features of words) and its effect on reading times over and above lexical predictability. We found that for predicting reading times, cloze and corpus-based measures of both lexical and morphosyntactic predictability explained the same amount of variance.

However, cloze and corpus-based lexical probabilities both independently contributed to a better model fit, whereas for morphosyntactic probabilities, the contributions of cloze and corpus-based measures were interchangeable. Therefore, morphosyntactic but not lexical corpus-based probabilities can substitute for cloze probabilities in reading experiments.

Our results also indicate that in languages with rich inflectional morphology, such as Russian, when people engage in prediction, they are much more successful in predicting isolated morphosyntactic features than predicting the particular lexeme and its full morphosyntactic markup.

Source link

Related Articles


Please enter your comment!
Please enter your name here

Stay Connected

- Advertisement -

Latest Articles

Read previous post:
Does Asparagus Make Your Urine Smell Like a Skunk? | Nutrition Fit

Most people detect a distinct sulfurous odor in their urine shortly after eating asparagus. The smell of their urine within...