AI Identifies Language

In a joint effort between the Massachusetts Institute of Technology (MIT) and other U.S. institutions of higher education, scientists have shown off an AI system with the ability to pick up on the nuances of human language with relative ease. In addition to being able to learn lower-level language patterns automatically, this model is also capable of automatically learning higher-level language patterns that can apply to numerous languages.

The researchers trained and evaluated the model using problems from 58 distinct language-specific linguistics textbooks. Each task contained a list of words with appropriate word-form modifications. For 60 per cent of the problems, the model was able to provide a correct set of rules to explain these word-form modifications.

This technique could be applied to the study of linguistic hypotheses and the investigation of minor parallels between the ways in which distinct languages modify words. The system discovers models that can be easily comprehended by people, and it learns these models from modest amounts of data, such as a few dozen words.

Additionally, the system makes use of numerous tiny datasets rather than a single large one. This is closer to how scientists propose hypotheses, which is to look at numerous related datasets and develop models to explain phenomena across those datasets.

The researchers chose to investigate the relationship between phonology and morphology in their endeavour to create an AI system that could automatically train a model from numerous related datasets (the study of word structure).

The researchers utilised a machine-learning method called Bayesian Programme Learning to create a model that could learn grammar or a set of rules for putting words together. Using this method, the model creates a computer programme to address a challenge.

The grammar that the model believes is most likely to explain the words and their meanings in a linguistics problem is known as the programme. They created the model using Sketch, a well-known software synthesiser created by Solar-Lezama at MIT.

Read more