Can an AI tool developed by an American engineer accurately assess meanings and nuances in other, unrelated languages?
That’s at the heart of research that Scott Jarvis, applied linguistics professor at NAU, is tackling with funding from the U.S. Department of Defense’s Minerva Initiative. It’s an important question for the DoD, which is interested in “understanding how culture, philosophy, and ideology directly shape AI development from planning to execution [and] how those practices shape the technology’s (un)intended effect(s) on populations or places that may not be co-located with the developers [of the AI technology].”
“All departments of the military are using various kinds of AI tools to figure out what’s going on in other countries,” said Jarvis, who came to NAU this year from the University of Utah. “It’s not spying—it’s having these tools look at social media and see what sorts of major events are being talked about and may be about to happen.”
Misunderstanding, misinterpreting or just missing the correct meaning of that information could have negative results, such as the inability to anticipate or appropriately respond to social unrest or other types of conflict.
The project has two parts; Gwyneth Sutherlin at the National Defense University in Washington, D.C., is coordinating the work on how existing AI tools designed and developed in the Western context categorize the content of social media posts in non-Western language and cultures—specifically, South Korea and Taiwan. Jarvis is coordinating the second half of the study, which looks at whether individuals who live in these countries and speak the languages of these cultures interpret social media posts in the same way the AI does.
“It’s not just a matter of whether the tools are interpreting the various kinds of grammatical features correctly,” Jarvis said. “It’s more a matter of whether people in different cultures actually express the same ideas in different ways or if they might be expressing different ideas that we simply don’t express.”
There is a multitude of evidence that different cultures express ideas in distinct ways that might not be intuitive to people of other cultures, Jarvis said. One example: Unlike English speakers, Macedonians use different grammar to indicate whether they actually witnessed the past-tense event that they are talking about, or whether they heard about it from someone else. The Korean language allows speakers to differentiate an action caused by a person (e.g., “She pushed the ball down the hill”) from an action that occurred without a human agent (e.g., “The ball rolled down the hill”). And when speakers of Finnish and Swedish were shown a silent Charlie Chaplin film and asked to describe a scene, the Finnish speakers said the characters were sitting on the grass (chosen because of proximity to a building) while Swedish speakers said they were sitting in the grass (chosen because of the height of the grass). By contrast, many American English speakers would say they were sitting in someone’s yard.
Given these differences, it seems likely that a bot created in the United States would not be able to properly interpret the nuances of these languages.
“We’re going in with an open-book approach, though we already know some of the differences in languages, the habitual patterns of thought between the Western world and East Asia,” Jarvis said. “We just don’t know if we’re going to find them in social media—we can find them in a well-designed experiment, but this isn’t an experiment, just observation. We want to know how people are talking about their everyday lives.”
The results of this research will shed light on differences in how people from different cultures perceive, categorize and describe the same types of events, and will also show how accurately current Western-designed AI technology is able to interpret what the original writers of the social media posts meant. Crucially, the research will also highlight the importance of having members of the target cultures participate in the future design of AI technologies used to interpret the information conveyed within those cultures.
Heidi Toth | NAU Communications
(928) 523-8737 | heidi.toth@nau.edu