A New Tool to Measure the Impact of Foreign Influence Campaigns

Researchers from the University of Maryland and LAS identify 11 factors that determine someone’s likelihood to act on information they receive. The top two include a perceived threat and the hope for a better future. 

Influence operations, or influence campaigns, have been a tool for warfare and strategic geopolitical competition for governments as long as there has been recorded history. We can see the impact of such influence operations firsthand around the world, ranging from Chinese government efforts to spread disinformation regarding the origins of the coronavirus to Russian efforts to influence American elections. Mis- and disinformation have become an increasingly effective tool for nefarious actors to influence elections, policy and public sentiment in the United States. In other words, influence operations are an effective and low-cost means for our strategic adversaries and competitors to advance their own goals.

Given the immense implications of such influence operations, researchers from the University of Maryland’s National Consortium for the Study of Terrorism and Responses to Terrorism (START), in collaboration with NC State’s Laboratory of Analytic Science (LAS), identified an urgent need to advance methodologies to measure and assess the impact of adversarial influence operations. A team of START and LAS researchers identified a gap in current literature on influence operations, which failed to address the relative importance of different variables within a message in determining the likely effectiveness of foreign influence operations on a target audience.

Starting in January 2021, a team of researchers led by Steve Sin advanced a project to develop a rigorous methodology for assessing and measuring the impact of foreign influence operations. Sin serves as the director of the Unconventional Weapons and Technology Division at START. The team’s research drew on the fields of psychology, sociology, marketing and communications to identify variables that are likely to enhance an individual’s propensity for action.

By May 2022, the team had developed the influence-to-action model (I-AM) and the prototype influence-to-action campaign/narrative impact assessment tool (I-AM assessment tool). The prototype tool, which draws from literature on social movement diffusion and the role of threat in social movements, is designed to assess the relative impact that a given influence campaign would have on a target audience. The tool utilizes 11 weighted narrative variables, which are grouped into either internal (individual) or social categories, in its assessment of influence campaign artifacts. The variables determined to have an impact on an individual’s propensity to act were:

Internal factors

1. Emotional appeal: The message includes an attempt to appeal to the emotions of the target audience. For example, in order to entice a specific ethnic diaspora living in the United States, a nation-state may frame the message in the “perspective” of a member of that ethnic diaspora and tell the hardship of living in the United States in a way that most members of the diaspora can identify with.
2. Threat to identity: The message identifies threat(s) to the commonly perceived/accepted identity of the target audience. For example, the message clearly identifies that people with certain ideologies in the United States are not approving of people of certain ethnicity.
3. Threat to culture (way of life): The message identifies threat(s) to the commonly perceived/accepted culture of the target audience. For example, the message clearly identifies that people with certain ideologies in the United States do not approve of people living in a diaspora who are not assimilating to the “American culture.”
4. Identity affirmation: The message conveys that the target audience is of a certain identity (that aligns with the reader’s identity), and that you should be proud of your identity.
5. Victim affirmation: The message conveys that the target audience is being victimized due to an aspect of identity, social status, economic status, race, ethnicity, etc. For example, the message implies that people who meet certain criteria are not able to advance further in the society because another group within society (e.g., another social class) is systematically placing barriers to prevent the advancement of those people.
6. Victimization: The message conveys there is a need to “right the wrong” or “fight against injustice;” therefore, your actions against those who have made you a victim are justified.

Social factors

7. Persistent and increasing threat (“things are getting out of hand”): The message conveys persistent and worsening conditions if something is not done to reverse the course. For example, messaging about potential worsening of COVID cases around the holidays.
8. Clear alternative future (“the world will be better by doing something about it”): The message identifies that by taking action, there is an alternative future where threats against the target audience will diminish or even be eliminated.
9. Purported lack of response (“nothing is being done about it”): The message identifies that no one is doing anything to stop and/or reverse the threat against the target audience.
10. Clearly identified in/out group (“it’s the right thing to do for me and my social group”): The message identifies there is an in-group (the one that the target audience belongs to) and (an) out-group(s). One needs to take action because “it is the right thing to do for the ‘team’.”
11. Social validation (“it increases my reputation”): The message states that doing something about an identified injustice is correct, justified, and/or legitimate. The message may also imply that “everyone is doing it,” and that those who do are “in the right” and will be “recognized” for doing so.

The tool and its variables were validated via case studies of one historical and two present-day Chinese and Russian influence operations. These three case studies were used to test the influence-to-action model tool against large-scale, state-sponsored influence operations across different time periods and origins. The tool was successfully used to analyze all three cases and assess their impact. 

Which factors are most important?

The approach was further validated by two surveys of adults in the U.S. general population, which had questions designed to tease out the relative importance of each aspect of an influence campaign. 

From results of those surveys, weights were assigned to each variable contained in the influence-to-action model. Based on survey results, the most important variables within the individual category were:

1. victimization,
2. identity affirmation,
3. threat to culture (way of life).

Within the societal category, the most important variables were:

1. clear alternative future, and 
2. clearly identified in/out group. 

Researchers were surprised to learn that the relative weights for these variables remain consistent across different demographic groups. While there was some nuance in the importance of variables for each demographic group, the most important variables in determining the impact of a piece of propaganda (identified above) remained consistent.

Using these weights, the influence-to-action model is able to calculate the relative impact of any given piece of propaganda. The tool provides analysts a systematic methodology for the analysis of influence operations across multiple types of media and platforms. This allows for consistent analysis of influence operations material, regardless of the origin or mode of delivery, and puts a new and unique tool in the analyst’s toolbox.

Resources

View the influence-to-action model project video from the LAS 2021 Symposium: 

Guest post by Susanna B.