Community Notes and potential vote brigading
A look at rating activity patterns for Community Notes on popular X posts
With the 2024 U.S. election less than a week away, it’s a great day to look at a different type of voting: specifically, the rating system used as input by X’s Community Notes crowdsourced fact-checking feature to determine which notes are publicly displayed. In order to make the vote system resistant to efforts at manipulation, Community Note visibility is NOT determined by simple majority vote; rather, it requires substantial numbers of “helpful” ratings from users with a variety of perspectives in order for a note to be shown. Nonetheless, that doesn’t stop some people from trying, and a number of recent notes received large belated infusions of either upvotes or downvotes. Interestingly, in every case where one of these potential brigades involved a political post, the belated votes were always in the direction favorable to the political right.
Although X’s API is largely locked behind a paywall, Community Notes data still remains freely available for download and study. Data used in this analysis was downloaded on October 30th, 2024, and contains data through roughly 3 AM UTC on October 28th, 2024. Data on notes created prior to October 1st, 2024 was discarded in order to focus on recent content, leaving 86537 notes. 6588 of these (7.6%) are currently rated “helpful” and are shown on the relevant posts; almost all of the remainder are still awaiting more ratings from users with varied perspectives.
The remainder of this analysis will focus on the 500 notes with the largest number of user ratings, ranging between 1100 and 6697 ratings each.
These 500 notes are attached to 279 distinct posts, 8 of which have been deleted. 45 of these notes have been rated “helpful” and are shown alongside the relevant posts. The user whose posts appear most frequently in the dataset is @elonmusk, followed by @realDonaldTrump, @KamalaHarris, and @KamalaHQ; several other U.S. political accounts turn up as well. 456 of the 500 proposed notes (91.2%) are in English, with the remainder in Japanese, Spanish, or Portuguese.
The majority of the 500 most-voted-on proposed Community Notes in October 2024 were written to provide context or corrections for political posts. The notes in the dataset that are currently rated “helpful” are indeed factually accurate; however, so are some of the notes that are not currently shown, such as the note on crowd size in the collage below. (This note, along with most of the notes that are not presently shown, has yet to accumulate enough ratings from users from varied perspectives to be deemed “helpful” or “not helpful” by the Community Notes algorithm.)
The Community Notes data contains the time of each “helpful” or “not helpful” vote received by every Community Note. This data can be used to study the accumulation of votes over time for a given note or set of notes, and it turns out that voting patterns for most notes (at least, for most notes that receive a lot of user ratings) are basically the same: votes accumulate quickly in the initial hours after a note is written, and then level off over time. All but 19 of the 500 most-voted-on Community Notes in October 2024 follow this pattern, regardless of whether upvotes or downvotes are more common for the note in question.
In the remaining nineteen cases, the note in question received a rapid infusion of hundreds of additional upvotes or downvotes (but generally not both) after several hours or several days. Although it’s possible that these additional surges of activity are the result of the post the note is attached to gaining additional exposure, this is rather unlikely due to the popularity of the accounts in question, which include X owner Elon Musk and multiple candidates for national U.S. political office. These bursts of activity may be attempts to manipulate the Community Notes system through vote brigading, although it’s important to note that none of these efforts changed the status of the notes in question, as none were visible either before or after the potential brigade.
Fifteen of the nineteen potentially brigaded notes are attached to political posts, and in each of these 15 cases, the political valence of the flurry of votes is the same: favorable to the political right, and to the MAGA movement and Donald Trump in particular. A breakdown of these vote barrages:
9 downvote barrages on notes on posts from right-wing accounts (@alx, @america, @DefiyantlyFree, @realDonaldTrump, and @robbystarbuck)
3 upvote barrages on notes on a post from Democratic VP candidate @Tim_Walz
3 downvote barrages on notes explaining why the notes on the @Tim_Walz post are unnecessary
The remaining four potential brigades are all swarms of downvotes on notes that attempt to correct the word “post” to “tweet” in a post from the official @X account. While I personally agree that these four Community Notes are not helpful, the possibility that they may have been targets of attempted vote manipulation is nonetheless interesting, and I am including them for the sake of completeness.
Interesting analysis, thanks.