Semi-automated bullshit detection

Using Jev to find potential contradictions in the alleged personal histories of suspicious social media accounts

People who run inauthentic or misleading social media accounts often fabricate details of their alleged personal histories and daily lives, and sooner or later, some of these falsehoods end up contradicting one another. Searching a suspicious account’s posts for conflicting claims regarding things like how many children the operator has, what they do for a living, or where they grew up can be a useful technique for rooting out various sorts of fake accounts. It can also, however, be a time-consuming manual process, especially if the account in question posts prolifically.

Can the process of identifying these contradictions be automated? It turns out that it can be, to some extent, although manual review to filter out false positives is still necessary. By using Jev to estimate the likelihood that pairs of autobiographical posts contradict one another, it is possible to quickly find at least some of the contradictions in a social media account’s back story without performing extensive manual searches of the account’s posts. (Jev is an AI model that takes a chunk of JSON data and natural language questions about that data as input, accompanied with the data type that should be returned in response to each question.)

import pandas as pd
import sys
from typesafe_sdk import TypeSafeClient, Noul 


in_file = sys.argv[1]
out_file_auto = sys.argv[2]
out_file_contradict = sys.argv[3]
api_key = sys.argv[4]

with TypeSafeClient(api_key=api_key) as client:
    # test anything 100 chars or longer for autobiographical content
    df = pd.read_csv (in_file)
    df = df[df["text"].fillna ("").str.len () >= 100]
    results = []
    for text in df["text"]:
        response = client.system_one (
            state={"text" : text},
            questions={
                "autobiographical": Noul (
                    instructions="does the text contain an autobiographical claim?"
                 )
            }
        );
        results.append (response.nouls["autobiographical"].noul)
    df["autobiographical"] = results
    df.to_csv (out_file_auto, index=False)
    df = df[df["autobiographical"] >= 0.9]

    # test pairs of posts for potential contradictions
    results = []
    texts = list (df["text"])
    for i in range (len (texts) - 1):
        text1 = texts[i]
        for j in range (i + 1, len (texts)):
            text2 = texts[j]
            response = client.system_one (
                state = {
                    "text1" : text1,
                    "text2" : text2
                },
                questions={
                "contradiction": Noul (
                    instructions="does anything in text1 contradict anything in text2?"
                 )
            }
            );
            results.append ({
                "text1"         : text1,
                "text2"         : text2,
                "contradiction" : response.nouls["contradiction"].noul
            })
    df = pd.DataFrame (results).sort_values ("contradiction", ascending=False)
    df.to_csv (out_file_contradict, index=False)

The code above reads a user’s social media posts from a CSV file, discarding any post shorter than 100 characters to prevent false positives from short general statements (i.e. “I’m in a good mood”) that superficially “contradict” portions of longer, more complex posts. Jev is then used to provide a probability that each post contains autobiographical content, using the question “does the text contain an autobiographical claim?”; posts with a probability lower than 90% of being autobiographical are discarded. Jev is then used to check each possible pair of remaining posts for contradictions, using the question “does anything in text1 contradict anything in text2?”.

ten years, which is the same as seventeen years

To test this process, I downloaded all original posts from the Bluesky account @rsouissivibes.bsky.social, and ran them through the Python script featured earlier in this article. This account was selected as a test since there were already several known contradictions present in the account’s autobiographical posts, such as the inconsistency regarding marriage duration in the collage above. (The account in question also has other suspicious characteristics, such as extensive use of plagiarized photos.)

all of these posts indeed contain autobiographical content

The table above contains the posts that were rated as having the highest probability of containing autobiographical content, and each post indeed qualifies. The question “does the text contain an autobiographical claim?” appears to have been a reasonably effective method of causing Jev to identify posts containing autobiographical claims. The subject matter of the posts includes (among other things) alleged family history, alleged military service, alleged facial hair, and alleged participation in past elections.

some of the pairs of posts contain genuine contradictions, but some are false positives

The contradiction test was somewhat more error-prone than the test for autobiographical content, but still sufficiently effective at detecting genuine contradictions to be useful. For the example account, roughly half of the pairs of posts scored as having a high probability of containing contradictions turned out to be genuine contradictions after manual review. Despite the need to manually screen for false positives, this technique was nonetheless reasonably effective at flagging inconsistencies in this particular Bluesky account’s back story.

good knees, which are also bad knees

What kind of contradictions turned up? In addition to the aforementioned inconsistency regarding marriage duration, the operator of the @rsouissivibes.bsky.social account simultaneously has good knees and bad knees, worked for USAID for 10 years, which is also 15 years, and left his job at USAID in 2022, which is also 2024. There are numerous other contradictions related to the account operator’s alleged military service record, including inconsistency in overall duration, inconsistency in the set of countries allegedly visited, and inconsistency in the amount of time allegedly spent in or near Gaza specifically.

Removing the minimum post length requirement turns up several additional contradictions, including variation in the number of dogs allegedly living with the account operator, and inconsistency regarding whether the account operator uses Facebook. This also results in a higher false positive rate, however.

there’s a lot going on here

While this method of finding autobiographical contradictions is not perfect, and still requires some degree of manual review to remove false positives, it was effective at identifying multiple contradictions in the test account’s back story without the need to perform numerous keyword searches. This experiment was relatively simple, and it is likely that the technique can be improved significantly. One obvious spot for improvement is the step that determines whether a given pair of posts are contradictory; bringing down the false positive rate without meaningfully worsening the true positive rate would make the technique more effective. Other enhancements are likely possible as well.