One of the creators of Botometer— a web tool Elon Musk used to estimate Twitter’s spam rate for a court filing — allegedly said Musk’s calculation “means nothing.” Kai ChengYanga doctoral candidate at Indiana University, “questioned the methodology used by Mr Musk’s team and told the BBC that they had not approached him before using the tool,” a BBC article said today.
A Musk Court Presentation on August 4 stated that a Botometer analysis of Twitter firehose data in the first week of July “shows that, during that time period, fake or spam accounts accounted for 33 percent of viewable accounts.” But as Yang pointed out, the botometer provides scores from 0 to 5, with 5 being the most bot-like, and Musk’s court filing doesn’t say where he drew the line between human and bot.
“To estimate the prevalence [of bots] you have to pick a threshold to lower the score,” Yang told the BBC. “If you change the threshold from three to two, you’ll get more bots and fewer humans.” Because Musk’s court filing “doesn’t make the details clear,” Musk “You have the freedom to do whatever you want. So the number means nothing to me,” Yang said.
“Technically, you can choose any threshold you want and get the result you want,” Yang said in a previous statement. interview with yahoo. The Botometer is a project of the Social Media Observatory and the Network Science Institute at Indiana University.
Botometer rated Musk as a likely robot
Botometer itself once “indicated that Elon Musk’s own Twitter account was likely a bot, scoring 4/5”, as Twitter noted in a court presentation. Musk’s botometer score has reportedly it fluctuated between 0.5 and 4, showing that the tool classifies Musk as human on some days and more bot-like on others.
Twitter also noted that Musk and his team “have not indicated what score they are applying to conclude that an account constitutes spam; therefore, their allegation is not verifiable.” Twitter further noted that an account could be a bot without being what the company considers a fake or spam account. Twitter gave examples such as bots “reporting earthquakes as they happen or weather updates.”
The Botometer may consider other types of legitimate accounts as likely bots. The Botometer gave my own verified Twitter account a bot score of 3 out of 5 today, and rated Ars Technica’s verified account 3.6 out of 5.
The Botometer website Frequently asked questions warns against labeling every account above a certain number as a bot. “It’s tempting to set an arbitrary threshold score and consider everything above that number to be a bot and everything below that a human, but we don’t recommend this approach… We think it’s more informative to look at the distribution of scores across a sample of accounts,” says the FAQ section.
Yang surprised Musk didn’t come up with a better tool
which also spoke to CNN recently, expressing surprise that Musk used the botometer instead of creating something more precise. “To be honest, you know, Elon Musk is really rich, isn’t he? I assumed that he would spend money hiring people to build a sophisticated tool or methods himself,” Yang told CNN.
The botometer is best used “to supplement, not replace, your own judgment,” says the tool’s FAQ, noting that “humans and machines have different strengths when it comes to pattern recognition. Some bot accounts /humans ‘obviously’ according to a human observer will fool a machine learning algorithm. For example, Botometer sometimes classifies ‘organizational accounts’ as bot accounts. Similarly, an algorithm can confidently classify some accounts with which humans have difficulties.”
Twitter sued Musk in Delaware Chancery Court after he tried to back out of his commitment to buy the company for $44 billion. Musk has defended his attempt to break the merger deal by questioning Twitter’s public disclosure that less than 5 percent of its monetizable daily active users (mDAUs) are spam or fake.
Twitter defends the accuracy of its estimates, saying they are based on “multiple (replicated) human reviews of thousands of randomly selected accounts each quarter using public and private data.” Twitter also says that Musk has no right to opt out of the merger deal based on the number of spam accounts.
Musk has plans for more comprehensive spam analysis, according to his court filing. “Defendants’ experts are continuing their analysis even now and, in anticipation of Twitter’s production of additional data (including ‘private’ data that Twitter makes available to its human reviewers and maintains that its report needs to be verified less than 5 percent spam and fake user rate), intend to conduct a more comprehensive analysis and expect to present updated estimates and findings in expert reports and at trial,” Musk’s attorneys wrote.