Best-in-Class
The Genaios AI research team is officially the best in the world at detecting machine generated content!
We are enormously proud to announce that our machine-generated text detection system won first place in a competition against 140 teams across the globe, including participants from top universities and organizations.
SemEval is the preeminent workshop on semantic evaluation, where participants compete in Natural Language Processing tasks, and our technology obtained a whopping 96.9% accuracy rate in the following shared task:
Task 8: Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection.
Monolingual Subtask A) Binary Human-Written vs.Machine-Generated Text Classification
“Large language models (LLMs) are becoming mainstream and easily accessible, ushering in an explosion of machine-generated content over various channels, such as news, social media, question-answering forums, educational, and even academic contexts. Recent LLMs, such as ChatGPT and GPT-4, generate remarkably fluent responses to a wide variety of user queries. The articulate nature of such generated texts makes LLMs attractive for replacing human labor in many scenarios. However, this has also resulted in concerns regarding their potential misuse, such as spreading misinformation and causing disruptions in the education system. Since humans perform only slightly better than chance when classifying machine-generated vs. human-written text, there is a need to develop automatic systems to identify machine-generated text with the goal of mitigating its potential misuse.”
Our approach, consisting of a Transformer Encoder that mixes token-level probabilistic features extracted from four LLMs, ranked 0.8% higher than the runner-up. The mean of the participants was around 74.64, which means we were 22.88 points higher than the mean.
We will present the research in June at SemEval, co-located with the NAACL conference, in Mexico City.
This comes hot on the heels of our open-source release of TextMachina, an extensive framework and pipeline of tools for compiling high-quality, unbiased, Machine Generated Text datasets, to facilitate the community in connecting to the preferred LLMs, and developing effective, fair, and ethical systems, for automatically detecting Machine Generated Text.
TextMachina had more than 1300 downloads in 30 days, and it is being constantly updated with new LLM providers, supported tasks, and prompting strategies!
Due to the rise and democratisation of Generative AI, there is a pressing need to reliably and automatically detect human content, detect AI-generated content, and determine the boundaries between the two, in an ethical and unbiased manner.
For this reason, at Genaios we are committed to promoting research and innovation in the field of generative AI moderation. Thus, in 2023 we organized the AuTexTification shared task at IberLEF, where participants aimed to detect and attribute machine generated text in English and Spanish.
In that spirit, in 2024 we are organizing the Iber AuTexTification shared task, where we extend our previous task in three dimensions: more generation models, more domains, and more languages from the Iberian Peninsula (English, Spanish, Catalan, Basque, Galician, and Portuguese), aiming to build more generalizable detectors and attributors.
Genaios is proud to lead this global effort, be at the forefront of AI research efforts, contribute to the community, and build best-in-class technology to enable this.
—————————————————————————————————————————————————
The Genaios team is working hard to bring out our first product beta version!
If you would like to be one of the first users of the beta version, to help test and define the product, please sign up!