South Africa-founded startup launches AI model with 10 million-token memory

South Africa-founded startup Refiant AI has made a significant splash with the launch of Protea, a suite of large language models (LLMs) that can process up to 10 million tokens in a single prompt. Mathew Haswell, co-founder of Refiant AI, is at the forefront of this endeavour, having spent years building machine learning systems to reduce compute costs and improve AI model efficiency. What's at stake for Haswell and his team is the ability to provide a more efficient and robust AI experience for enterprises, allowing them to process and retain significantly more information at once before generating a response.

Protea comes in three versions, each with a different context window: one million, five million, and 10 million tokens. This is a significant departure from existing LLMs, which typically have much smaller context windows. For instance, Anthropic's Claude supports context windows of up to 500,000 tokens on certain Enterprise offerings, while Google's Gemini offers up to one million tokens on its higher-tier plans. By offering larger context windows, Refiant AI is betting that its technology will give enterprises an edge when analysing large datasets.

The launch of Protea comes as AI companies increasingly compete to expand the amount of information their models can process at once. This is a significant development in the AI landscape, where companies are moving away from surface-level integrations and towards more robust execution. Viroshan Naicker, CEO and co-founder of Refiant, notes that long-context AI has been talked about for over a year now, but hasn't really been commercially available. With Protea, Refiant AI is taking a major step forward in making this technology a reality.

The launch of Protea also marks a significant milestone for Refiant AI, which raised a $5 million seed round led by VoLo Earth Ventures just three months ago. This funding is set to help the company expand its AI optimisation platform and deepen research partnerships. With Protea, Refiant AI is offering a free platform that allows customers to access its LLMs without a waitlist or approval process. A notable step, as it opens up the technology to a wider range of customers and allows them to test, break, and build with the models from day one.

With Protea, Refiant AI is setting a new standard for AI models in terms of their ability to process and retain information. As the company continues to develop its technology, we can expect to see even more significant advancements in the field of AI. In fact, Refiant AI has already demonstrated an internal prototype capable of processing up to 100 million tokens and is exploring how to benchmark and bring the technology into production. With three planned product releases in the works, Refiant AI is clearly committed to pushing the boundaries of what is possible with AI.

In the coming months, we can expect to see more companies follow in Refiant AI's footsteps, offering larger context windows and more robust AI experiences. As this technology continues to evolve, one thing is clear: the future of AI is long-context, and Refiant AI is at the forefront of this revolution.