A startup referred to as Letta has simply emerged from stealth with tech that helps AI fashions keep in mind customers and conversations. Created in UC Berkeley’s famed Labs startup manufacturing facility, it additionally introduced $10 million in seed cash led by Felicis’ Astasia Myers, at a $70 million post-money valuation.
Letta can also be backed by a who’s who of angel traders in AI, like Google’s Jeff Dean, Hugging Face’s Clem Delangue, Runway’s Cristóbal Valenzuela, and Anyscale’s Robert Nishihara, amongst others.
Based by Berkeley PhD college students Sarah Wooders and Charles Packer, this can be a extremely anticipated AI startup launch. That’s as a result of it’s a toddler of Berkeley’s Sky Computing Lab and is the business entity of the favored MemGPT open supply mission.
Berkeley’s Sky Computing Lab, led by acclaimed professor and Databricks co-founder Ion Stoica, is the descendent of RISELab and AMPLab, which spawned such firms as Anyscale, Databricks, and SiFive. Sky Lab, particularly, birthed quite a few well-liked open supply massive language mannequin (LLM) initiatives just like the Gorilla LLM, vLLM, and the LLM structured language SGLang.
“A ton of initiatives in a short time, inside a 12 months’s timeframe, got here out of the lab. Simply folks sitting subsequent to us,” described Wooders. “So it was type of an unbelievable time.”
MemGPT is one such mission and is such a sizzling commodity that it really went viral earlier than it even launched.
“Somebody scooped us,” Packer instructed TechCrunch. The founders had posted a whitepaper on Thursday, October 12, 2023, and deliberate to launch a extra in-depth paper and the code to GitHub the next Monday. Some random individual discovered the paper, posted it to Hacker Information on Sunday, and it “went viral on Hacker Information earlier than we had an opportunity to correctly launch the code, launch the paper, or, like, do a tweet thread or something like that,” he stated.
The explanation for the joy was that MemGPT mitigates a pernicious downside for LLMs: Of their native type, fashions like ChatGPT are stateless, which means they don’t retailer historic information in long-term reminiscence. That is problematic for AI apps that depend upon attending to know and be taught from a consumer over time — every thing from buyer assist bots to healthcare symptom-tracking apps. MemGPT manages information and reminiscence in order that AI brokers and chatbots can keep in mind earlier customers and conversations.
The submit on the paper stayed atop Hacker Information, the favored web site for programmers run by Y Combinator, for 48 hours, Packer recounted. So he spent his weekend and the subsequent few days answering questions on the location whereas attempting to get the code able to be launched. As soon as the mission was accessible on GitHub, a hyperlink to it went viral on Hacker Information, once more. YouTube interviews and tutorials, Medium posts, 11,000 stars and 1.2K forks on GitHub occurred rapidly.
VC Felicis’ Myers found Wooders and Packer by studying about MemGPT, too, and instantly acknowledged the tech’s business potentialities.
“I noticed the paper when it was launched,” she instructed TechCrunch, and he or she promptly reached out to the founders. “We had an funding theme round AI agent infrastructure and appreciated {that a} actually essential part of that was the information and reminiscence administration to make these conversational chat bots and AI brokers efficient.”
The founders nonetheless just about traipsed round Sand Hill Highway doing Zoom calls with VCs earlier than going with the one which cherished them first.
In the meantime, Stoica brokered introductions to Dean, Nishihara and different big-name Silicon Valley angels. “A variety of the professors at Berkeley, simply as a consequence of being at Berkeley, are very effectively linked,” Packer recalled about how straightforward the angel investor course of was. “They’ve their eye on initiatives out of this lab which can be going to be commercialized.”
Competitors and the specter of OpenAI o1
Whereas MemGPT is already out within the wild and getting used, Letta’s business variant, Letta Cloud, is just not but open for enterprise. As of Monday, Letta is accepting requests for beta customers. It can provide a hosted agent service that permits builders to deploy and run stateful brokers within the cloud, accessible by way of REST APIs, a programming interface that may keep state. Letta Cloud will retailer the long-term information mandatory to take action. Letta may even provide developer instruments for constructing AI brokers.
With MemGPT, Wooders sees a big span of makes use of. “I believe the primary use case that we see is principally, extremely customized, very participating chatbots,” she says. However there are additionally cutting-edge makes use of like “a chatbot for most cancers sufferers” the place sufferers add their historical past after which share ongoing signs so the bot can be taught and provide steerage over time.
Price noting that MemGPT isn’t alone in engaged on this. LangChain might be its finest recognized competitor, and it already gives business choices. The most important mannequin makers additionally provide AI agent-making instruments as effectively, like OpenAI’s Assistants API.
And OpenAI’s new o1 mannequin could make the necessity to repair state a moot level for its customers. As it’s a multistep mannequin, it essentially should keep state to a point to be able to “suppose” and truth test earlier than it replies.
However Wooders, Packer, and Myers see just a few key variations to what Letta is providing versus what 800-pound market gorilla OpenAI is doing. Letta claims it’ll work with any AI mannequin and expects its customers to make use of lots of them: OpenAI, Anthropic, Minstrel, their very own homegrown fashions. OpenAI’s tech at present solely works with itself.
Extra importantly, Letta is utilizing open supply MemGPT and jumping firmly into the open source side of the FOSS vs. black box LLM debate, saying open supply is a more sensible choice for AI software programmers.
“We’re positioning ourselves because the open different to OpenAI,” Packer says. “I believe it’s really very, very onerous to construct excellent AI functions, particularly if you care about, like hallucination, if you happen to can’t see what’s occurring below the hood.”
