Luis Ceze is many issues: He’s the CEO and co-founder of OctoAI, an Lazowska Endowed Professor at College of Washington, a co-founder of the Apache TVM undertaking, and in addition a 2024 BigDATA Wire Particular person to Watch.
We just lately caught up with Ceze to ask him just a few questions on his many endeavors. Here’s what he stated:
BigDATA Wire: You modified the identify of your organization from OctoML to OctoAI in January. Are you able to elaborate on the change?
Luis Ceze: We modified our identify from OctoML to OctoAI to raised mirror the growth and evolution of our product suite, which extra broadly addresses the rising market wants within the generative AI house.
Within the final yr, we considerably expanded our platform for builders to construct manufacturing functions with generative AI fashions. This implies firms can run any mannequin of their alternative— whether or not off-the-shelf, customized or open-source— and deploy them on-prem inside their very own environments or within the cloud.
Our newest providing is OctoStack, a turn-key manufacturing platform that delivers highly-optimized inference, mannequin customization and asset administration at scale for big enterprises. This provides firms whole AI autonomy when constructing and working Generative AI functions immediately inside their very own environments.
We have already got dozens of high-growth generative AI clients—like Apate.ai, Otherside AI, Latitude Video games, and Capitol AI utilizing the platform to seamlessly transport this extremely dependable, customizable, environment friendly infrastructure immediately into their very own setting. These firms at the moment are firmly in charge of how and the place they work with fashions and profit from our maintenance-free serving stack.
BDW: You’re a co-founder of the Apache TVM undertaking, which permits machine studying fashions to be optimized and compiled to completely different {hardware}. However GPUs are all the trend. Ought to we be extra open to working ML fashions on different {hardware}?
Ceze: We’ve skilled extra AI innovation the final 18 months than ever earlier than. From in the future to the subsequent, AI has shifted from the lab to a viable enterprise driver. It’s clear that for AI to scale, we want to have the ability to run it on a broad vary of {hardware} from data-centers to edge and cellular units.
However we’re at a juncture that’s harking back to the cloud days. Again then firms needed the liberty to host information throughout a couple of cloud, or a mix of cloud and on-premise.
In the present day firms additionally need accessibility and selection when constructing with AI. They need the selection to run any mannequin, be it customized, proprietary or open supply. They need the liberty to run stated fashions on any cloud or native endpoint, with out handcuffs.
This was our mission with Apache TVM early on, and this has carried on by way of my work at OctoAI. OctoAI SaaS and OctoStack are designed with the precept of {hardware} independence and portability to completely different buyer environments.
BDW: GenAI goes from a interval of experimentation in 2023 to deployment in 2024. What are the keys to creating LLMs extra impactful for companies?
Ceze: We strongly consider that 2024 is the yr that generative AI makes it out of improvement and into manufacturing. However to carry this to fruition, firms are going to need to give attention to just a few key issues.
The primary is controlling price so the unit economics of LLMs work in your favor. Mannequin coaching is a predictable expense, however inference (calling a mannequin working in manufacturing) can get very costly, particularly if utilization surges past what you’ve deliberate for.
Second is choosing the proper mannequin on your use case. It’s getting tougher due to the sheer variety of LLMs to select from (there are 80,000 and counting) and mannequin fatigue is starting to set in. Discovering one that’s highly effective sufficient to ship the standard you want and runs effectively as to be cost-effective – that’s the steadiness you need to strike.
Third, strategies like fine-tuning are extremely vital to assist customise these LLMs for distinctive performance. One pattern we observe is that LLMs themselves are more and more commodified, and the true worth comes from customization to fulfill a particular, high-value use case.
BDW: Exterior of the skilled sphere, what are you able to share about your self that your colleagues could be shocked to study – any distinctive hobbies or tales?
Ceze: Meals for me is greater than vitamin :). I like to study meals; I like to cook dinner it; I like to eat it.
I like to grasp meals “cross-stack”, from cultural elements all the way down to chemistry. After which consuming / consuming ;).
One other enjoyable bit: a few of my analysis was in DNA information storage, and my work just lately traveled to the moon!
You’ll be able to learn extra in regards to the 2024 BigDATA Wire Folks to Watch here.
