Twelve Labs: Customer Spotlight
1. Tell us about your company: When did you start it? What is your product?
Twelve Labs was started in March 2021; we build proprietary video understanding AI that supports many downstream tasks in the video space. We started with video search. Given text and a pool of videos, users can search through the video. We recently released a classification API that lets users organize videos into certain classes, and we’re trying to develop more APIs.
2. Who is your typical customer?
It varies across different industries; some of the early adopters use our search in their own applications for video content recommendations. Some want to analyze what kind of information is in a video to use for targeted advertising. It’s also very popular in the video creator industry; creators have lots of raw footage and it can be hard for them to go through hours of raw footage to find the content they need.
3. How is AI used in your platform development?
There are two parts: the first is the infrastructure used to process the videos and run the models; the second part is actually building the models. Our engineering team really focused on serving the large model in the beginning; now we have our infrastructure, we are focused on training really powerful foundation models for videos that can be applied to multiple domains without having to finetune. We are very much focused on multimodal AI. Most AIs focus on LLMs, but we believe the next wave will be video and we want to be one of the first players to commoditize foundation models for videos.
4. Why did you choose MosaicML as your development partner?
We were introduced to MosaicML through Oracle, our infrastructure provider. We were looking into investing in larger GPU infrastructure when Oracle reached out to us. They knew their strength was in hardware, so they introduced us to your CEO, Naveen. We now had to evaluate whether to build the in-house software to manage a large cluster or look for an external solution. We had heard some horror stories about another open-source workload manager so we did an internal evaluation and that’s how we decided to work with you.
5. What challenges in model development have MosaicML helped you resolve?
The most important has been the GPU management system. You have this really nice interface for us to just interact with the client library, instead of us having to manually go into all these servers to set up distributed training. Your open-source libraries like StreamingDataset were also very helpful. All of our videos are stored in object storage and can’t be stored in local storage because of their size, so we use StreamingDataset to dynamically access datasets while training. Now we’re in the process of testing your Composer library.
The MosaicML team has been really helpful whenever we have any questions or errors or bugs; it’s been great.
6. How has MosaicML expanded your company's capabilities and product offerings?
So far, you guys have helped us make the training of our large models so much faster. We are actually looking into the possibility of adding the ability for users to fine-tune our model using your platform.
7. Do you have any predictions for what is coming next with Generative AI?
Traditionally, large language models and foundation models have been closed-source, but recently there has been a large push to open-source. I think in the next six months there will be lots of open-source models that will be comparable with closed-source. I’m very interested to see what new models will come out.
