AI models that can understand videos just like text have the potential to unlock a whole new world of applications. That’s the belief of Jae Lee, the co-founder of TwelveLabs. Granted, Lee might be a bit biased since TwelveLabs specializes in training models for analyzing videos, but there seems to be some truth to his claim.
With TwelveLabs’ models, users have the ability to sift through videos to find specific moments, summarize clips, and even ask questions like “When did the person in the red shirt enter the restaurant?” These capabilities are powerful, which is why the company has attracted support from major names like Nvidia, Samsung, and Intel.
Lee, a data scientist, always felt that basic search methods were inadequate when it came to videos. Keyword searches are useful for titles, tags, and descriptions, but they fall short in capturing the actual content of the videos. Hence, TwelveLabs was born with the mission to bridge this gap by training models to understand the text and content of videos, including actions, objects, and background noises.
While companies like Google and Microsoft offer models for video analytics, Lee argues that TwelveLabs’ edge lies in its customization options, allowing customers to personalize models using their own data.
Despite concerns about bias in AI models, TwelveLabs has committed to releasing ethics-related benchmarks and datasets to address these issues. The company emphasizes responsible AI development, respecting civil liberties, and driving technological advancements.
TwelveLabs is not just about video analysis anymore. The company is venturing into areas like “any-to-any” search and multimodal embeddings. One of their models, Marengo, can search across images and audio, in addition to videos, providing a versatile tool for various applications.
The company’s expanding product portfolio has attracted clients in enterprise, media, and entertainment sectors. Databricks and Snowflake are among the major partners integrating TwelveLabs’ technology into their offerings.
With recent investments totaling $30 million, TwelveLabs plans to focus on product development and hiring to drive future growth. The addition of Yoon Kim, former CTO of SK Telecom, as president and chief strategy officer, highlights the company’s dedication to aggressive expansion plans.
The goal for TwelveLabs is to expand into new verticals, such as automotive and security, in the coming years. The involvement of In-Q-Tel suggests potential opportunities in national security, aligning with TwelveLabs’ commitment to making a positive impact while adhering to ethical guidelines.
