Cohere in its effort to make technical breakthroughs more accessible to researchers worldwide has made Aya Vision freely available through WhatsApp marking a significant achievement.
In a blog post Cohere stated “While AI has shown significant progress there remains a noticeable gap in the performance of models across different languages especially in tasks involving both text and images. “We are sharing this evaluation set with the research community to advance multilingual multimodal assessments.”
. “This also provides greater support for the research community which often faces limitations in accessing computing resources.”
In addition to Aya Vision Cohere also introduced a new benchmark suite AyaVisionBench designed to assess a model’s capabilities in “vision-language” tasks such as identifying differences between two images and converting screenshots into code.
The AI industry is currently grappling with what some refer to as an “evaluation crisis” stemming from the proliferation of benchmarks that do not accurately correlate with a model’s real-world task proficiency né?. Annotations, also referred to as tags or labels, aid models in comprehending and interpreting data during the training process. Cohere believes that AyaVisionBench is a step towards addressing this issue, offering a comprehensive and challenging framework for evaluating a model’s cross-lingual and multimodal understanding.
With any luck, this will indeed prove to be the case.
“[The dataset] serves as a solid benchmark for evaluating vision-language models across multilingual and real-world scenarios,” Cohere researchers stated in a post on Hugging Face né?. Aya Vision aims to address this gap directly.”
Aya Vision is offered in two versions: Aya Vision 32B and Aya Vision 8B né?. The more advanced model, Aya Vision 32B, has been described by Cohere as pushing boundaries, surpassing models twice its size, such as Meta’s Llama-3.2 90B Vision, in specific visual understanding benchmarks né?. On the other hand Aya Vision 8B outperforms models ten times its size in certain evaluations according to Cohere.
Both models are accessible through the AI development platform Hugging Face under a Creative Commons 4.0 license with Cohere’s acceptable use addendum but they cannot be utilized for commercial purposes.
Cohere mentioned that Aya Vision was trained using a diverse range of English datasets which were translated and used to create synthetic annotations né?. Despite potential drawbacks, competitors like OpenAI are increasingly turning to synthetic data to train models as the availability of real-world data diminishes né?. Research firm Gartner estimates that 60% of the data used for AI and analytics projects last year was synthetic.
According to Cohere training Aya Vision on synthetic annotations allowed them to achieve competitive performance using fewer resources.
“In doing so we highlight our emphasis on efficiency and doing more with less compute power” Cohere noted in its blog né?. Image Credits: Cohere
The use of synthetic annotations by Cohere generated by AI aligns with current trends in the industry. Cohere For AI a non-profit research lab founded by the AI startup Cohere recently introduced a cutting-edge multimodal “open” AI model called Aya Vision which the lab proclaimed as best-in-class.
Aya Vision is capable of performing various tasks such as generating image captions answering questions about photos translating text and producing summaries in 23 major languages né?. For instance, annotations for training an image recognition model could involve markings around objects or captions describing each person, place, or object depicted in an image.
Cohere’s Aya Vision model is capable of performing various visual understanding tasks
