On Thursday, French tech company Mistral launched a new API designed for developers dealing with complex PDF documents. Mistral OCR is an optical character recognition (OCR) API that can convert any PDF into a text file, making it easier for AI models to process.

Large language models (LLMs), like OpenAI’s ChatGPT, are most effective with raw text. This means companies looking to streamline their AI workflows need to ensure data is stored and indexed in a clean format for easy AI consumption.

Unlike typical OCR APIs, Mistral OCR is a multimodal API, meaning it can identify illustrations and photos within text blocks. This API creates bounding boxes around graphical elements and incorporates them into the output. Additionally, Mistral OCR formats output in Markdown, a syntax commonly used by developers to enhance plain text files with links, headers, and other formatting elements.

LLMs heavily rely on Markdown for their training data sets. AI assistants, such as Mistral’s Le Chat or OpenAI’s ChatGPT, often generate Markdown for tasks like creating bullet lists, adding links, or emphasizing certain elements. These assistants seamlessly convert Markdown output into rich text, highlighting the increased importance of raw text and Markdown in the growing field of AI.

Mistral OCR can be accessed through Mistral’s API platform or cloud partners like AWS, Azure, and Google Cloud Vertex. For companies handling classified or sensitive data, Mistral also offers on-premise deployment. The Paris-based company claims Mistral OCR outperforms OCR APIs from Google, Microsoft, and OpenAI, especially with complex documents containing mathematical expressions, advanced layouts, or tables, as well as non-English content.

Mistral believes its OCR model is faster than other options due to its singular focus. This speed is evident when compared to multimodal LLMs like GPT-4o, which includes OCR capabilities along with other features.

Mistral also utilizes Mistral OCR in its AI assistant, Le Chat. When a user uploads a PDF file, Mistral OCR analyzes the document to provide understanding before text processing. Companies and developers are likely to integrate Mistral OCR with a Retrieval-Augmented Generation (RAG) system to leverage multimodal documents as inputs for LLMs. There are various potential use cases, like law firms utilizing Mistral OCR to efficiently navigate large volumes of documents with RAG, a technique that provides context for generative AI models.