Posted by Mumbai: Databricks, the data and AI company, has announced new Mosaic AI capabilities to help customers build production-quality Generative AI applications.
Databricks is investing in Mosaic AI in three key areas: support for building compound AI systems, capabilities to improve model quality, and new AI governance tools. These efforts will give customers the confidence to build and measure production-quality applications, delivering on the promises of Generative AI for their business.
Organisations are struggling to transition Generative AI projects from pilot to full-scale production due to privacy, quality, and cost concerns. While foundation models have all significantly improved, they still struggle to produce high-quality results. The highest-performing models may still give responses that are inaccurate, unsafe, or expose confidential data.
To address these challenges, organisations are going beyond deploying one extremely large model to deploying compound AI systems. This approach uses multiple components, including various models, retrievers, vector databases, and tools for evaluation, monitoring, security, and governance. As a result, compound AI systems offer much higher production quality, allowing organisations to deliver more accurate, safe, and governed AI applications efficiently.
“We believe that compound AI systems will be the best way to maximise the quality, reliability, and measurement of AI applications going forward, and may be one of the most important trends in AI in 2024,” said Databricks’ Co-Founder and CTO Matei Zaharia.
“Databricks is uniquely positioned to capitalise on these trends with the investments we’re making to improve quality, augmenting the model with real-time data and agents and tools to give it new capabilities it has little knowledge of,” added Zaharia.
To help customers build production-quality Generative AI applications, Databricks is launching Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Tools Catalog, Mosaic AI Model Training, and Mosaic AI Gateway.
Mosaic AI Agent Framework and Mosaic AI Tools Catalog help organisations build compound AI systems
Databricks has introduced several new capabilities to help customers deploy enterprise-ready compound AI systems. RAG is a type of compound AI system because it uses multiple components like a vector database, and tools for monitoring, evaluation, security, and governance to improve the accuracy of the LLM. Last month, Databricks announced the general availability of Mosaic AI Vector Search as a serverless vector database seamlessly integrated into the Data Intelligence Platform.
It announced Mosaic AI Agent Framework, which makes it easy for developers to quickly and safely build high-quality RAG applications, using foundation models and their enterprise data. They can evaluate the quality of their RAG application with Mosaic AI Agent Evaluation, iterate quickly, and redeploy their application easily.
Mosaic AI Agent Evaluation is an AI-assisted evaluation tool that automatically determines if outputs are high-quality and provides an intuitive UI to get feedback from human stakeholders. Collectively, these capabilities help organizations to deploy production-quality Generative AI solutions.
Compound AI systems often take advantage of tools as functions that equip these systems with new capabilities to interact with the world, such as intelligently generating and executing code, searching the web, calling APIs, and more. Mosaic AI Tools Catalog lets organisations govern, share, and register tools using Databricks Unity Catalog. This ensures models that are tool-enabled can use these in a secure and governed manner, as well as making these tools discoverable across the organisation.
Mosaic AI Model Training enables fine-tuning for foundation models, increasing model quality and decreasing cost
Mosaic AI Model Training fine-tunes open-source foundation models with an organisation’s private data, giving it new knowledge that is specific to its domain or task. These fine-tuned models are fully owned and controlled by the customer and produce higher-quality results for specific use cases because they have been trained on the organisation’s private data for specialised tasks. In addition to being more accurate for specific domains, smaller models fine-tuned by Model Training are also faster and less expensive to serve than larger proprietary models because they have fewer parameters and require less computing power.
Mosaic AI Gateway offers governance across all GenAI apps and models
Mosaic AI Gateway provides a unified interface to query, manage, and deploy any open source or proprietary model, enabling customers to easily switch the large language models (LLMs) that power their applications without needing to make complicated changes to the application code.
It supports usage tracking and guardrails, letting organisations track who is calling the model, set up rate limits to control spending from their enterprise users, and filter for safety and personally identifiable information (PII) regardless of which model is being used. Finally, it offers built-in governance and monitoring to continuously help ensure quality.
Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Model Training, and Mosaic AI Gateway are now in public preview. Mosaic AI Tools Catalog is in private preview.