Wednesday 16 April 2025
The marriage of artificial intelligence and databases has long been a topic of interest for researchers and developers alike. Now, a new project aims to bridge the gap between these two worlds by integrating large language models (LLMs) into relational databases.
FlockMTL, as it’s called, is an open-source extension for DuckDB, an embeddable analytical database that allows users to perform complex analytics on structured and unstructured data. By incorporating LLMs, FlockMTL enables semantic analysis and hybrid search capabilities within SQL queries.
The system works by allowing users to define reusable resources in the form of models and prompts. These resources can be scoped to a specific database or made available globally across all databases on a machine. Users can then write SQL queries that incorporate these resources, using functions such as filtering, summarization, and extraction to analyze their data.
One key feature of FlockMTL is its ability to automatically optimize the use of LLMs for improved efficiency. For example, the system can batch tuples together for a single API call, reducing the number of requests made to the LLM. It also caches predictions for reuse within and across queries, ensuring that repeated calculations are avoided.
A demonstration of FlockMTL’s capabilities has been released, allowing users to interact with multiple tabular datasets sourced from Kaggle. The system can generate SQL queries augmented with FlockMTL functions in response to natural language input, and provides a plan inspection interface for debugging and analysis.
The integration of LLMs into relational databases has the potential to revolutionize the way we analyze and manipulate data. By enabling semantic operations within SQL, FlockMTL opens up new possibilities for users to extract insights from their data without requiring extensive programming expertise.
However, there are still challenges to be addressed before this technology can be widely adopted. For example, the use of LLMs requires a significant amount of computational resources and storage space, which can be a limitation for smaller databases or those with limited budgets.
Despite these challenges, FlockMTL represents an important step forward in the development of AI-powered databases. As the technology continues to evolve, it will be exciting to see how users leverage its capabilities to gain new insights from their data.
Cite this article: “Unlocking the Power of Language Models in Relational Databases”, The Science Archive, 2025.
Artificial Intelligence, Databases, Large Language Models, Relational Databases, Flockmtl, Duckdb, Sql Queries, Semantic Analysis, Hybrid Search Capabilities, Open-Source Extension.
