Friday 31 January 2025
The kitchen can be a daunting space, especially for those who are new to cooking or lack confidence in their culinary skills. A team of researchers has developed an innovative solution to make cooking easier and more accessible: an iOS-based augmented reality (AR) cooking assistant powered by Google’s Gemini Large Language Model.
This prototype application uses Apple’s ARKit to create a seamless AR experience on iOS devices, allowing users to scan their ingredients and generate recipes in real-time. The Gemini model is used to identify the ingredients and suggest meals that can be made using only what the user has on hand. The app also provides personalized nutrition tracking, including information on calories, fat, carbohydrates, and protein.
One of the key features of this application is its ability to detect ingredients using computer vision (CV) technology. When a user scans an ingredient, the Gemini model uses the image recognition capabilities to identify it and add it to a list of detected ingredients. This information is then used to generate recipe suggestions that take into account the ingredients the user has available.
The app also includes a voice assistant and text-based chatbot that can help users with cooking-related questions and provide step-by-step instructions for each recipe. Additionally, the application features interactive cooking aids such as timers and shopping list generators.
In order to test the effectiveness of this application, the researchers conducted three rounds of user experience surveys with 21 volunteers. The results showed a significant improvement in usability scores over the course of the surveys, indicating that users were highly satisfied with the app’s performance.
This innovative solution has the potential to make cooking easier and more accessible for people of all skill levels. By providing personalized recipe suggestions based on available ingredients and offering interactive cooking aids, this application can help reduce food waste and improve overall cooking experiences.
The researchers’ use of Apple’s ARKit and Google’s Gemini Large Language Model demonstrates a commitment to using cutting-edge technology to develop practical solutions for real-world problems. This app has the potential to make a significant impact on people’s daily lives and could be an important step towards making cooking more accessible and enjoyable for everyone.
Cite this article: “Intelligent Cooking Companion”, The Science Archive, 2025.
Ios, Ar, Google Gemini, Cooking Assistant, Apple Arkit, Computer Vision, Nutrition Tracking, Recipe Suggestions, Voice Assistant, Chatbot, Cooking Aids.
