Thursday 23 January 2025
This paper presents a comprehensive approach for integrating Large Language Models (LLMs) into existing user interfaces (UIs) in a non-intrusive manner, focusing on the scholarly knowledge organization domain. The authors propose guidelines for LLM integration, which are grounded on usability heuristics and aim to facilitate the adoption of LLMs in real-world systems.
The paper describes six use cases implemented within the Open Research Knowledge Graph (ORKG) UI, including closed recommendations and open feedback features. These use cases demonstrate how LLMs can assist users in tasks such as recommending related predicates, objects, or literals, as well as providing feedback on the decomposability of resources, predicate reusability, and comparison descriptiveness.
The authors conducted a user study with six ORKG curators to evaluate both the approach and implementation. The results showed that participants found the Smart Suggestions helpful and appreciated the ability to receive open-ended feedback from LLMs. However, they also noted that LLM responses were not always accurate, which highlights the importance of human verification in knowledge graph curation.
The paper concludes by discussing limitations and future work, including plans for extending the implemented use cases and performing larger-scale evaluations based on real-world usage.
Overall, this paper provides valuable insights into the challenges and opportunities of integrating LLMs into existing UIs, particularly in the context of scholarly knowledge organization. The proposed approach and guidelines can serve as a foundation for further research and development in this area.
Some potential extensions or refinements to the work include:
1. Evaluating different LLM models and fine-tuning them for specific tasks.
2. Conducting larger-scale user studies to gather more comprehensive feedback on the usability and effectiveness of the Smart Suggestions.
3. Exploring additional use cases and domains where LLMs can be integrated into existing UIs.
4. Developing more sophisticated methods for evaluating the accuracy and reliability of LLM responses.
5. Investigating ways to improve the transparency and explainability of LLM decision-making processes.
By addressing these challenges and opportunities, researchers and developers can continue to advance our understanding of how LLMs can be effectively integrated into existing UIs, ultimately improving the usability and effectiveness of these systems for users.
Cite this article: “Integrating Large Language Models into Existing User Interfaces: A Comprehensive Approach”, The Science Archive, 2025.
Large Language Models, User Interfaces, Knowledge Organization, Scholarly Knowledge, Integration, Usability Heuristics, Open Research Knowledge Graph, Natural Language Processing, Human Verification, Ai Evaluation
