Thursday 23 January 2025
Editing charts and graphs can be a tedious and time-consuming task, especially when dealing with complex data sets or low-resolution images. But what if you could simply tell your computer what changes you want to make, and it would do the work for you? Enter PlotEdit, a new system that uses artificial intelligence to edit chart images based on natural language instructions.
PlotEdit is designed to tackle the common problem of editing charts in PDFs or scanned documents. These files often contain valuable information, but are difficult to modify without specialized software or expertise. Traditional methods require manual intervention and can be prone to errors, making it a time-consuming and frustrating process.
The PlotEdit system uses a framework of five artificial intelligence agents to accurately edit chart images based on user specifications. The first agent, Chart2Table, extracts data tables from the original chart image, while the second agent, Chart2Vision, identifies stylistic attributes such as plot colors, fonts, and labels. The third agent, Chart2Code, retrieves rendering code that specifies the chart type and layout components.
But here’s where PlotEdit gets really smart. The system uses three unique multimodal feedback signals – numeric, visual, and code-based – to rectify any errors during de-rendering. This means that if the AI makes a mistake while extracting data or identifying visual attributes, it can be corrected through visual comparison of the original chart with the replotted version.
The Instruction Decomposition Agent then breaks down the user’s editing request into a sequence of executable steps, using natural language processing to understand what changes need to be made. The Multimodal Editing Agent uses these instructions to modify the data table, visual attributes, and code, making sure that only the specified changes are implemented while preserving the integrity of the original chart.
The result is a system that can accurately edit charts in PDFs or scanned documents, without requiring extensive expertise or manual intervention. This has significant implications for professionals who work with data analysis and visualization, as well as individuals who need to modify complex data sets for publication or presentation.
In testing, PlotEdit outperformed existing baselines across various edit types, including style, layout, format, and data-centric edits. The system’s ability to use multimodal feedback signals to correct errors during de-rendering was a key factor in its success, allowing it to accurately implement user specifications while preserving the integrity of the original chart.
Overall, PlotEdit represents a significant advancement in the field of natural language-driven chart image editing.
Cite this article: “Intelligent Chart Editing with PlotEdit”, The Science Archive, 2025.
Artificial Intelligence, Chart Editing, Natural Language Processing, Multimodal Feedback, Data Analysis, Visualization, Pdf Editing, Scanned Documents, Image Editing, Chart Image Editing.
