Friday 28 March 2025
The quest for efficient artificial intelligence (AI) has led researchers to explore new frontiers, particularly in the realm of embedded systems. The Nvidia Jetson boards, with their powerful GPU hardware and widely supported software stack, have emerged as a promising platform for executing AI workloads in edge environments.
However, these boards offer more than just processing power; they also boast large configurability, allowing users to modify various hardware parameters such as CPU frequency, GPU frequency, and EMC frequency. This flexibility creates a vast design space that requires intelligent search algorithms to explore and identify the optimal configurations based on user requirements.
To address this challenge, researchers have developed JExplore, a multi-board software and hardware design space exploration tool. JExplore provides a common benchmarking ground for search algorithms, allowing developers to quickly test and compare their approaches. The tool also accelerates the exploration of user applications and Nvidia Jetson configurations by encapsulating host-client communication, configuration management, and metric measurement.
JExplore’s architecture is designed to be easy to use, with two main components: JHost and JClient. JHost runs on the host computer, serving as an interface between the search algorithm and the Jetson boards. It communicates with multiple Jetson devices over SSH, using ZMQ Push/Pull sockets to transfer data.
The JClient component runs on each Jetson device, responsible for configuring the board and software parameters based on input from JHost. The workload is then executed, and measurement results are sent back to JHost. This seamless communication enables efficient exploration of the design space.
To demonstrate JExplore’s capabilities, researchers implemented two generative AI workloads: Llama2-7B and LLaVA-v1.5-7B. These models take input text prompts or images and generate text based on the inputs. The team randomly sampled 200 Nvidia Jetson Orin configurations from the search space and used JExplore to collect results.
The data revealed interesting patterns, with power consumption and inference latency showing inverse correlations. A clear pareto frontier emerged in the plot, indicating that as power consumption increased, inference time decreased. This relationship is critical for AI workloads, which often require a balance between processing speed and energy efficiency.
JExplore’s capabilities were also demonstrated by analyzing the data points that showed up in separate clusters from the main data points. These clusters corresponded to specific hardware parameter settings, such as EMC frequency, which had a cutoff effect on inference time.
Cite this article: “Exploring AI Workloads on Nvidia Jetson Boards with JExplore”, The Science Archive, 2025.
Artificial Intelligence, Embedded Systems, Nvidia Jetson, Gpu Hardware, Software Stack, Design Space Exploration, Jexplore, Search Algorithms, Generative Ai Workloads, Power Consumption
