Spatial Intelligence with Local LLMs using an MCP Server
A prototype for a Model Context Protocol (MCP) server that controls a map using small on-device LLMs.
Project Name
Project Date
Affiliation
Team
Joshua's Contribution
It is difficult for non-experts to build an intuitive understanding of how AI usage creates environmental and productivity impacts, partly owing to disagreements over how to quantify such impacts. We propose that interactive visualisation can help build intuition for navigating these disagreements. butterfl.ai is an interactive installation comprising two stations. Each station is powered by a computer connected to a primary monitor and a larger screen (or projector screen). The primary monitor shows a chatbot interface as well as a prototype AI-powered geospatial visualisation tool. These two pieces of software are connected through a Model Context Protocol (MCP) server, demonstrating a realistic use case for tool-enabled AI. The larger screen shows an interactive visualisation demonstrating the energy, environmental, and productive impact of the user's activity. butterfl.ai aims to help people more tacitly understand trade-offs between inference speed, output quality/usefulness, resource cost, and human effort. butterfl.ai is an Interactivity Track submission to the ACM Designing Interactive Systems 2026 conference.
The push across organisations and countries to adopt large language model-driven agentic AI workflows has led to debates over whether productivity “gains” are meaningful enough to offset the costs, not only in terms of API bills and direct energy usage but also in terms of the pressure placed on infrastructure. This question is particularly urgent in energy-intensive sectors like the built environment industry.
AI is positioned as a way to make design processes more efficient and improve environmental outcomes, but the demand for data centre capacity arguably drives the worst infrastructure expansion and land use changes we've seen in recent years.
It is difficult for non-experts to build an intuitive understanding of the environmental and productivity impacts of large language models (LLMs). How much energy exactly do LLMs use for real-world tasks? Are agentic AI systems really cheaper or faster than human effort when it comes to unstructured real-life work problems? Might a larger model justify itself by doing a task in less time, or can smaller ones utilise tools to get things done?
butterfl.ai is an interactive exhibit where users can try controlling a GIS app through a chatbot. With every interaction, a larger screen visualises the impact of the user's interactions, from cost to energy usage.
This exhibit came out of ongoing work at the Singapore-ETH Centre Software Chapter, together with the Future Cities Laboratory Global Engagement Platform team, exploring if smartly-designed Model Context Protocol (MCP) servers can create more intelligent agentic AI systems for geodesign using smaller language models that run entirely on-device. That project in turn was inspired by the principles of the Public AI initiative.
Click on an image to expand it.
Tap on a photo to expand it.


butterfl.ai is an Interactivity Track submission to the ACM Designing Interactive Systems (DIS) 2026 conference. It was accepted and will be exhibited at the conference in Singapore in June 2026.
DIS 2026 Conferencecode + design + sound © 2026 joshua vargas