GRASP: Municipal Budget AI Chatbots for Enhancing Civic Engagement

Abstract—There are a growing number of AI applications, but none tailored specifically to help residents answer their questions about municipal budget, a topic most are interested in but few have a solid comprehension of. In this research paper, we propose GRASP, a custom AI chatbot framework which stands for Generation with Retrieval and Action System for Prompts. GRASP provides more truthful and grounded responses to user budget queries than traditional information retrieval systems like general Large Language Models (LLMs) or web searches. These improvements come from the novel combination of a Retrieval- Augmented Generation (RAG) framework (“Generation with Retrieval”) and an agentic workflow (“Action System”), as well as prompt engineering techniques, the incorporation of municipal budget domain knowledge, and collaboration with local town officials to ensure response truthfulness. During testing, we found that our GRASP chatbot provided precise and accurate responses for local municipal budget queries 78% of the time, while GPT- 4o and Gemini were only accurate 60% and 35% of the time, respectively. GRASP chatbots greatly reduce the time and effort needed for the general public to get an intuitive and correct understanding of their town’s budget, thus fostering greater communal discourse, improving government transparency, and allowing citizens to make more informed decisions.

We used LangChain’s Agent framework to implement the agentic workflow with the following steps:

1. Create each tool as a derived class of the LangChain’s

BaseTool class and implement each tool’s business logic

in the run/arun methods

2. Use LangChain’s ChatOpenAI to construct the LLM and

bind it to the list of tools

3. Setup the agent prompt template using LangChain’s ChatPromptTemplate class with the following components:

• LLM system instructions

• Chat history (using LangChain’s MessagesPlaceholder

class)

• Agent scratchpad, used by the agent to keep track

of intermediate steps (also using LangChain’s MessagesPlaceholder

class)

• User prompt

4. Instantiate LangChain’s AgentExecutor with the agent prompt template, LLM, and LangChain’s OpenAIToolsAgentOutputParser (which processes tool output). The AgentExecutor will be run each time a user passes in the query.

C. Prompt Engineering

We employ prompt engineering on two types of prompts: 1) User prompt: due to the conversational nature of the interactions between a user and the chatbot, many queries must be implied based on the chat history. For example, if a user asks “What was the municipal school budget in FY2025?” followed by “What about the two years before?”, the chatbot would imply the second query actually means “What was the municipal school budget in FY2023 and FY2024?”. While the chatbot can understand these implied queries phrased exactly as the user typed them, such queries are usually lacking the keywords required for the similarity search performed by the RAG framework (in this case, the keywords would be school budget, FY2023, and FY2024). Hence, we instruct the LLM to rephrase queries to explicitly state all necessary information using the below prompt:

Rephrase the following query to explicitly state the question and the year(s) in which the question is being asked: {currentQuery} There may be additional context that is found in the previous user query: {lastQuery} We then use the rephrased query for similarity search to enhance the relevance of retrieved documents and thus truthfulness.

2. System prompt: we have been working with local officials to find domain-specific knowledge regarding municipal budget. We incorporate this domain knowledge into the LLM’s system prompt.