Unlocking Trends in Local Authority Productivity: AI Powered Analysis for the LGA

The Local Government Association (LGA) engaged Local Partnerships to analyse 317 productivity plans and create an AI powered chatbot to make insights from the data more accessible.

Local Partnerships’ Data & Analytics team orchestrated a full-spectrum analytical solution to meet the LGA’s requirements. We expertly pre-processed all 317 plans, ensuring data quality, consistency, and readiness for advanced analysis. We conducted word-frequency visualisations, n-gram and topic modelling, and keyword extraction to surface recurring themes and strategic priorities across the plans. These findings were synthesised into a comprehensive report.
To bring data to life, we implemented a sophisticated RAG chatbot using a local-first LLM approach—ensuring secure handling of sensitive documents while allowing users to interact contextually with the data. This solution enabled rapid insight generation, intuitive querying, and enhanced data accessibility, transforming manual review into an AI-powered, efficient, and secure exploration experience.

Client requirements

LGA commissioned Local Partnerships to undertake a comprehensive textual analysis of 317 productivity plans submitted by various local authorities. The primary objective was to “identify key themes and trends” embedded within these plans. Additionally, the LGA required an in-depth review of a 10% sample (32 plans) selected from a diverse range of local authority types, to provide more granular insights into their content. A crucial part of the requirement was to integrate these 300 documents into an AI Large Language Model (LLM) RAG (Retrieval Augmented Generation) chatbot, enabling interactive exploration of the data.

How we supported the client 

Our Data & Analytics team addressed the LGA’s requirements through a multi-faceted approach, leveraging advanced analytical techniques and AI capabilities:

• Text Pre-processing: We began by meticulously pre-processing the 317 productivity plans to ensure data quality and consistency, preparing them for robust analysis.
• Word Frequency Analysis and Visualisation: We calculated word frequencies to identify common terms and visualised these to highlight prevalent concepts and language used across the plans.
• N-gram Analysis: We employed n-gram analysis to identify frequently occurring phrases and multi-word expressions, revealing deeper insights into recurring themes and approaches.
• Topic Modelling: Advanced topic modelling techniques were applied to analyse the distribution of various topics across all the plans, providing a high-level overview of the strategic focus areas.
• Keyword Extraction: Key information and actionable insights were extracted from the documents through targeted keyword extraction.
• Comprehensive Report Generation: The findings from the textual analysis were synthesised into a comprehensive report summarising the key themes and trends identified.
• AI LLM RAG Chatbot Implementation: We engineered a sophisticated RAG model approach to ingest all 300 documents into an AI chatbot. This involved:
  • Contextual Data Interaction: Utilising a combination of local LLM models and chatbot models, allowing users to “talk to the data” for context and retrieve related documents within the collection.
  • Custom Prompt Engineering: Developing custom prompts to effectively query the LLM, ensuring the identification of key areas aligned with the LGA’s requested deliverables.
  • Local-First Approach: Implementing a local-first approach for the RAG model, ensuring a secure environment for sensitive documents and enabling focused analysis on specific datasets.

The impact

As a direct result of our work:

Can we help?

Do you have a project you would like help with, or could we answer a question about our services? Please message us here, contact Joran Mendel directly or fill in the form on this page. We would be delighted to hear from you.