400% increase in number of cities and utilities used to generate actionable insights from infrastructure projects
433% increase in relevant documents identified, boosting the depth and breadth of data extraction
71% reduction in average document processing time, expediting analysis
7.5 million new infrastructure document segments processed into clear, actionable intelligence each month
Citylitics built a predictive intelligence platform on Google Cloud that uses advanced AI to support data-driven decision-making about North American infrastructure investments.
Citylitics built a predictive intelligence platform on Google Cloud that uses advanced AI to support data-driven decision-making about North American infrastructure investments.
Enabling data-driven infrastructure investment decisions
Over coffee in Seattle a few years ago, Sunit Mohindroo and Ahmed Badruddin explored how they could leverage data analytics and their engineering backgrounds for meaningful societal impact. "Trillions of dollars worth of infrastructure project decisions are made every day by cities and utilities, but predictive signals that can significantly influence these projects' costs and timelines are buried in millions of unstructured documents that no one has the time to analyze or the right lens to interpret it all together," says Mohindroo.
Mohindroo and Badruddin launched Citylitics in 2018. Their goal was to empower municipalities and utilities – and the architecture, engineering, and construction (AEC) firms supporting them – with the intelligence to better understand and forecast critical infrastructure projects.
With the help of Google Cloud, we reduced average document processing time by 71%. We can now source more data inputs, achieving a 400% increase in represented cities and utilities and a 433% increase in identified infrastructure documents.
Sunit Mohindroo
Chief Product Officer, Citylitics, Inc.
They initially focused on reliably extracting specialized infrastructure-related signals from massive datasets that were fragmented, inconsistent, and largely unstructured. It quickly became evident just how challenging it would be to translate all of this data into actionable market intelligence.
As Citylitics grew, its complex tech infrastructure became increasingly hard to manage, with negative repercussions. "Given the volume and complexity of data we need to capture and process, we needed to ensure that our data assets were as accessible and reliable as possible," explains Mohindroo.
The company needed a solution to easily scale its data pipeline operations, reduce operational overhead, and provide fresher, more comprehensive data to its customers. Citylitics turned to Google Cloud thanks to two key services: Dataflow and Cloud Run.
Using Vertex AI to deliver relevant, timely insights for better investment choices
By extracting and transforming fragmented, unstructured public data from diverse sources into structured, digestible insights on a single platform, Citylitics is enabling communities across North America to benefit from a communal repository of intelligence that reduces their planning timelines and design costs for major infrastructure projects.
The data and insights Citylitics provides have recently helped decision-makers evaluate the necessity, the strategic placement, and financial viability of new forms of physical infrastructure, like broadband and EV charging.
"The infrastructure industry is naturally risk-averse given its prioritization of public safety and infrastructure security, so it's essential that city officials and project stakeholders see proof-of-concept installations of new infrastructure in environments similar to their own before dollars are allocated," explains Mohindroo. "They want to understand the trends and themes, learn from the actions of other comparable communities, and they want to be data-informed in their own decisions, similar to how prospective homeowners review real estate comparables."
We are especially excited about our recent developments that leverage agentic applications on top of our data using Google's Agent Development Kit (ADK). Thanks to the impressive efforts of our Engineering Teams, our data is well-formed and provides a strong basis for model training. This strong data foundation allows agents to generate accurate, complete suggestions that our customers can use with confidence.
Yannish Sewraz
Head of Data Engineering, Citylitics
Citylitics aggregates data from over 40,000 sources, demanding a flexible approach to structuring and extracting meaningful signals from raw information. Ensuring traceability of these data sources is essential to building trust with Citylitics' core customers — the AEC firms that provide their expertise before and during municipal and utility infrastructure projects. Vertex AI provides Citylitics with the capabilities needed to achieve this level of precision and transparency.
As a participant in the Google Cloud for Startups program, the Citylitics team began utilizing Vertex AI in 2021 to improve signal identification during data ingestion processes, and enhance downstream data workflows, refining the context and clarity of each insight delivered to platform users. They subsequently deployed advanced cascading layers of natural language processing, traditional machine learning, and, ultimately generative AI, to supply deeper associations between their datasets, and unlocking a new recommendation engine capable of deriving personalized insights from massive volumes of unstructured text data.
Leveraging the vector database capabilities of BigQuery with Vertex AI drove even greater efficiencies, says Mohindroo, "By storing and analyzing large volumes of data directly in BigQuery, we can train our ever-evolving algorithms to quickly find high-value documents within vast and varied datasets."
The impact has been enormous: Citylitics' foundational dataset now covers over four times as many cities and utilities, along with a 433% increase in identified infrastructure documents — highlighting data and insights decision-makers likely overlooked before using Citylitics.
Re-architecting for a stronger, more scalable technology foundation
The Citylitics team was ambitious. They wanted comprehensive data coverage and freshness; faster time to market for new data sources; efficient engineering resource allocation; cost predictability and control; and an ever-reliable data foundation.
The company decided a fundamental re-architecture of its core data ingestion and web crawling pipeline was the only way forward. They wanted to transition from a self-managed, infrastructure-heavy environment to a fully managed, serverless, and cloud-native framework, built to handle even more terabytes of processed data.
After evaluating options, Citylitics chose to center their new architecture on Google Cloud's serverless compute and data processing services enabling them to efficiently manage large-scale web crawling, document storage, data extraction, and downstream processing with much-needed flexibility and scale.
Cloud Run provided Citylitics with the environment for stateless, highly scalable instances to handle the particularly nuanced and challenging elements of unstructured data ingestion and pre-processing, removing most of the infrastructure management overhead and letting the team focus on surfacing the value of the intelligence they uncover. Dataflow then became the backbone for orchestrating and processing the vast streams of data, handling parallelism and fault tolerance without breaking Citylitics' processes.
This new Google Cloud-based architecture gave Citylitics a straightforward path to rapid scaling while keeping costs under control – critical for any fast growing company managing an ever-expanding volume of raw data. "More importantly," says Mohindroo, "It kept our focus almost entirely on the core code itself — solving the complex challenge of extracting and processing the most relevant data for our users — rather than getting bogged down in managing sophisticated cloud infrastructure."
While the strategic shift and re-architecture required significant effort and conviction, the technical hurdles we faced before adopting Cloud Run and Dataflow have largely been addressed by the capabilities and flexibilities of these services.
Sunit Mohindroo
Chief Product Officer, Citylitics, Inc.
Citylitics provides predictive intelligence on where, how, and why infrastructure investments will be made in North America. Transforming over a billion documents buried in 40,000 cities' and utilities' data sources, Citylitics delivers high-value insights for business development by architecture, engineering, and construction firms.
Industry: Technology
Location: Canada
Products: Google Cloud, BigQuery, Cloud Run, Cloud Storage, Dataflow, Vertex AI