Diagnosing a Failing Smart Water System

How Raybern Consulting uncovered the root cause of an AMI system failure.

Background & Challenge

A large U.S. city invested in an Advanced Metering Infrastructure system in 2013, deploying approximately 30,000 smart water meters. Within eight years, the system was failing on every front: over 30% of monthly bills were estimated, data collectors went offline for days at a time, endpoints took additional days to resume reporting after collector reboots, and more than 30% of endpoints were non-active — some since the year of installation. A customer usage portal intended to improve transparency instead exposed data gaps to residents and triggered a surge in complaints.

Our Approach: Equipment vs. Data

Raybern centered the engagement on one critical question: is this a data problem or an equipment problem? The answer would determine everything.

• Billing Data Analysis — Compared the city's billing and AMI databases and found them fully in sync. The vendor's custom interface was working as intended, ruling out data integrity issues and pointing squarely to faulty equipment.

• Gap Analysis — Benchmarked contract terms against industry best practices and actual delivery. Key gaps: no contingency read device for field staff, no endpoint diagnostic tools, minimal maintenance despite an active support contract, and warranty terms far below what the contract implied. Notably, adding three new collectors caused non-performing endpoints in those areas to grow by the hundreds.

• Data Cross-Reference — 60% of accounts with estimated bills had non-active endpoints. Of those, hundreds had never produced a single read — every bill estimated from day one.

• Broader Research — Identified a consistent pattern of identical failures at other utilities using the same system.

Key Findings

Faulty Equipment, Not Data Billing and AMI databases were in sync. The root cause was defective hardware — not integration or configuration failures.	Contract Gaps The support contract was not being executed. No contingency reads, no diagnostic tools, and warranty terms far below industry standards.
Systemic Vendor Failures Identical failures were documented at multiple utilities using the same platform.	Portal Backfire Launching a customer-facing usage portal on a broken system exposed data gaps to residents and multiplied service call volume.

Recommended Paths Forward

• Negotiate with the vendor — leverage documented precedents to secure remediation or full system replacement at little to no cost.

• Procure a new system independently — recommended including Network-as-a-Service agreement to mitigate risk of future failures.

• Implement immediate operational fixes — contingency read methods, AMI monitoring programs, and electronic work order management to reduce staff burden during negotiations.

Key Takeaways

🔍 Data vs. Equipment Separating data failures from equipment failures early determines the most cost-effective path forward.

💬 Portal Timing Never launch a customer-facing portal on a broken system — it multiplies complaints and destroys confidence.

🚫 Contingency Reads Without a fallback for non-reporting endpoints or collector failures, estimated bills are inevitable regardless of system design.

Facing similar challenges? Raybern Consulting helps water utilities diagnose AMI failures, correct billing data, and build a clear path forward.

www.raybernconsulting.com