How Philadelphia's Public Records Got Buried Under Years of Duplicate Images — and What the City Is Doing About It

Philadelphia's Office of Innovation and Technology has been working through a backlog of digitized public records that city archivists say is riddled with duplicate image files — redundant scans that have clogged the municipal document management system, slowed retrieval times, and made it harder for residents, lawyers, and researchers to access city records through the PhilaDocs portal.

The problem did not appear overnight. It is the compounded result of at least three separate digitization waves stretching back to the early 2000s, each of which ingested material into separate systems that were later merged without adequate deduplication protocols. Every merger left ghost copies behind.

A Paper Trail That Grew Its Own Shadow

The first large-scale scan campaign began around 2003, when the Philadelphia City Archives on Broad Street partnered with a vendor to digitize property deed records dating to the 1790s. A second push came after 2012, when the city began moving departmental files onto a centralized content management platform. Then a third round followed the post-pandemic push to expand remote access to government documents, which accelerated sharply between 2020 and 2022. Each round pulled from the same underlying paper collections, and without a shared unique identifier system locking records together, the same image — sometimes the same physical page — could end up scanned, uploaded, and catalogued three separate times under slightly different file names.

Staff at the Free Library of Philadelphia's Government Publications section on Vine Street have fielded complaints from researchers who say searching the city's online portals returns the same document multiple times, inflating apparent search results and complicating citation work. Civic technologists affiliated with Code for Philly, the volunteer brigade that works on open government tools, flagged the issue in internal discussions as far back as 2021, noting that duplicates were consuming a disproportionate share of server storage and degrading search relevance scores.

The scale is significant. City officials have not released a full public accounting, but IT procurement documents reviewed for a related infrastructure contract in fiscal year 2025 referenced a content repository holding more than 40 million document images across city departments. Even a conservative duplication rate of 5 percent would mean roughly 2 million redundant files sitting in the system — each one occupying storage, slowing indexing, and potentially misleading anyone who interprets document count as a proxy for record completeness.

Cleanup Efforts and What They Mean for Residents

The Office of Innovation and Technology, headquartered at 1234 Market Street, launched a deduplication initiative under its broader Smart City PHL program in late 2024. The effort uses hash-based file comparison — a standard technique that generates a unique digital fingerprint for each image — to flag identical files before a human reviewer makes a final deletion decision. The goal is to retire redundant copies without destroying any record that might be a legitimate variant, such as a re-executed document or a corrected version of an earlier filing.

For ordinary Philadelphians, the practical stakes are real. Property owners in neighborhoods like Fishtown and West Philadelphia who pull deed records to settle boundary disputes or verify renovation permits depend on accurate document retrieval. Title insurance companies operating in the city routinely run searches through the same system. Attorneys at community legal aid organizations — including Philadelphia Legal Assistance, which serves low-income residents across the city — have noted that duplicate records create confusion about which version of a document is authoritative.

The cleanup is ongoing, and officials have not announced a completion date. What archivists and civic technologists tend to agree on is that the deeper fix requires something the city has resisted committing to fully: a persistent, department-wide unique identifier assigned to every physical record before it is ever scanned, so that no future digitization wave can produce the same tangle. Until that standard is in place, the deduplication work being done now risks becoming a one-time clearing of a drain that will clog again. Residents who believe a specific city record may have been affected can contact the Philadelphia City Archives directly at 3101 Market Street.