Philadelphia's Archives Scramble to Fix Duplicate Image Problem as Digital Records Backlog Grows

Philadelphia's push to digitize decades of municipal and historical records has run into a stubborn, unglamorous problem: thousands of duplicate images sitting inside public databases, eating up server space, confusing researchers, and in some cases overwriting the originals they were meant to preserve. The issue surfaced prominently this week after staff at the Philadelphia City Archives, located on Cabot Street in the Northeast, flagged a backlog of roughly 14,000 redundant image files accumulated since a 2023 mass-scanning initiative began.

The timing matters. Fourth of July weekend typically draws thousands of visitors to historical sites across the city — from Independence Hall on Chestnut Street to the Betsy Ross House on Arch Street — and staff at several institutions said the duplicate-image problem has been quietly slowing down public-facing search tools for months. With summer tourism at its peak and extreme heat already canceling outdoor events citywide this weekend, more Philadelphians than usual are expected to turn to digital portals to access local history. A broken or unreliable archive search is not a minor inconvenience right now.

What Went Wrong — and Where

The problem traces back to how multiple city departments, including the Department of Records and the Office of Innovation and Technology, handled file transfers during the 2023 digitization push. Scanning contractors, working under tight deadlines, submitted image batches that were ingested without deduplication checks. The Philadelphia Free Library's digital collections portal, which hosts neighborhood photograph collections going back to the 1870s, absorbed some of those duplicates when archivists attempted cross-agency data sharing in late 2024.

The Library's Digital Collections team, based at the Parkway Central branch on Vine Street, began a systematic audit in May 2026 after cataloguers noticed that certain search queries — particularly for Kensington and Fishtown neighborhood images — were returning the same photographs under different accession numbers. In some instances, a single image appeared under four or five distinct catalog entries, each with slightly different metadata, making it impossible for researchers to know which record was authoritative.

The City Archives estimates the deduplication effort will require reviewing approximately 38,000 image files in total, of which around 14,000 are confirmed or suspected duplicates as of this week. Correcting the metadata on each verified file takes an average of 12 to 18 minutes of staff time, according to internal workflow documentation the department shared with community stakeholders at a June 30 public meeting. At current staffing levels, the full correction is projected to take until at least March 2027.

What Institutions Are Doing Now

The Free Library's digital team has prioritized the Kensington and South Philadelphia collections first, given high researcher demand from academics at Drexel University and Temple University's Special Collections. Staff are using open-source deduplication software to flag near-identical files before human reviewers make final calls on which version to keep. The Library has also temporarily added a visible advisory banner to its digital portal — as of July 2 — warning users that some photograph collections may show incomplete or duplicated results during the audit period.

The City Archives, separately, is working with the Office of Innovation and Technology to build an automated checksum system that would catch duplicate uploads at the point of ingest, preventing the problem from recurring. That system is currently in a testing phase and is not expected to go live before the fourth quarter of 2026.

For residents and researchers who rely on these collections, the practical advice right now is straightforward: cross-reference any image found in the Free Library's portal with the City Archives' standalone catalog before citing or downloading it for formal use. If both databases return the same image under different accession numbers, users are encouraged to report the discrepancy through the Archives' online feedback form, which feeds directly into the deduplication audit queue. Community historians affiliated with groups like the Preservation Alliance for Greater Philadelphia have already begun doing exactly that, helping staff catch errors that automated tools miss. The more eyes on the database this summer, the faster the backlog clears.