Longitudinal Study of Web Page Lifespans and Evolution via Wayback Machine Sampling

To quantify web page longevity, our study samples from the Internet Archive’s Wayback Machine to curate the NYPW Sample, comprising 27.3 million HTML URLs spanning 26 years of web history (1996–2021). Our empirical findings indicate an average web page lifespan of 5.0 years (median 2.3 years), with a substantial disparity between root URLs (median 8.8 years) and deep links (median 1.3 years). Traditional quantitative analyses of webpage longevity generally consider a URL as a static identifier, which may not capture the complex lifecycles in which underlying content, ownership, or semantic purpose changes while a link remains active. To address this limitation, we proposed the Multi-Resident URL framework, which models six distinct lifecycle phases and formally separates structural URL stability from evolving content identity. Furthermore, our redirection analysis demonstrates the fragility of relocated content, revealing that only 50% of redirecting URIs resolve to active pages and illustrating how redirection shapes domain states and creates sink URIs. Through an integrated analysis of web resource survival, temporal stability, URL lifecycle evolution, and redirection behavior, we aim to improve the accuracy of web-archive studies, reduce reliance on detailed memento analysis, advance archiving practices, and deepen understanding of long-term web content persistence.