This article examines a critical issue in digital literacy assessment design when technological changes are happening with escalating speed in our society. There have been many assessment studies of digital literacy (DL) for diverse purposes and across different geographic and socioeconomic (geo-socioeconomic) contexts. While the assessment framework, instrument design, and technology platforms used for conducting these assessments differ, what remains common is the lack of explicit discussion about the possible role of the technology used and item design in affecting the measure DL. There is an apparent, implicit assumption that DL assessment is similar to the assessment of other academic achievements such as reading literacy and numeracy, which should ideally be measured independent of the specific technologies or task contexts adopted in the assessment. Recent evidence from a United Nations Educational, Scientific and Cultural Organization (UNESCO) commissioned study on a Digital Literacy Global Framework (DLGF) shows that the DL needed to accomplish the same task is heavily dependent on the devices and tools used under different the geo-socioeconomic contexts (Law et al. in A global framework of reference on digital literacy skills for indicator 4.4.2, 2018). Drawing on the DLGF findings and a critical examination of the assessment designs in large-scale international assessment tests, this paper puts forward a product-lifecycle and experience dependence (PLED) perspective to guide the design and interpretation of DL assessment.