In low-stakes testing contexts, there are minimal personal consequences associated with examinee performance. Examples include assessments administered for research, program evaluation, test development, and international comparisons (e.g., Programme for International Student Assessment [PISA]). Because test-taking motivation can suffer in low-stakes conditions, the Student Opinion Scale (SOS) is commonly administered to measure test-taking effort and how personally important the examinee feels it is to do well on the test. Although popular, studies of the scale's internal validity yield conflicting findings. The present study critically evaluates the creation of the SOS and considers its factor structure across six samples of college students differing in their college experience level and version of the SOS administered. Because findings only support the internal validity of the effort subscale, further study and development of the importance subscale is recommended.