Open educational resources (OER), are openly licensed text, media, and other digital and analog assets that are useful for teaching, learning, and research. Recent research has shown that in courses where open textbooks are assigned, students perform as well as or better than students in similar courses with commercially licensed textbooks. Despite cost savings and demonstrated effectiveness, perceptions about quality, relevance, ease of access, and other concerns persist. The objectives of this study were twofold: (a) to develop a practical and reusable measure for evaluating open textbook quality in terms of pedagogy, openness, accessibility, and relevance; and (b) to use the measure to rate the quality and relevance of open textbooks for use in higher education in The Bahamas. The study confirmed the viability of the quality measure as a practical tool to assess open resources and found that the open textbooks studied were accessible and well matched to course content, but of varying quality. More study is needed to explore ways to increase faculty adoption, use, adaption and production of OER.