The students are urged to do something without expecting anything in return and only in the name of God. Every islamic student becomes something ideal if they can internalize and implement sincerity. Many people are willing to do something because of an ulterior motive. The importance of sincerity in humans is the background for developing a sincerity scale that measures personal sincerity. One of the figures who gave an example of sincere behavior was Kyai Haji (KH) Ahmad Dahlan, so the sincerity scale was adjusted to the character's sincere behavior. This scale consists of four indicators: having transcendental motives, the ability to control emotions, the absence of a sense of superiority, and being a servant of God. Two experts in Islamic psychometry tested KH Ahmad Dahlan's version of the sincerity scale. Besides that, construct validity involved 252 students. We employed Cohen's Kappa inter-rater reliability (IRR) coefficient and Rasch analysis as data analysis methods. The findings from Cohen's Kappa indicate that both experts concurred on the accuracy of the instrument's statement items. Additionally, the Rasch analysis reveals that the sincerity scale, based on KH Ahmad Dahlan's version, is reliable, accurate, and fits well with the model.