Children with autism spectrum disorder (ASD) display difficulties with response to joint attention in natural settings but often perform comparably to typically developing (TD) children in experimental studies of gaze following. Previous work comparing infants at elevated likelihood for ASD versus TD infants has manipulated aspects of the gaze cueing stimulus (e.g. eyes only versus head and eyes together), but the role the peripheral object being attended to is not known. In this study of infants at elevated likelihood of ASD (N = 97) and TD infants (N = 29), we manipulated whether or not a target object was present in the cued area. Performance was assessed at 10, 14, and 18 months, and diagnostic assessment was conducted at age 3 years. The results showed that although infants with later ASD followed gaze to the same extent as TD infants in all conditions, they displayed faster latencies back to the model's face when (and only when) a peripheral object was absent. These subtle atypicalities in the gaze behaviors directly after gaze following may implicate a different appreciation of the communicative situation in infants with later ASD, despite their ostensively typical gaze following ability.