Although faces of in-group members are generally thought to be processed holistically, there are mixed findings on whether holistic processing remains robust for faces of out-group members and what factors contribute to holistic processing of out-group faces. This study examined how implicit social bias, experience with out-group members, and ability to process in-group faces holistically might predict the magnitude of holistic processing for faces of two out-groups: other-race and other-age groups. In Experiment 1, Caucasian participants viewed Caucasian (own-race) and East Asian (other-race) faces. In Experiment 2, young adult participants viewed young adult (own-age) and baby (other-age) faces. Each participant completed a composite task with in-group and out-group faces, an implicit association test, and questionnaires about their experience with in-group and out-group members. We found that while the participants had relatively extensive experience with the other-race group, they had limited experience with the other-age group. Nonetheless, implicit social bias was found to positively predict the magnitude of holistic processing for both other-race and other-age faces. Exploratory analyses on the interactions among the predictors suggest that the effect of implicit social bias was primarily observed in participants with strong holistic processing ability of in-group faces but with low level of experience with members of the out-groups. These findings suggest that observers utilize different kinds of information when processing out-group faces, and that social features, such as race or age, are incorporated to influence how out-group faces are processed efficiently.