Free time in a working memory task often improves the recall performances of the to-be-remembered items. It is still debated whether the free-time effect in working memory is purely proactive, purely retroactive, or both proactive and retroactive. In the present study, we used the single-gap paradigm to explore this question. In Experiment 1, we measured the gap-length effect (i.e., the difference in memory performance elicited by the gap-length difference) under three long-short-gap combinations (i.e., 2,500 ms/100 ms, 2,500 ms/500 ms, 2,500 ms/1,000 ms). Proactive effects have been observed in all the three combinations whereas retroactive effects have only been found in two of them (i.e., 2,500 ms/100 ms, 2,500 ms/500 ms). To rule out the possibility that the retroactive effects found in Experiment 1 were simply due to the temporal grouping caused by the gap, in Experiment 2, the 2,500 ms/500 ms combination was retested, with the memory materials being changed from letters (the material used in Experiment 1) to words. The results showed that the range of the retroactive effect (i.e., the number of affected memory items prior to the gap) increased when the memory material changed from letters to words, which cannot be explained by temporal grouping. Taken together, the two experiments provided solid evidence that free time in working memory could produce both retroactive and proactive effects that cannot be explained by temporal grouping. These findings also provide insight into the underlying mechanism of working memory, for example, whether rehearsal would occur during the free time.