That wouldn’t make sense either. They said it was the afternoon, and that they now had the whole day free, the sentence wouldn’t make sense if you swapped “went to sleep” for “are still asleep”. There must be some weird timezone thing that the author expects us to know about them already.