The best solution for these cases of mild inconvenient noise is to mask it with more noise, ideally a fan or a pink/brown noise generator, or even sounds of rain or a waterfall.

You want the noise to be indistinct enough that your mind won't focus on it. So avoid a bad fan that has a whiny pitch, that will drive you crazy.