Most sites that implement infinite scrolling effectively "unload" content that's far enough off the screen, much like they delay-load content that's below your scroll area. The net memory usage shouldn't be dramatically worse.