There’s no reason that you wouldn’t be able to go directly from compression artefacts to a decent output with a correctly trained network. It’s slightly erroneous to call them artefacts as they do hold information, just in a not readily reversible way with conventional upscaling techniques.