This is a pretty old paper, but I hadn't seen the live demo. That's pretty cool.
A problem with this algorithm is that it's pretty slow. It would be hard to use it for real time scaling.
I think that it's pretty cool though for any precalculated conversion to vector format. It would also be pretty cool if there were software that allowed you to create a pixel art image with a live preview in this format and the ability to export into a vector format.
One of the benefits of pixel art IMO is that it's got a pretty distinct style and its simplicity and restrictions means it's easy to be aware of composition where if you are given an unrestricted canvas that freedom can make it hard to know where to start. If you are given a 8x8 pixel grid to indicate "happy" there's only so many ways to do it, a symbolic heart, a little smiley face, etc. Given a whole sheet of paper you'll have more difficulty.
With the high res displays, there's more to consider than just the upscaling of old programs, the fact is people (especially indie game devs) still want to work with pixel art, and the number of titles with the term "8-bit" in their name that have come out recently. Part of that's because it's in vogue now, but a big part of it is simply that a poor artist can make a passable 24x24 sprite.
But it's still a shame that on high res displays they end up looking like a blocky mess. If you had a program that let you author your sprites under the same conditions, but let you preview an upscaled version and left you with a vector image (or high res texture with mip maps) you can make use of the fidelity while maintaining the simplicity. I think that would be cool.
Is the video based on applying the algorithm to the entire screen, or is it applied to the sprites individually, then they are plotted to the screen?
Just wondering, because the samples in the paper are all lone objects against a clear background. If the same object is drawn in front of a detailed background and then the upscaling is applied, this would affect the patterning. It's a little tricky to tell from the video, since the graphics (especially the backgrounds) use large blocks of plain colours.
Worst case, you could imagine Mario walking in front of the background and his shape shifting madly with every step he took?
(Edit: From freeze-framing the video, I'm guessing that the algorithm is being applied to the whole screen. The scaling works great when Mario is in front of the pale background. There are small artifacts when he crosses the black outline of the green bushes. Very minor in this video, I wonder how it would affect more intricate backgrounds or those that aren't such contrasting colours to the foreground?)
> Worst case, you could imagine Mario walking in front of the background and his shape shifting madly with every step he took?
This brings up something interesting: In terms of media containing sprites and minimal layering (e.g. NES-like games), wouldn't it be computationally cheaper to perform the scaling on the sprites and textures independently of one another, instead of post-processing? I wonder if there is any emulator that does this. I'm thinking not, as the NES wouldn't be capable of layering such detail (large sprites + textures). Upsampling emulators.. Hmm...
It'd be kind of hard, as what makes up a "sprite" lives in two different places - one that stores all the tile data, and another that basically describes which tiles make up which sprites. The latter is likely to change every frame, and potentially the former, too, so you'd still be rescaling some stuff every frame. You'd also have to rescale some things that are affected by palette changes, which sometimes change every frame (popular way to animate water, for example). Would you end up saving time this way? I'm not really sure. It's definitely an interesting idea.
i dont understand, as long as you figure out how to get the sprites into the right formats this would be immensely computationally cheaper ("time-saving"?) than post-processing every frame
The thing is, there's no "right format" for sprites, as they're used in the game. See http://benfry.com/deconstructulator/ for an example of how sprites are handled by the NES for Super Mario Bros. Mario is split into 8x8 pieces that are swapped out as needed: how do you determine which pieces are "Mario's sprite" and which are "that coin's sprite"? Remember that some pieces will change while others won't.
yea i dont doubt it would be complex & to build an upsampling sprite rendering engine you'd have to understand all this stuff. With old games like this it would probably involve some manual work cuz you'd have to recompose images, smooth as a unit, then decompose somehow back to the original tiling. I mean.... I'm not gonna do it & wouldn't bother trying
the computational price will be much much cheaper tho. swap in upsampled sprite (36 tiles to every 1 in old format, let's say) rather than post-processing every single frame in real-time with a smoothing/upsampling algorithm.
it would also keep the pieces distinct from each other rather than having objects/characters/backgrounds morph in & out of one another
Am I crazy for preferring the one on the left (nearest-neighbour)? The other two really mess up certain details such as the point numbers, the stem of the fruits and pokey's eyes.
They also tend to cause a very subtle distortion of the outlines of foreground sprites as they move over the background.
I preferred hq4x for much the same reason. Removed the large blockiness while still preserving the pixelated feel (and staying truer to the shapes in the original).
No, you aren't. I prefer nearest-neighbor too, I think it preserves the "spirit" of the game. Generally speaking, I prefer it for any kind of "pixel art".
That said, those algorithms are impressive, and I can see why someone could prefer them over a simple nn.
This reminds me of staring at those tiny screenshots on the back of NES boxes with sweaty palms as a six year old.. lol.. Pixel art was not yet embraced as an art form, I guess. (well, not by the adults)
As interesting as these smoothing algorithms are, they can't add any information to the sprites that wasn't already there. Why make everything look smooth and plasticky when the original game was carefully hand-drawn, pixel by pixel?
I think there is a purpose for this in video, but I'm not sure it's an all or nothing game. Especially here. For starters, the gameplay looks smoother in nearest neighbor because your eyes defocus while playing. Hence, you need sharp pixel edges to dilettante and your eyes do the rest. In a still shot, that's not the case at all. You want focus and smoothness.
Well SuperEagle seems even slower for me, and I know for a fact that I can emulate with SuperEagle scaling on and it works fine (>30 FPS). So I don't think their algorithm is out of the question.
> If you had a program that let you author your sprites under the same conditions, but let you preview an upscaled version
Combine Atwood's law with a proposed law of mentioning ideas on HN being implemented shortly thereafter and there may be a HTML5+JS implementation popping up as a "Show HN" soon.
A problem with this algorithm is that it's pretty slow. It would be hard to use it for real time scaling.
I think that it's pretty cool though for any precalculated conversion to vector format. It would also be pretty cool if there were software that allowed you to create a pixel art image with a live preview in this format and the ability to export into a vector format.
One of the benefits of pixel art IMO is that it's got a pretty distinct style and its simplicity and restrictions means it's easy to be aware of composition where if you are given an unrestricted canvas that freedom can make it hard to know where to start. If you are given a 8x8 pixel grid to indicate "happy" there's only so many ways to do it, a symbolic heart, a little smiley face, etc. Given a whole sheet of paper you'll have more difficulty.
With the high res displays, there's more to consider than just the upscaling of old programs, the fact is people (especially indie game devs) still want to work with pixel art, and the number of titles with the term "8-bit" in their name that have come out recently. Part of that's because it's in vogue now, but a big part of it is simply that a poor artist can make a passable 24x24 sprite.
But it's still a shame that on high res displays they end up looking like a blocky mess. If you had a program that let you author your sprites under the same conditions, but let you preview an upscaled version and left you with a vector image (or high res texture with mip maps) you can make use of the fidelity while maintaining the simplicity. I think that would be cool.