It's getting much more complicated. Here's an hour long talk on how Unreal Engine 5's Nanite works.[1] This is a huge breakthrough in level of detail theory. Things that used to be O(N) are now O(1). With enough pre-computation, rendering cost does not go up with scene size. See this demo.[2] You can download the demo for Xbox X/S and PlayStation 5, and soon, in source form for PCs. Explore 16 square kilometers of photorealistic city in which you can see a close-up view of anything.
The format of assets in memory for Nanite becomes far more complicated. GPUs need a redesign again; UE5 has to do more of the rendering on the CPUs than they'd like. What they need to do is simple but not well suited to current GPU designs. It's such a huge win this approach will take over, despite that.
New game engines will need to have something comparable. Maybe using the same format. Epic doesn't seem to be making patent claims, and the code for all this is in the Unreal Engine 5 download.
(Basic concept of how this works: A mesh becomes a hierarchy of meshes, allowing zoom in and out of on level of detail. The approach to subdivision is seamless. You divide the mesh into cells of about 20 triangles. Then, each cell is cut by one or two new long edges across the cell. These become the boundaries of a simpler cell with about half as many triangles, but no new vertices. The lower-resolution cell has no more edges than the higher-resolution cell. There's a neat graph theory result on where to cut to make this work. This process can be repeated recursively. So, one big mesh can be rendered at different levels of resolution depending on how close the camera is to each part. The end result is that the number of rendered triangles is determined by the number of pixels on the screen, not the complexity of the model. Hence, O(1)).
Then, all that needs to stream in from the network. That's probably Unreal Engine 6. In UE5, the whole model has be downloaded to local SSD first. Doing it dynamically is going to require precise download scheduling, and edge servers doing partial mesh reduction. Fun problems.
Then the metaverse starts to look like Ready Player One.
The closest thing to the metaverse in the form of open worlds are flight simulators right now. They have an enormous amount of content to model the entire planet already. A lot of that content is generated rather than designed. MS obviously did a nice job with this and e.g. X-plane and Flightgear have pretty detailed worlds as well. Most of these worlds are generated from various data sources (satellite imagery, altitude data, open street maps, etc. MS does the same but adds machine learning to that mix to generate plausible looking buildings and objects from the massive amount of data they have access to.
Fusing that with what Epic is doing is basically going to enable even more detailed gaming worlds where a lot of the content is generated rather than designed. Flight simulator scenery is bottle necked on the enormous amount of triangles needed to model the entire world. Storing them is not the issue. But rendering them is. Removing that limitation will allow much higher level of details. X-plane is actually pretty good at generating e.g. roads, rails, and buildings from open street map data but the polygon counts are kept low to keep performance up. Check out e.g. simheaven.com for some examples of what x-plane can do with open data. Not as good as what MS has delivered but not that far off either. And they actually provide some open data as well, which simheaven recently integrated for x-plane.
Flightgear actually pioneered the notion of streaming scenery content on demand rather than pre-installing it manually. X-plane does not have this and MS has now shifted to doing both installable scenery and downloading higher resolution stuff on demand while you are flying.
Once you hit a certain size, pre-installing all the content is no longer feasible or even needed. Once these worlds become so large that exploring all of it would take a life time (or more), it's more optimal to only deliver those bits that you actually need, when you need them. And doing that in real time means that even caching that locally becomes optional. And even the rendering process itself does not have to be local. E.g. NVidia and a few others have been providing streaming games with cloud based rendering. That makes a lot more sense once you hit a few peta/exa bytes of scenery data.
Another interesting thing is generating photo realistic scenery from simple sketches with machine learning trained on massive amounts of images. This too is becoming a thing and is going to be much more efficient than manually crafting detailed models in excruciating amounts of detail. MS did this with flight simulator. Others will follow.
I think that we put too much credit on the graphic part for the metaverse. As long as people can immerse themselves in X, I'd say it's good enough. We have had metaverse since the 80s. On my side I enjoy classic games much more than modern ones too, maybe I'm just weird.
Nanite doesn't do any rendering on CPU and does a lot of work that is traditionally done on CPU (culling) on GPU. Nanite does use a software rasterizer for most triangles, but it runs on GPU.
Oh, I think you're right. The nested FOR loops of the "Micropoly software rasterizer" are running on the GPU.[1] I think. I though that was CPU code at first. They're iterating over one or two pixels, not a big chunk of screen. For larger triangles, they use the GPU fill hardware. The key idea is to have triangles be about pixel sized. If triangles are bigger than 1-2 pixels, and there's more detail available, use smaller triangles.
Much like any other field, it is wide and takes a lot of learning and study. But it is not unobtainable. I started in 2017 I'd say and I feel like I have a good enough grasp of the field of graphics, enough to read and implement papers, and watch modern SIGGRAPH talks.
If you're not imposing the requirement that someone create a AAA game, then No, I definitely know enough artist+devs who can create a high end engine from scratch as well as the art assets.
I can definitely drop into any part of the graphics pipeline from graphics programming, to art creation and AI etc... and have done so on multiple projects.
That said, doing it all by a single individual is a massive undertaking and incredibly unlikely.
That's why it comes down to capability vs execution. It's definitely possible for a single individual to be able to do each part of the process over multiple projects, but unlikely they could do it all on a single one by themselves.
> artist+devs who can create a high end engine from scratch as well as the art assets...drop into any part of the graphics pipeline from graphics programming, to art creation
Is there a job title for this role? I'm not stellar at art or programming but I'm 'really good' at both and everything in between.
I say "Catch All" because understandably there's a massive variety in types of technical artists (FX, shaders, rigging, generalists, tooling focused etc..) but that's the effective term.
My thought when reading this article was surprise at that perspective. It's not one I've seen before. A lot of comments here as well as in the article itself hints at the issue being that it's grown so much in the last decade that there's too much to get into for a beginner.
I've worked with graphics programming since before these changes, so I might simply have had the benefit of watching it unfold while already having a decent grasp of the state of the art at the time.
Is it more complex than any one person can handle?
If we're talking grasp of the theory, I don't think so. I feel I have a rough idea of what tech is out there and how things work throughout most layers of hardware and software. And I'm not the most knowledgeable. But I dedicate quite a few hours every week to reading up or brushing up on things.
If we're instead talking about a single person being responsible for the full graphics pipeline in a AAA game that certainly seems like too much. The projects I've been part of have ranged from teams of ten programmers using an in house engine to AAA productions based on fully featured engines maintained by a dedicated team. On both ends there were multiple programmers working on the graphics. I've also been the sole graphics programmer on a AA/large indie game using Unity. That's definitely possible and required me to understand and make modifications to most levels of the graphics pipeline.
Depends on how many is "many". Most graphics programmers on AAA games have 5-10 years of experience and all of them understand the whole pipeline. Games are realtime so you have the natural ceiling of how complex you can get before you run out of time and drop frames. In the VFX you can have people specializing in something like vegetation or fabrics with 20 years of experience just in that and nobody else in the team even approaching their level of expertise.
You can learn it all, just that one person isn't enough to do all the work required for a big project. It is like full stack web engineers, you can learn backend and frontend but a big project has enough work that people don't need to know both.
Maybe, but AAA as a positive value descriptor is a misnomer.
I've come to realize that AAA gamedev is a way to replace actual artistic taste with "output" that can then be scaled deterministically via the normal corporate software engineering levers we all know and hate.
You can make an absolutely beautiful game as an individual or small team. This game can be 2D or 3D or anywhere in between. You can pass on things that AAA games can't because you can have good taste & artistry and AAA doesn't even try to have those things.
It's just programming & digital art - people mythologize shit like this way too much. Can you replicate a AAA game on your own? No ofc not. But you can make a game as visually stimulating and enjoyable.
So overall..I just ignore any talk that acts like "AAA" is anything of value or meaning besides an org chart turning labor into $60 products.
