I never said I don't think sensor fusion is possible. My only argument has always been "adding more sensors does not mean you will have a better self-driving car". Richer data does not mean better data. It does not mean more actionable data either. All it means is that you have more data points. I absolutely agree that this can be a good thing, and probably is in the case of self-driving.
But maybe it's not. Maybe the additional flops required for sensor fusion would be more useful if given over to the planning part of the problem. After all, you don't have infinite cycles. This is what I mean when I say it has not been proven at scale. The edge cases matter (arguably they are all that really matter), and "solutions with more sensors work better at object detection in artificial benchmarks" is not nearly as convincing to me as it seems to be to you. Again, object detection does not exist in a vacuum, and a solution that generally behaves better in a very constrained environment might actually behave worse when you throw all the edge cases at it and add on prediction and planning.
> Richer data does not mean better data. It does not mean more actionable data either.
How actionable you make it depends on how good your engineers are. Diverse input data is definitely more actionable than just RGB pixels from images.
> But maybe it's not. Maybe the additional flops required for sensor fusion would be more useful if given over to the planning part of the problem. After all, you don't have infinite cycles.
On the contrary, it reduces your compute requirements. For example, adding lidar means you are getting direct distance measurements instead of wasting compute cycles predicting distance using voxels. It's a much more effective use of compute.
You are getting direct distance measurements for some things, but not for everything, so you still need a system that is predicting distance using voxels.
LIDAR doesn't work great in the rain/snow, so you still need a system that is predicting distance using voxels.
Doesn't really sound like less compute to me. I feel like I'm repeating myself but again this is what I mean by scale matters. A self-driving system that performs better in perfect conditions is not even a good system if it doesn't work in the rain.
Which is the point the detractors of "fuse all the things" have been making. Don't get me wrong, I think probably those same detractors (e.g. Elon Musk) have also not been entirely intellectually honest with their arguments and have ulterior motives like cost savings and saving face, but that doesn't make the arguments wrong.
But maybe it's not. Maybe the additional flops required for sensor fusion would be more useful if given over to the planning part of the problem. After all, you don't have infinite cycles. This is what I mean when I say it has not been proven at scale. The edge cases matter (arguably they are all that really matter), and "solutions with more sensors work better at object detection in artificial benchmarks" is not nearly as convincing to me as it seems to be to you. Again, object detection does not exist in a vacuum, and a solution that generally behaves better in a very constrained environment might actually behave worse when you throw all the edge cases at it and add on prediction and planning.