I am curious if something like this is an opportunity for a whole new type of controls and feedback. Since the operator doesn’t have to be in the excavator physically they could take on any position: standing, sitting, lying down, etc. Instead of sending haptic feedback to the joystick it could be sent to a vibrating wrist band. You could hook up the equivalent of a Nintendo Power Glove to have the scoop operated by the operator simulating scooping action. Turning the excavator can be controlled by the operator turning their head and moving it around can be done by the operator walking on an infinite treadmill. Motor strain can be done via color of light or temperature rather than sound. You could have a VR helmet that can also show you a birds eye view from a companion drone, overlay blueprints, show power and water lines, measure depth, etc. I don’t know if it is possible but maybe you could even measure soil composition somehow to show large rocks, soft soil that is dangerous to drive over, inclination angles where the excavator is about to drive, etc.
I imagine skilled operators prefer familiar controls but perhaps there are interesting improvements unlocked by exploring alternatives. It might also fundamentally change how accessible it is for non-professionals to use these machines. I rented an excavator from Home Depot a few years ago to dig a foundation and the learning curve was not shallow. I wonder if a more “natural” interface would help keep people safer.
These are really really interesting thoughts around the remote teleop interface. A few things you mentioned have been on my mind. Teleop UX is underdeveloped and its affects is underestimated, and it could turn out to be a huge thing for us and humanoid companies if autonomy is harder than it seems now.
I don't believe construction equipment operation today is optimal. We just go with it as that's what we have. It's far from intuitive and easy for a newbie to start operating to give some context and requires hours of practice. There would be a bit of training to be done for remote teleop as well. Might as well make it easy to interface and improve on the existing experience, which hasn't really evolved for years and decades!
I'd like to have a chat with you if you're up for it: contact at useflywheel dot ai
I am curious if something like this is an opportunity for a whole new type of controls and feedback. Since the operator doesn’t have to be in the excavator physically they could take on any position: standing, sitting, lying down, etc. Instead of sending haptic feedback to the joystick it could be sent to a vibrating wrist band. You could hook up the equivalent of a Nintendo Power Glove to have the scoop operated by the operator simulating scooping action. Turning the excavator can be controlled by the operator turning their head and moving it around can be done by the operator walking on an infinite treadmill. Motor strain can be done via color of light or temperature rather than sound. You could have a VR helmet that can also show you a birds eye view from a companion drone, overlay blueprints, show power and water lines, measure depth, etc. I don’t know if it is possible but maybe you could even measure soil composition somehow to show large rocks, soft soil that is dangerous to drive over, inclination angles where the excavator is about to drive, etc.
I imagine skilled operators prefer familiar controls but perhaps there are interesting improvements unlocked by exploring alternatives. It might also fundamentally change how accessible it is for non-professionals to use these machines. I rented an excavator from Home Depot a few years ago to dig a foundation and the learning curve was not shallow. I wonder if a more “natural” interface would help keep people safer.