I was playing with OpenFrameworks and Kinect years ago, and you could pretty much do this, so I am curious why you would TF.js instead of simple OpenCV or other libraries that don't need to to use ML or DL. Or am I mistaken, and it is using simple bits of TS.js?
We had a large, flip-disc (or dot) wall at the site in Macau in 2012 that we purchased from a company that was a wall of black and white discs or dots that would flip to create a cool effect by tracking people in front of the wall with Kinect units in real time. It also made a cool clicky sound like old train/airport physical arrival/departure boards did.
I had a feature on my HTC phone or early Skype over 10 years ago where a cartoon cat mimicked my mouth, eyes, and head movement live on camera, which I can't find the reference to, but I have a screen recording of it when talking with my kids in the US from Macau.
I also remember using animata, a software that animated 2D puppet-like cut-outs to the music, I played with over 7 years or so ago, that was really cool [1 YouTube].
Re the feature in HTC or Skype, it was Skype, there were a set of novelty faces that mimicked the movement (I can only remember the dog, but there were a few, some more effective than others).
