I'm wondering why they have decided to airgap the models inside docker containers. IMO, this would have been a better comparison if the models were allowed to perform tool calls.

A small suggestion that immediately came to my mind, why not try making it a JSON serialized data and base64 encode it just like JWT. So that it can be shared and loaded effectively.

Just thinking out loud here ;)

This is what I would do too but one problem I think there is a max HTTP URL length

it's really wild west out there with AI(LLMs) trying to augment every job in the tech :|

I have used serveo.net in the past for the same use case, this looks cool !