Yeah, I've been thinking a bit about that kind of problem too.
My hunch is you could do it with a much more complex setup involving OpenAI functions - by trying different things (like "list just input elements with their names and associated labels") in a loop with the LLM where it gets to keep asking follow-up questions of the DOM until it finds the right combination.
My hunch is you could do it with a much more complex setup involving OpenAI functions - by trying different things (like "list just input elements with their names and associated labels") in a loop with the LLM where it gets to keep asking follow-up questions of the DOM until it finds the right combination.