Well, the challenge involves using a python LLM framework to build a simple RAG system for recipes.
It's not a hidden requirement per se to use LLM assistance, but the candidate should have a good answer ready why they didn't use an LLM to solve the challenge.
Why is it a negative that the candidate can solve the challenge without using an LLM? I don’t really understand this.
Also, what is a good answer for not using one? Will you provide access to one during the course of the interview? Or I am just expected to be paying for one?
It's not negative that the candidate can solve it without an LLM, but it is positive if the candidate can use the LLM to speed up the solution. The code challenge is timeboxed.
We are providing an API key for LLM inference, as implementing the challenge requires this as well.
And I haven't heard a good answer yet for not using one, ideally the candidate knows how to mitigate the drawbacks of LLMs while benefiting from their utility regardless.
A good answer in this situation would focus on demonstrating that you made a conscious decision based on the problem requirements and the approach that best suited the task. Here’s an example of a thoughtful response:
"I considered various approaches for solving this problem. Initially, I thought about using an LLM, as it's great for natural language processing and generating text-based solutions. However, for this particular challenge, I felt that a more algorithmic or structured approach was more appropriate, given the problem's nature (e.g., the need for performance optimization, a specific coding pattern, or better control over the output). While LLMs are powerful tools, they may not always provide the precision and control required for highly specific, performance-critical tasks, so I chose to solve the problem through a more traditional method. That said, if the problem had been more open-ended or involved unstructured data like text generation, I would definitely consider leveraging an LLM."
This answer reflects the candidate's ability to critically assess the problem and use the right tools for the job, showing maturity and sound judgment.
Ah so you expect mind readers who can divine something from your brain that goes against 99.99% of interviewers' practices and would get them instantly disqualified from an overwhelming majority of interviews. Nice work good luck finding candidates.
Indeed, looks like it is just an unspoken rule and an interview trick after all. I would not want to interview with this person, much less work with them.
It's not a hidden requirement per se to use LLM assistance, but the candidate should have a good answer ready why they didn't use an LLM to solve the challenge.