"Solving a problem with someone looking over your shoulder and forcing you to talk to explain what you are thinking" is actually how problems are generally solved in most fields. Software engineering is perhaps one of the lone exceptions, and I think the pair programming movement has a bone or two to pick with your premise.
There's a big difference between someone looking over your shoulder and forcing you to explain what you're thinking, and someone working with you to collaboratively solve problems. In the latter case, you can ask lots of questions and throw out lots of dumb half-baked ideas, and actually get a meaningful, useful response from your partner/pair in response. If you're whiteboard interviewing, good luck getting your interviewer to meaningfully engage with you, since they don't want to "give away the answer".
The best interviews I've had were legitimate pairing interviews, where the two of us were tasked with solving a problem in a real codebase that the interviewer hadn't solved before.
(The best best interviews made it an issue in an open source project, so it can be a not-canned interview without you writing potentially production-ready code for a for-profit company without being paid)
THIS right here is the dream interviewing scenario and in my opinion the singularly best way to test a candidates competence, aptitude and personality. Solving problems together exercises all of these key things in harmony.
There's also the fact that some people, like me, freeze up in these interview scenarios. I didn't get past my first interview a few weeks ago because I couldn't think straight knowing I was being judged going through this process. As soon as the interview was over and I was more relaxed, I thought of several good answers, like I normally do when employed.
People are very diverse, some are fast and chatty, others are quiet and slow. Usually I perceive the fast and chatty types a bit arrogant. Personality and cultural background is probably an influence. Best teams have a great mix. Often caring about the work is more important than having a genius. (Of course people should be competent).
Another aspect of the funhouse of one-way mirrors that is the modern conventional tech interview process:
Candidate is fast and chatty: "I don't know. He seems smart, but he was also talking a lot of fluff. Kind of pretentious and arrogant."
Candidate is slow and hesitant (and perhaps pauses awkwardly now and then): "I don't know. He seems smart, but he seems a bit underconfident. We need someone with more energy."
See, we're having trouble even testing for competency. I passed a pen and paper interview recently (did confidently well on coding, blew the SQL part because I had only used APIs and still thought I should put that on the resume) and still got the position. I'm still scared shitless that I'm going to flub day 1 and I've been cramming and exercising basic knowledge for the past two weeks. I'm worried that the test was just a screen and the real requirements are hidden.
On the other hand, everyone's favorite anecdote: "senior engineers" not passing basic tests like mine.
I still don't feel qualified for entry level (and other peoples' interview experiences don't help!) and that's because entry level and other competency levels are subjective right now. What knowledge should you know from memory? What knowledge are you safe delegating to Google to remember? However, my feelings about my skill could just be nerves.
Lines in the sand would really help the self taught crowd and could be used to bring outdated, lagging CS programs and their graduates, up to speed.
Different people are going to have a different answer to your question, but here's how I approach things:
You should commit things to memory the things your memory naturally keeps. If it doesn't it means you aren't using it often enough to bother remembering it. There's so much information I used to keep re-learning and forgetting because I never used it. I finally realized that it was a waste of time.
I guess the problem is when the job you're interviewing for will need vastly (or slightly) different things committed to memory than what your current job does.
I also like something between google and memory: cheatsheets. I have a dev notebook full of checklists and cheatsheets for various technologies I've used and tasks I've done.
As a self-taught developer, I have reached the same conclusion regarding memorization. I find your concept of 'cheatsheets' inspiring, and would love to know more about how you organize that kind of information.
I used to just use text files. Now I stick everything in Evernote. Text files are good enough though.
Organization is pretty simple. Just need a decently named file with the relevant information. E.g. I would have an sql.txt file containing sql database operations that I don't use often enough to remember, but use often enough to be annoyed by having to search the web on how to do them.
Another example might be a centos5.txt file, giving me a quick rundown of the various setup options I've wanted in the past, and the location of various configuration files.
For programming languages, there's many that I use maybe once every 3 or 4 months. So ruby.txt might contain a quick rundown of how to do basic things: how to define a function, a class, perform loops, instantiate an object, access command line arguments, etc. I find it much better than having to hunt down a tutorial which will also be more wordy than needed.
I also have some for more computer science related things, such as tables of graph and search algorithms, along with their tradeoffs.
Checklists are another good thing. I have checklists for processes I've messed up.
E.g. committing new code. If I don't use my checklist, I always seem forget to forget to update a sample file. Or add a file.
Another big checklist is for giving estimates. For example, one of our legacy products is translated into 5 languages. This checklist reminds me to think of translations, because they have been a huge bottleneck in the past.
I do coding interviews exclusively since I find whiteboard interviews even more artificial. I try to match the interview to a persons skill set and ask them to implement things they should know or know how to look up if their resume isn't a lie. I don't care if they talk or not while doing the interview and I take the role of a product designer who doesn't know how to code. If they get interview anxiety I give them the space they need. If they want a keyboard & mouse I would give it to them.
What else can you do that would be an accurate simulation of their job that only takes 1hr?
What else can you do that would be an accurate simulation of their job that only takes 1hr?
First 1/2 hour: Discuss their code samples in detail -- asking pertinent questions about each as to what they do, and why they did things they way they did. Drill down for detail, ask how they might have done things differently for different use cases, etc.
Second 1/2 hour: Bring out some of your own production code and do the same (allowing them space to ask questions this time). Works best if you let them see something that's a bit "raw", i.e. quick and dirty, which you know you could have done a lot better if you had more time or knew better about the requirements (or perhaps you're simply older and wiser now).
In both sessions, nuance (both in what they can observe, and how they chose to express themselves) is key. And it'll come out a lot more freely than in in a whiteboard interrogation precisely because they interaction will be natural, uncontrived and unforced.
Simple, non-confrontational and 100% reflective of what engineering work is like. That's what we do during the day, 99% of the time -- digging up (sometimes woefully) less-than-perfect components of production systems, and trying to make them a little bit better.
But as to how often we drag someone to a conference room, throw them at a smudgey whiteboard with creaky pens and no eraser, and force them to solve some abstract problems while we boredly look at our watch, and interrupt them with hints?
Basically never. Except, that is, in coding interviews.
I think that what you do makes sense if you take the time to read the resume, code samples, articles, and ask relevant questions. Asking to write some code is perfectly fine to see how the person works. References are also very important for more experienced candidates with a track record.
