We know resumes, references and "previous work" are all part of the hiring process. But like the Facebook guy's "competition" you see to get the right guys, as a non-programmer how do you create an organic test that you cannot "study for" and offer it to potentials to see how they do under pressure and also their actual competency? It is a big risk to start paying someone for 3 months only to realize they did not use best practice and used code that now needs to be rewritten for future purposes. Now you are $$$ and 3 months behind and investors blame you.
I am not averted to combining "standard" tests with this, but I am looking for the best way to create a CUSTOM test. Thanks!
Great question, when hiring developers (something we do very often) we follow this format once the developer has passed the interview stage:
We keep detailed notes in a spreadsheet regarding each developer, their progress in testing periods and even beyond that once they are hired. This is to make sure we are investing time and effort in growing our talent.
As a non-technical person I wouldn't risk it. You may end up with a working prototype but if the developer made the wrong choice of technologies, didn't annotate the code or provide good documentation then you could find yourself with something you'll have to scrap completely and have rebuilt from scratch.
what @Guy has said makes most sense. HackerRank is another similar website where you can ask candidate to perform.
Aslo, we have our Custom Tests/Training Tasks which we can share with you. Just drop me a PM (or write me at firstname.lastname@example.org) and will share it with you.
This is little away from what you have asked but i think it is important for you. Apart of HARD(Programming) SKILLS, i think more important things are ETHICAL VALUES and SOFT SKILLS. For example, when you go for a haircut, you ask the artist about what suits the best on me? Now even if artist has great skills, he should UNDERSTAND YOU first without any SELF INTEREST. Not only that, he also needs to COMMUNICATE IN SIMPLE TERMS (not technical terms) so that you know what to expect at the end. Here, "without self interest" is an Ethical value and "communicate in simple terms" is soft skill. This applies to every corner in service industry but often disregarded while hiring Programming team. It is not about the sharp sword they carry, it is about how and when they use it (can be destructive or productive equally).
Our experience working with hundreds of programmers says, when you trust a reasonably experienced team member and keep things simple, they don't disappoint you. After all, being human works.
Feel free to touch base for any further assistance.
If you look at the music industry how an orchestra hire a violin player ? If you don't know anything about violin playing then maybe finding someone who does may be helpful ?