$400 and a few days work is fair payment for a priceless education about writing things down. I firmly agree with the small claims court response, but if he don't respond, it will only work if you have proof. At the end of the day it's not about moral or not, or right or wrong, it's all about what you can prove in court. You are paying a low price for the education of writing things down and contracts.
The first time you take on a large job, with a substantial payroll, I'm sure you will remember this small speed bump. I believe you should thank your God for the lesson. Good luck with your business.