The Casio VL-Tone VL-1 may just be the most affordable and tiny synth available. It will store one patch which is created by entering numeric values. Oh, and it is a calculator, too!
Enter a patch by going to Calculator mode. Choose the ADSR preset. Press the “MC” button. Enter the patch numbers from the site in the order they appear here. Press the “M+” button. Set the mode to Play.
Note: since you can’t enter leading zeros on a calculator, simply leave them off when entering the digits (00001234 becomes 1234).
SynthLib