The difference is exponential. 1 quaternary digit can encode twice as much information as 1 bit, yes. But 2 digits can store 16 possible combinations, whereas 2 bits can store 4.
X digits of binary vs quaternary can represent 2^x vs 4^x possibilities.
Even at 10 digits, that is 2^10 = 1024 vs 4^10 = 1048576.
X digits of binary vs quaternary can represent 2^x vs 4^x possibilities.
Even at 10 digits, that is 2^10 = 1024 vs 4^10 = 1048576.