VISA and Mastercard are household names, known and trusted by all. Those little rectangular bits of plastic with the logo's somewhere have an implicit trust in most peoples minds such that rarely does anyone think twice about using them.
How many years did it take them and the banks to get folk here to adopt contactless payments (even then it was mostly done by stealth).
Who the heck is Apple? They're a highly priced consumer goods company with no real provenance in the financial sector. Who the heck knows how secure their systems will be and what they'll do when it fails (because its when ... not if ... visa et al lose billions every year to fraud/chargebacks etc)
Just chat with the folk from Paypal and they'll tell you its one of their biggest hurdles in market penetration and they've been around a lot longer.
I'm not trying to say Apple are 'untrustworthy' they're just an unknown.