We shouldn't use screen size as our only measure though. The iPhone 6 is a 4.7", but is quite a bit bigger than the 4.7" Moto X 2013, and about equal in size to the 5" Nexus 5 [1].
Apple uses a bit more bezel on their phones, so in physical dimensions they are closer to some of their Android competition than you might think based on screen size.
Using screen-size as a measure of physical size let Apple pull off a pretty epic PR coup. The iPhone 6 is essentially the same size as modern 5" Android phones, but has a significantly smaller screen.
I can echo what he says. The iPhone has absolutely enormous bezels, so much so that the iPhone 6 Plus is definitely larger than my Galaxy Note 3 despite its having a 5.7" screen. That was the first thing I noticed when I held them side by side at a store a few weeks ago.
People tend to hold their phones in "portrait" mode, rather than "landscape". So, width would typically refer to the second largest dimension when talking about (most) phones. Additionally, given that phones are 3d objects, interacted with in physical space, the "feeling" of width would also include the phone's thickness, to some extent.
In this context, the iPhone 6 is actually smaller than the Nexus 5, by about 20%.
Apple uses a bit more bezel on their phones, so in physical dimensions they are closer to some of their Android competition than you might think based on screen size.
[1]: http://www.phonearena.com/phones/size/Motorola-Moto-X,Apple-...