OS X doesn't have native support for MTP (Media Transfer Protocol), the protocol that the Nexus 5 and other modern Android devices use when connecting to a computer in order to transfer files. It is possible to transfer files to and from devices that use MTP by installing a tool called Android File Transfer on your Mac.

I was having difficulty getting Android File Transfer to detect that my Nexus 5 (running KitKat 4.4.4) was connected via USB, when I discovered that MTP mode has to be manually enabled on the phone in order for Android File Transfer to detect it. The setting is hidden in Settings -> Storage, and can be found by tapping the menu button and tapping "USB computer connection". After enabling MTP mode using the checkbox, Android File Transfer detected the phone a couple of seconds later and file transfer then worked as expected.

Thanks to the original blog post I found explaining the setting and where to find it.