Academy Records in the East Village and Greenpoint (great selection of used) Brooklyn Record Exchange (in Bushwick, large used selection) Captured Tracks (decent, near Greenpoint Academy Records) A1 (decent, a little rundown, in East Village) Stranded Records in the Village (mostly new)
Rough Trade Records. Brooklyn location was great with a small live room in the back. I attended the Ben Harper and the Innocent Criminals record releaser party there. Haven't been to new location yet.