Disk Union for records. Heaps of stores round Tokyo especially Shinjuku. I dig the 2nd hand store there, spent way too much money last trip
Get a map from the store with all the locations.
Nat Records http://natrecords.shop-pro.jp/
Nice shop in Shinjuku, cool punk selection.
BASE records in Koenji was another store I went to, highly recommended.
I haven't been back to Osaka for a while but you gotta hit up Time Bomb Records, massive collection, a total punk rock wet dream. It's close to a lot of other stores.