Is my CD collection too large? Never!
Today I was driving home and heard the excellent "Girlfriend" by Matthew Sweet over my XM radio receiver. Then I was wondering to myself: do I have that CD? Yes, yes I do.
If I remember correctly, Nick Hornby wrote in "High Fidelity" that you're not really a true music fan until you have 500 albums. I would also submit that once you start forgetting CDs you own because of your vast collection, that'll work as well.
(That reminds me: I need to pick up the new Belle & Sebastian and Jason Mraz.)