Van Dyke Parks - Jump!
New Pornographers - "Slow Descent into Alcoholism"
El Boxeo - "My Job at the Party is..."
The Kingsbury Manx - s/t
Momus - "Tape Recorder Man"
Harry Nilsson - The Point!
Big Black - "Kitty Empire"
Mike Severance's Fall 2001 CD-R
Raymond Scott & Jim Henson - "Journey into the Brain"
The Housemartins - London 0 Hull 4
Also, I just noticed that two of those albums' titles include and exclamation point.