Armed Forces was the album which first got me into Costello, but listening to it again it's just a little too polished really - spares too much effort in attempting to be contemporary, which was a bad move really. Oliver's Army is an absolute classic though.
His discography really is very diverse and interesting, it's been a whole barrel o' laughs reviewing it
I know what you mean about liking his bad songs though - depending on my mood sometimes I actually find myself listening to them.