I think you have a pretty good handle on it. The Pentium 4 is not actually a 686. The Pentium 2,3,and celeron processors are 686. The P4 is backwards compatible with 686 though.
Essentially what makes the compatibility levels are added CPU instructions. That's why they are backwards compatible. The 686 knows all the instructions of the 586, 486, 386. The 486 doesn't know the new instructions for 586, so if they are used, the cpu barfs.
x86, in the broadest sense, means a cpu which is compatible with the instruction sets. Athlon's are x86. In a sense, so are the crusoe chips, because they have code morphing, even though their architectures are radically diferent.