Ah yes, an unfortunate WinModem user. I pity you, truly.
I recommend you ditch your current modem entirely and buy a new internal one. Look at US Robotics, 3Com, whatever, and find one that specifically has a hardware controller. It'll be more expensive than a WinModem, but real modems don't need drivers *at all* in Linux. \o/
If you really need to save the money, you could find a different WinModem that has a Linux driver. But, that's probably more trouble than it's worth.
As for the video...what card did you select for XFree86 config? Redhat 9 (if I remember correctly) was out before an X driver for the Extreme existed. What driver is working?
You can go and add an entry for 32-bit video if you want.
I dug up an old (read: XFree 3.3) config file for what you're looking to do. It would look something like this:
Notice the last section for "Depth 32" - that's what you don't seem to have. Add that, and all the resolutions you would like to use. Then, use redhat-config-xfree (or System Settings > Display, I think), to switch to 32-bit.
Hope this helps.