As an opening disclaimer, let me just say that all my code: CSS2 or HTML4.01 passes validation, i.e., they are standards compliant. So, if the website displays differently on your browser compared to my vision, it is your browser that is broken.
The problem with Internet Explorer is that, as far as I know, it doesn't support Cascading Style Sheets (CSS). More specifically, it doesn't support the entire CSS2.1 standard (to my understanding it supports CSS1). (There have been rumours that Internet Explorer 7, which is to be bundled with Windows Vista (codename Longhorn) and released in 2007, will increase its support of CSS2.)
Cascading Style Sheets belong to an effort of separating the Content of a webpage from its Presentation. Before CSS was introduced, an auteur électronique writes the entire documment in HTML, often resorting to trickeries with TABLES and graphics to align the text and increase readability. This leads to two problems: 1) the source code for the webpages becomes hard to maintain as sometimes the presentation tricks becomes too deeply tied in with the content that, a change of a single word in the document can break its entire presentation; and 2) this leads to a lot of unnecessary overhead in internet traffic--if the formatting information is the same for multiple pages, why bother sending it over the internet once for every page? Why can't it be such that the formatting information is recycled and reused?
As its name suggests, the CSS provides Style to the document's contents. In fact, the webpages using the HTML + CSS couple would still be readable if only the HTML half were offered. It will not look pretty, but it will be understandable. In one extreme example, the appearance of CSS makes website more legible for people who can't see/read! Several of the programs that "reads aloud" the content of a webpage for people with disabilities work significantly better on HTML files stripped of the formatting (in particular, in the days of old when many webpages format using tables, sentences gets broken illogically and the reader leaped across paragraphs because the website designer failed to consider accessibility).
Let us take a look at an example, using my website http://www.math.princeton.edu/~wwong/blog/index.html.
With CSS, all the contents are there, just some are hidden until we need to show it (in this particular case). In general, much more can be done with CSS to increase the usability of a website.
Well, to be frank, I have nothing against browsers that does not support CSS--some of my personal favourites in the browser world include lynx and links2, both of which operate in a pure-text environment and thus ignores CSS-tags completely.
lynx displays the same webpage from above.
The problem with Internet Explorer is that it choose to support some CSS, and omits others. That usually isn't so bad, if done appropriately. But the team at Microsoft chooses to do this in the worst way possible: supporting part of the functionality of a tag while ignoring other parts. To put it in crude language: "Their effort is half-assed." In the end, the presentation rendered by their browser is often less readable than the pure-text versions, which very much defeats the purpose of CSS.