See Ninth Art for a variety of frustrated musings on this very topic. The history is complicated, and has to do with the way comics started out as "the funny pages" in newspapers and thus from the start had an inferior, red-headed-stepchild position, seen as being good enough for people who couldn't handle the real news; then English-language comics got their own magazines, which were churned out by a bunch of teenagers, basically (not exclusively, but Joe Kubert, who is still drawing and writing, started off professionally at the age of 13, and he wasn't all that exceptional), printed on bad-quality paper, and generally treated as unimportant ephemera...
Then, in the '50s, just as things were beginning to look a bit better, Frederic Wertham came along, kicked up a stink about how these comics (which were intended for an adult audience) would corrupt children and turn them into juvenile delinquents. There was a Senate hearing, EC Comics went out of business, and The Comics Code Authority was brought into being as a self-regulation body so that comics publishers wouldn't have to submit to federal controls.
The Comics Code was the equivalent of requiring every movie ever made to have a G rating. Only worse. Naturally, adults didn't really read comics after that, because there was nothing there for them except maybe nostalgia.
The '60s saw a revival of the notion of comics for adults (Robert Crumb and the like), but these comics couldn't be distributed through normal channels because they weren't Code-compliant. So they were sold through head shops etc. Now, the direct market -- i.e. specialist comics shops as we know them today in the English-speaking world -- is essentially descended from that model. Head shops were/are places that didn't draw attention to themselves, that often didn't make a profit, and that were run by enthusiasts for enthusiasts. The same is the case with direct market comic shops.
This completely sucks, because you have to be in the know to want to go to one of these places -- they're not friendly to the casual browser at all -- and because they're largely run by superhero fans for superhero fans, superhero comics are, by and large, what you get there. It's extraordinarily difficult to get non-superhero comics to sell well through the direct market, because the retailers won't buy them. The publishers will, and so will the public if they know where they are (comics by comics-creators operating outside the direct market system do very well in bookshops, e.g. Raymond Briggs, Posy Simmonds, art spiegelman etc.); but the retailers aren't interested.
This sucks, and is why I want to start up a chain of really decent comics shops for the English-speaking world, so that we can catch up with the French. The creative talent is there: there's just a bottleneck in the retail sector. But that can be fixed. If somebody wants to fix it.
Katherine; that explanation doesn't let the UK, Ireland and the Antipodes off the hook.
French dictionaries have pictures. The monolingual, thick ones that kids get when they go to college. That threw me. What else ... I remember hanging around with a lyonnais grad student who was doing--evidently legitimate--background reading for a paper on the Rwandan genocide, and this background reading was a graphic novel. Can you imagine that here?