what about Lemonjelly then? remember that silly single from last year "Nice Weather for Ducks" well, their albums are excellent (imo) bizarre & melodic, and still experimental, without being crap. Even went to see them live. Great fun!

Prodigy generally managed it, Underworld, Leftfield, Orbital, well going down that route you could cite Bjork, who somehow managed to combine being totally weird with chart success. Groove Armada do bostin' trippy tunes that are not just charters but get used on adverts (Renault) and film s'tracks (Miss Congeniality) and oh well there's lots more; often people will not have a clue who the artist is but will remember the song and its associations in the wider media. Yet you wouldn't call any one of those examples mainstream. So it can be done.

but in general I agree, you can't make it BIG big without sacrificing a lot of artistic integrity and creativity*, and the music industry is so cynically professional at churning out charters that you could get a wet cabbage on a video, put a track to it, and market it as the next big thing and it would sell.

*except Radiohead, and a few other select acts.