A better solution would be to have both at the same time.
Browser says: x number of CAs say that this site is authentic (click here for a list). Do you trust this site? Certificate fingerprint: … Certificate randomart: …
And then there would be options to trust it once, trust it temporarily, trust it and save the cert. The first 2 could also block JS if wanted.
I can see this would annoy the mainstream users, so probably this should be opt-in, asked at browser installation or something like that.