Is marrying someone for their money worse than marrying for their looks?

At one point when Fanny's criticized for marrying Job for his money, I think she points out that after all, HE only married HER because of her looks. It reminds me of something Marilyn Monroe's character says in How to Marry a Millionaire, about how a man being rich is like a girl being pretty.

Why is it that many people are really critical of a woman (or a man) (essentially, someone without money of their own) marrying someone because they're rich? I mean, Job doesn't really "love" Fanny...he barely knows her. Had she had a huge scar on her face or been plain, it's unlikely he'd have been head-over-heels in "love" with her, or comissioned her portrait.

The insulting thing about being "loved" for your looks is that people are admiring you and praising you for something YOU HAD NO CONTROL OVER. It's just the way a bunch of random cells came together. I wonder why people are so okay (or were, in the past) with others "loving" someone purely on the basis of their looks...but saw someone marrying another person on the basis of their income as very distasteful.