Is it better in general to take over anothers city or start a fresh one from scratch?

From scratch you don't make others mad (aside from barbarian).

Taking over city (Buildings built) but people likely dislike you (aside from barbarian).

I have taken over a barbarian's city. I have started cities. I am looking at a guy who has 4-5 cities and 25,000 prestige (if even that).

I don't have the resources to take over a lv. 10 barbarian nor defend it so don't suggest it would be the best to take.