I know some people have a “villager harassment” routine that you can go through but as a long time fan of the series my philosophy has always been live and let live, basically.
Like you said, Animal Crossing is about building a community with what you have. It’s a lot of randomization and that, for me, is the charm of the game. You don’t get to choose the fruit or flowers that grow naturally in your town, you don’t get to choose your neighbours. If I don’t like a villager I wont talk to them often or write them letters like they’re a person in my neighbourhood who I’m sort of forced to be friendly with through proximity, but I never actively try and make someone leave. (Also I’m not here in New Horizons yet, but in the old games people will eventually leave to make space for new villagers, so you can usually wait these things out)
On the other hand, AC for a lot of people is about making something perfect and that includes who lives in your town. I don’t necessarily have a problem with this and the games have progressively made it easier for people to make their ideal town (officially codifying path and fence making, being able to report villagers you don’t like etc.) I don’t really see a problem with engaging in the game like this (outside of the harassment stuff people do which just seems kind of weird to me). It’s obviously become an intended style of play, and furthermore Hew Horizons does let you pick the first three villagers that move in through the Nook Miles Tickets. This game is really leaning into it
So, tl;dr. I wouldn’t feel too bad about booting a villager you don’t like. Nintendo is actively putting systems in the game to let people do this and I don’t really think it’s a problem to find a middle ground between the perfectionist / procedural play styles AC has defined for itself that you feel comfortable with.