Elite means best. Well, it used to mean best.
In recent years, the words “elite” and “elitist” have come to mean something more like “people who think (incorrectly) that they are better than everyone else.”
People who are accused of being “elite” in this sense usually have a lot of something their accusers don’t, like money or knowledge. So: the Wall Street elite and the academic elite.
Elite isn’t quite a slur, but it’s usually meant to make you dislike the “elite” person, to make you think he or she is different, not to be trusted, out of touch with regular people like you.
The word doesn’t tell you anything about the allegedly elite person, but it does reveal a few things about the people who say it. For example: They are uncomfortable with the idea that people are all basically the same, even people who have a lot of something they don’t have. Or: They are just repeating a thing they heard someone else say.