The tricky thing about communism is that social hierarchies are not going to abolish themselves. In order to have a social revolution, some active force must destroy the existing order. This force must be raised and guided by some conscious element. So, it should come as no surprise when some on the left claim this role for themselves. The tendency goes back at least as far as the Conspiracy of Equals in 1796.
But to rationalize their actions, something must set members of such a vanguard apart from everyone else. For the Bolsheviks, it was an esoteric way of knowing. What made Lenin and his comrades special was their “revolutionary consciousness.” The practical meaning of this peculiar notion was explained here by Leszek Kolakowski:
Marx alleges that the working class carries, simply because it is the working class, a kind of privileged knowledge, “revolutionary consciousness,” of the course of history […] But this cognitive privilege, while it may have existed as something much to be desired in the minds of Marx and Engels, has to this day failed to materialize in the minds of the workers. Lenin (and before him Kautsky) thought that this little practical difficulty could be overcome by adding a supplement to Marx’s theory: since the proletariat was incapable of spontaneously generating “revolutionary consciousness,” it had to be instilled from without. This was to be done by the “vanguard” of the proletariat, the Communist party; and the Party–now sole repository of the true purpose of history–is vested with the right, indeed the duty, to discard the immature, empirical consciousness of the masses and lead them, through revolution, to the classless society. And Lenin added–which is an important point–that what the workers could produce of themselves was a bourgeois consciousness, since in a capitalist society only two basic forms of consciousness could exist.
The implication of this theory is that the Party knows better what lies in the genuine interests of society, and what constitutes the will of society, than society itself, and once the spirit of the Party is incarnated in the will of one man, Marxism-Leninism comes to mean the dictatorship of one man over the proletariat. Thus Marx’s hypothesis that the working class has a privileged knowledge of the final purpose of history culminates in the assertion that Comrade Stalin is always right.
The real interests of society, the true course of history, and the inner workings of the system of oppression–these are known only to the vanguard. With this special knowledge comes incredible authority. Only its members can say what must be done to liberate humanity. They are, as Kolakowski said, “vested with the right, indeed the duty” to do whatever they deem necessary to this end. The rest of us cannot appeal their judgments, because we lack their superior insight. Our objections merely reveal our ignorance of oppression, or our complicity in it.
Of course, an egalitarian elite is a self-contradiction. But as Kolakowski warned elsewhere, contradictions are what we should expect from utopian projects. Hence the self-defeating absurdity of an elite to end all elites.