The collar and robe seem to me to be a residual of the clergy class, meaning more than just the teacher, preacher, and wine and bread blesser and giver-outer. They had a place in society, and in the UK they still do, with the 24 bishops and 2 arch-bishops entitled to their seats in the House of Lords. If the nobles dressed and lived differently than the commoners, and royal family different still, the idea of class would also have dictated that clergy dress different from parishioners.
While in the USA there are no societal class distinctions between clergy adn laity, they may still exist within various churches and individuals. I went to a kindergarten at an Episcopal day school, and every week (on Wednesday, I think) we had chapel, where the rector (though I didn't know that term then) had on his flowing yellow robe and there were altar boys to light candles and kneel in prayers who also wore some kind of robes, per se. It was scary, in a way. To not regard that robed priest with awe made us feel like we did not regard God with awe. And he always spoke so solemnly and clearly, never joking or talking about the weekend picnic or fishing trip (it seems Baptist preachers always find ways to get such subjects in, and a few chuckles, at least). Anyway, there was something about such rigidity, as if the message we got was "don't even think it-- 'it' meaning the procedures or the respect given, et al.