Compared to Knight Armor, Elite Knight Armor offers higher defenses in all 4 physical categories, but offers less protection against magic and fire damage. So, you may prefer to use (regular) Knight Armor against enemies that do those types of damage. As your character's endurance increases (which also increases your carry weight), you can freely equip the Elite Knight Armor Set, and not suffer any sort of movement penalty.
Most of Eastern Armor's starting stats are FAR superior, but it requires Twinkling Titanite to upgrade instead of regular titanite. (It's also slightly heavier.)
Armor of the Glorious also has staggeringly better starting stats, but is heavier, and can't be upgraded at all. (By the time you upgrade Elite Knight Armor to +3, it will start to have better stats in most categories, and by the time you upgrade it to +8, it will be better in all categories... with the exception of curse resistance.)