[size=150]
I enchanted 2 items to epic rarity and then removed all enchantments. Later I had a full inventory and sorted all items (or switched pages) the item rarity of the icon (see picture) would be any rarity above normal, but not legend.
It is determined by the color of the item you clicked before clicking it. Clicking a legend then returning to the item with no affixes will indeed result in it appearing as a legend.