For one thing if your headphones are wired backwards (the left sound comes out of the right earpiece) that is a good indication that they are fakes.
If you are listening to music then only musicians care if it sounds like the drummer is playing a left-handed kit. But, films will use sound to help place the actor or action on the screen, or indicate events off-screen.
For some games it's essential. You want to know which side the enemy is coming from.