I did the math once, but I forgot the exact number; having a full set of runes and masteries vs. no runes and masteries is roughly equivalent to an expensive tier 3 item. So if you have no runes, and everyone else is loaded with tier 3s, they basically start the game with an extra 1k-2k gold on you. A player with good farming skills will net about 12k over the course of 40 minutes on most champions.
A curious quirk of a game with a large player base and skill-based matchmaking is that it really doesn't matter all that much, since you will end up with an appropriate challenge either way. So does it matter? For getting a big Elo number and winning tournaments, yes. For just enjoying the game, no; your rating will automatically adjust so you're performing at the same level as your opponents either way. Because of this, I generally advise you only get runes to change the proportion of your effectiveness (e.g. to slant things more into cooldown because you enjoy spamming, or slant things into flat AD/AP/pen runes because you like going for first blood) rather than worry about changing your absolute effectiveness total.
