The bottom line is, D&D is a lot of fun. But it's confusing at first. You have to do some homework, familiarizing yourself with some complicated rules. But soon it all starts to flow and you get the idea. Don't listen to all the other whiners about how the 4th edition rules ruined everything about the game, etc. Rules just provide a rough framework. The role playing/imagination element is up to the people playing the game. This isn't chess. You CAN just make it up as you go. The starter set gets you going and has what you need to make several cool characters and take them through a pre made adventure with battles and challenges up to level two.
However, the D&D product line is very frustrating and confusing for the beginner, like me. It's full of redundant products, so I couldn't figure out what I really needed to buy beyond the Starter Set. I've wasted a lot of money. So here is what I wish someone had told me.
You DON'T need to buy the "Core Rulebooks" ($66). They can be easily substituted with other things. For instance, if you buy the Starter Set, you'll make characters using powers and abilities that don't even come from the Player's Handbook (PHB), but come from "Heroes of the Fallen Lands", which is a "supplement," but fills the same role as the PHB. If you buy Heroes of the Fallen Lands instead of the PHB, you'll have info about making characters that's more compatible with your Starter Set.
What about the Dungeon Master's Guide(DMG)? Buy the Dungeon Master's Kit instead. It comes with an equivalent book, plus a bunch of tokens for characters and monsters, maps and a cool pre written adventure to take characters from level 2 to level 4. Oh, and a dungeon master's screen, too.
And the Monster Manual? The "Monster Vault" has an equivalent book (a few less monsters, but still plenty to work with) plus more monster tokens, maps and another adventure ready made to send 4th level adventurers up to level 5.
To summarize, if you get the "Core Rulebooks" you'll spend $66 and have three cool books, but no stuff with which to actually play the game (maps, dice, something to represent characters, monsters, etc. and you'll have to create your adventures from scratch, which can be fun but difficult. OR, you can spend $73 for the Starter Set, Monster Vault, DM Kit, and Heroes of the Fallen Lands and have the equivalent of the core books, PLUS all the other cool stuff to play with and get characters up to level 5 and beyond. By then, you'll have all the experience you need to create your own adventures a lot more easily. You won't need to buy minis, graph paper or battle mats or anything, except maybe a few more sets of dice for each player.
You'll probably end up buying the core books sometime anyway, just because they're cool. But do it only because you want to, not because you think you need them to start and enjoy a long career playing D&D.