My son was 2 when I got him his first HO set. Here's what I did, and why I did it.
1 - I purchased a Thomas the Tank engine Bachman EZ Track System. This gives you a nice little oval of track, the track all snaps together for ease of use and doesn't require a platform, it works great on carpet, hardwood or even a table top.
2 - Then I purchased the figure 8 expander
This turns the oval into a nice over under figure 8. Now we're getting somewhere.
With this basic system, he's able to put the trains on the track, get them running and begin designing the layout like he wants. For more complex systems you can just start adding switches, moving crossing gates, and all kinds of additional track.
I'm a firm believer in sectional track for starting off. Flex track is great for gentle curves and limitless design, but needs to be cut properly to link together, and requires a level of skill to make sure curves are a consistent radius, don't kink and other operational issues.
The thomas engine and cars are all rugged, with only 2 wheels on most cars, they're very easy to put back on the track, and with the moving eyes are very very cool.
Once your son's around 5 or 6 then it's time to move up to the next level and build scenery and stuff, but at this point you want something that will get you both running trains quickly.