Of course many children can be a massive distraction in a car... various mothers I have known agree that at many points their children have been more of a distraction than almost anything else!
All distractions when driving / riding are not good or best practice, but we can control 'when' we 'allow', a distraction. Making a small volume adjustment of a volume control, is distracting but we can choose 'when' it is totally safe, and for the briefest moment, thus keeping safety at the forefront and completing that action perfectly safe.
So .. can we therefor conclude that it is only when an action 'takes over' from an existing conscious action, that it enters the danger area of taking over from all other activities. It is when people turn off the driving / riding that the distraction occurs and it is then that it becomes a danger?
There are only a few reports/surveys showing distraction but some are based off simulators which I don't think can be taken too seriously as it is not a real world condition.