I agree with the elephant ride annoyance but, assuming they have been restricted to limit the danger that something weighing tons can do on impact, if they can be helped move along during relatively safe hours they would surly spend less time with the rest of us causing danger and frustration during the less safe times. The elephant rides would be reduced while I'm behind them during the day hopefully.
This may be a different topic: I don't know if you cycle Paul but if you de-restrict them, as a cyclist, you
do not want a HGV passing you at 70 mph.

(not on a motorway obviously) It's bad enough at 50 mph but if they're doing it a 3 a.m in the morining I'm not likely to be on the road and few others will either.
On a small motor bike, or car for that matter, the pressure wave from a HGV passing at 70 mph is considerable!