What do you mean they are putting him down too often during the day and too early?
Are you sure you've never woken him?