Yes, they did. From what I gather, people in hunger-gatherer societies worked half as long as people worked after agriculture came along. Moreover, their work was not nearly as dangerous, or physically taxing. For those reasons, earlier societies had more time to spend with their children, and each other.

The creation of 'rich men' was a very costly experiment that has, in my opinion, been a miserable failure - and it is time to end it.