Putting them in prison, probably not. But there’d normally be some level of intervention at some point, by some level of authority regarding their perceived fitness of parenting.
Apart from their own guilt and suffering, they seem to have glided through any legal consequences that accompany capitalised instances of neglect. Unless, of course, I am mistaken.
I understand people assume the extremes, but I’ve seen first hand very real consequences for parents in instances of neglect that many would perceive had a less profound consequence for the parties involved.