I think I get the Walt thing now. I should have figured this out before, but I was blinded by hope that he’d come back. Jacob probably had him as a Candidate, maybe even his prime Candidate. There was no doubt he was Special. The Others saw him as a good prospect so they wanted to make him one of them. That didn’t work so they let him go. That doesn’t mean Jacob let him go, he might have still been a Candidate. But then when he went back to the real world he found a real life. Sure he was haunted by the past, but we can’t pretend that he was Special anymore. Like Kate he found a reason to live off the Island. So Jacob let him go. To force him to come back would have been against his policy.
(In case you hadn’t noticed, I refuse to accept the reason for Walt’s absence was his age and growth. They could have easily got around that.