You see, this is a slow burner. Think about it, the lad has a few key points about him.
1) Friends with Arya, potential partner for her.
2) Most important, he is the only guy aboit who can work with Valeryian steel. He made needle for Arya if you can recall?
3) Remember Davos likes the lad and...