The rich little boy picks up a starving beggar girl from the street when they were both small, and she became his playmate/maid. I also seem to remember he had a quite unhealthy obsession with panties...

Are you talking about Bocchama wa Ijiwaru?

Precisely, thank you. I couldn't remember the title.