Mmm....yes, you could say that.

[He watches a petal fall into his drink, floating on the glass.

For a moment, he looks...secretive. Like he knows something that Sherlock doesn't...before he shrugs and takes another drink.]

Indeed - I had all the chance in the world. Those countless phantasmal spirits were only a threat because I allowed them to become one. Of course, it was partly because of my fondness for our Master, but...

[He's silent for a moment.]

If I had fully absorbed you, I feared even I would lose control of who I was.