When I make a pull off like that, I mute the e string with my left hand index finger. I do it without thinking about it, and it will become something you do without thinking about it if you’ve done it enough times.
Kind regards /Emil