If you have a youtube.com account and you go to your Email Options, you will see that the first checkbox under Send individual emails for the events below. is:
One of my videos receives a comment or a video response.
I would like to get an email for a video response, but not for comments on my videos.