Is it possible to add email response as new case comment? When an internal user comment on the case the customer receives an email. I want that When the customer replies to that case comment email, a new case comment record on the same case gets created.
You want to use the Email to Case feature. Salesforce has good instructions on setting up Email to Case, so I'm not going to repeat them here.
The basic gist of it is that your agents will be sending emails from firstname.lastname@example.org which should be configured to point to your specific email to case email address in Salesforce. If the email contains a refid in the subject or body, Salesforce will attach it to the case. If not, a new case is created.