We can Integrated Canvas app with a java application deployed on a third party server but why does Canvas App creation wizard asks for Heroku authentication (Username/Password,API Key,OAuth) even though we have not deployed the third party application in Heroku?
Heroku is only used to deploy the Canvas Quickstart app, if you want to quickly see the features of Canvas. If you do not need the Quickstart app, you do not have to use it.
Just go to the Apps menu in Setup or the Connected Apps menu and configure your app for the location you want. Your app could be on Azure or your own data center, for example.