I need workbench connected app to be configured to use only for queries and disable the option to edit single/multiple records
Workbench authenticates to Salesforce as a user and has the same rights and privileges that are permissioned to that user. If you wish to restrict Workbench operation, you'll need to limit the permissions offered to users who connect through Workbench by, for example, placing them on a Read-Only profile. Running queries and modifying records both fall under the REST API and aren't permissioned separately as such for connected apps.
Alternately, since Workbench is open source, you could fork it and run your own copy for your organization with certain capabilities disabled. You'd need a PHP developer to do so.