I have two fields checkbox and a textbox field, if the user clicks on the checkbox checked than I want to enable the textbox field otherwise remain disabled. I'm not sure if this is possible using validation rule and I'm working on a salesforce standard page.
It sounds like your requirement can be stated:
Checkbox__cis checked, require
TextField__cbe populated. If
Checkbox__cis unchecked, require
You can validate as follows:
Checkbox__c == ISBLANK(TextField__c)
You will still need some user training on the layout itself (it won't enable/disable the input element), but the errors should be self evident to them.
You can't enable/disable textbox fields in a salesforce standard page while entering data. You can do this for a picklist fields using field dependency feature.
One workaround i can suggest if you want to achieve this after the page is saved, you can do that using RecordTypes and Pagelayout.
Create a record type called ‘Lock’, then a Page Layout with the textbox field as Read-Only. Then build a workflow to change the record type (and assigned page layout) when the checkbox is ticked.
Another workaround is as suggested by @Adrian, to use validation rule to throw error is the checkbox is not checked and the text field have some value.
Hope it helps.