I figured that the CreatedDate will not show the time in the reports section.
So I made a DateTime formula field to take the value from CreatedDate. Now this shows the time value inside the reports.
However now it seems like I cannot filter the records just using the time. Is there a different way to do that inside the reports? I want to be able to pull all reports for a certain date for a particular hour.