I have a free development account and created a new community in it, When I click on the link in Customize -> All Communities it will open the community page with me logged in. But if I try to login through the login page it says incorrect username or password
If you have created the community and if you are the admin user, then you will have to login as an employee.
If you are talking about a user then, you should have all the permissions and roles set so that you have the access to login via the username and password box over there.
Also make sure that the users have been added as members to the community -
You can go to Community setup>> Manage community>> Administration>> Members ---- Then add the profiles of the users or the permission sets you want to give access to the community to.
To expand on what @Divyaansha S said,
Also you need to tick the "Allow internal users to log in directly to the community" then only can you login directly from the community page.
All Communities > Workspaces > Administration > Login & Registration > Allow internal users to log in directly to the community
If you don't think "Allow internal users to log in directly to the community", then you have to click "Log in" as in below.