Refer this link: link
Found what was wrong with my coverage. There were no test classes written for a part of the project, so those classes never got called in any test class which lead to them being omitted from the list of classes covered and therefore them not being counted towards Overall coverage.
However, while uploading a package, after tests were run some classes didn't have any coverage at all which lead to Overall coverage being so low.
After writing a few more test classes(for the classes that were not tested), I got the coverage above 75%.
So if there are no test classes testing some class A, then this class A will neither show up in Overall Code Coverage class list, neither count towards it.