I am dividing this answer into three section so please go through them and try to understand importance of every section.
Testing is key to the success of your application, particularly if your application is to be deployed to customers. If you validate that your application works as expected, that there are no unexpected behaviors, your customers are going to trust you more.
Take a simple examples to clear why testing is important.
Requirement : Write a vf page/controller which save the record and code redirect to detail page of that record.
Expected Output : After Save it should redirect to detail page.
So if you have developed such requirement there are two ways of testing such application.
One is through the Salesforce user interface, important, but merely testing through the user interface will not catch all of the use cases for your application.
Second Write test methods to ensure functionality and verify using System.assert methods.
What to test:
Salesforce recommends that you write tests for the following:
Test to verify that a single record produces the correct, expected result.
Any Apex code, whether a trigger, a class or an extension, may be invoked for 1 to 200 records. You must test not only the single record case, but the bulk cases as well.
Test to verify that the expected behavior occurs through every expected permutation, that is, that the user filled out everything correctly and did not go past the limits.
There are likely limits to your applications, such as not being able to add a future date, not being able to specify a negative amount, and so on. You must test for the negative case and verify that the error messages are correctly produced as well as for the positive, within the limits cases.
Test whether a user with restricted access to the sObjects used in your code sees the expected behavior. That is, whether they can run the code or receive error messages.
When to write test cases:
In software development life cycle testing comes into the picture when we complete the development. It can be either fresh development or extended development. That is the right time to write test cases for your functionality and prior to delivering your functionality for your QA team (If you have) to test, reason being if you have tested your functionality with test methods prior to QA you are minimizing defects.
Code coverage: Salesforce enforce 75% code coverage limit but if you have written proper test cases (positiv/negative/bulk) there is no need to worry about the code coverage you will automatically cover and code will be ready to deploy without any defects.
Some reference to get started :
Getting started with Apex Test
Testing best practices