Short answer is yes, you will see all data in your org.
The longer answer is more complicated.
SeeAllData is meant as a mechanism for showing "live" data in a unit test. The better/best practice is to not specify this annotation and create your own mock data for unit tests. With your mock data, you can even set the ownership of the records and dictate what records are shared with / not shared with your code.
Second, by default unit tests run in the system context and can see any existing records (any live records if you use SeeAllData in addition to any mock records you create for testing). Sharing privileges really come into play when you use the "Test.runAs()" method and specify a particular user.
If you use the Test.runAs call, the context is changed to that user for anything that runs between the open and close brackets
//Runs as if I am executing code as Joe
//Anything after this does not run as Joe
As another best practice, if you are testing any Apex classes that use the "with sharing" modifier, it's a best practice to write your unit tests, using Test.runAs for users that both have access to the records and for users that do NOT have access to those records.
Edit: More food for thought: I found this related thread that states if you're attempting to test criteria based sharing rules, they are currently not supported in unit tests: Unit-Testing Sites Permission Involving a Sharing Rule with Unit-Test Created Data