Database.Stateful serializes the object being batched, not the entire memory graph of the transaction that spawns the batch process. I wouldn't know exactly "why" this is, but I'd guess that the underlying implementation uses some sort of binary serialization technique that stores just the object being executed upon, and not the entire memory graph of the execution context.
This makes sense from a performance and space-saving perspective, even if it might happen to be inconvenient at times. A carefully observant developer would notice that the object is serialized at the point where System.executeBatch is called, not at the end of the transaction. For example:
SomethingBatchable b = new SomethingBatchable();
b.someValue = 50;
System.executeBatch(b);
b.someValue = 100;
when start
is called on SomethingBatchable
, someValue
will be 50, not 100, even though we set it later. This is because b
was serialized and placed into the queue as-is. In fact, you can execute the same object more than once, with different values, and they will all behave as uniquely instanced values later (assuming, of course, you're in a context that allows more than one batchable call, such as Visualforce context).
I'd further suspect that the entire state of the object is stored in a single BLOB (binary large object) field, and couldn't support static variables correctly anyways without some significant effort, or using considerable database space. Reconstructing the entire memory graph of the object might even be prohibitively expensive time-wise, even if it were possible to do so under the current implementation.
However, at least there happens to be one upshot: you can store non-serializeable data in static variables, and they will be transient, avoiding otherwise fatal errors, such as trying to serialize SavePoint objects (which would cause invalid states if they were allowed to be serialized).
Finally, since we're not able to serialize the entire memory graph, it makes sense that the static variables of the class you're in are treated equally compared to the static variables of a completely unrelated class that might have been loaded in memory at the time. Defining the limits of the serialization to just the contents of a single object greatly simplifies the behavior and offers consistency.