The running user or context user for a batch class is the user who enqueued or scheduled the batch.
If a user goes into the Developer Console and executes
System.executeBatch(myBatch, 200);
that user is the running user and changes made by the batch will have their name in the audit fields.
If a user schedules a Scheduled Apex class that kicks off a batch, or makes some record update that fires a trigger than enqueues a batch, or in some other way precipitates the asynchronous execution of the batch, that user will be the context user and reflected by the audit fields.
You can't change or set this behavior at the system level; you must ensure that the batch job is enqueued as the user in whose context you'd like it to run. In many orgs, that will be a "System" or "Integration" or similar user.
This is true across all asynchronous functionality in Salesforce, the notable exception being Platform Events, whose triggers execute in the context of the Automated Process system user.