After a bit of testing

At time of writing, half yes and half no

custom attributes cannot be added to the access token itself, only the Id token
in some scenarios, the id token (a JWT) can be included in the same response that provides the access token

The best we've got

You can (once Enable OAuth Settings is checked in the Connected App's "edit" page, Setup -> Apps -> App Manager, then find your Connected App, click the triangle, and select edit) check Configure ID Token, which will show some additional checkboxes, one of which is Include Custom Attributes

You'll need to add the Access unique user identifiers (openid) scope too

The custom attributes are managed through the "manage" page for the Connected App (Setup -> Apps -> Connected Apps -> Manage Connected Apps, and find your Connected App's name)

For the Authorization Code flow (a.k.a. the "Web-Server" flow), and likely others

If you don't specifically send data in the scope part of your request body, or if you include the openid scope, you'll get the id token (as a JWT) in the same response as the one that provides the access token.

Some simple processing and Base64Url¹ decoding later, you could get at the custom attributes.

For the Client Credentials flow (and possibly others where a specific user isn't used)

The response with the access token will include the id key, which contains a url like https://login.salesforce.com/id/00D.../005....

Making a second request using that url (and including the access token (obtained from the first request) in the Authorization header, of course) will get you the Id token, which does contain the custom_attributes

There is no way around needing to make 2 requests in this case

^{1: not to be confused with Base64, there are two non-url-safe characters that are replaced in Base64 to make it Base64Url. Salesforce doesn't provide tools specific to Base64Url}

After a bit of testing

At time of writing, half yes and half no

custom attributes cannot be added to the access token itself, only the Id token
in some scenarios, the id token (a JWT) can be included in the same response that provides the access token

The best we've got

You can (once Enable OAuth Settings is checked in the Connected App's "edit" page, Setup -> Apps -> App Manager, then find your Connected App, click the triangle, and select edit) check Configure ID Token, which will show some additional checkboxes, one of which is Include Custom Attributes

You'll need to add the Access unique user identifiers (openid) scope too

The custom attributes are managed through the "manage" page for the Connected App (Setup -> Apps -> Connected Apps -> Manage Connected Apps, and find your Connected App's name)

For the Client Credentials flow (and possibly others where a specific user isn't provided in the request)

The response with the access token will include the id key, which contains a url like https://login.salesforce.com/id/00D.../005....

Making a second request using that url (and including the access token (obtained from the first request) in the Authorization header, of course) will get you the Id token, which does contain the custom_attributes

There is no way around needing to make 2 requests in this case

For the Authorization Code flow (a.k.a. the "Web-Server" flow), and likely others

If you don't specifically send data in the scope part of your request body, or if you include the openid scope, you'll get the id token (as a JWT) in the same response as the one that provides the access token.

Some simple processing and Base64Url¹ decoding later, you could get at the custom attributes.

If you include scopes in your request but omit the openId scope, the you get the id key with the url as the value, and you'd need to make a second request just like in the "client credentials" case

^{1: not to be confused with Base64, there are two non-url-safe characters that are replaced in Base64 to make it Base64Url. Salesforce doesn't provide tools specific to Base64Url}