Connected App - avoiding a limit on a number of issued tokens + token expiration

Question

We have configured our web application to use OAuth2 with our SFDC Connected App. The connected app is configured to never expire the refresh token unless manually revoked. When an admin connects the Connected App to our web application it stores the refresh token received so that we can communicate with SFDC's APIs on behalf of that user later one. "Offline_access" and "refresh_token" are properly set on scope for that admin login page.

The application will work throughout the day just fine but then suddenly returns the response below when attempting to retrieve a new access token using the stored refresh token.

{"error_description":"expired access/refresh token","error":"invalid_grant"}

I've looked over many settings and everything seems to be configured to never expire the refresh token. Are there other usages that can cause them to expire? Can using it too many times from our servers to request an access token cause it to expire? Are you supposed to refresh the refresh token?

What is the recovery process once this happens? Right now the only solution we have is for the user to reauthorize the app which is a really bad scenario to be in as all communication attempts in the meantime just die.

We also have normal users (non admin) who OAuth into a web app via our Connected App. Should we not be requesting "offline_access" and "refresh_token" in scope for normal users who just need to authenticate? We tried asking for nothing and bare minimums too but they don't seem to have an effect.

I've seen hints from other questions here that say you can only ask for 5 refresh tokens before the last ones expire. We've tried signing in as an admin and user dozens of times to reproduce the issue but we can't trigger the problem. Are there other IP address restrictions or things we could look into as well?

---

This may be related as well. When does the Use Count highlighted here increase? What does that number represent? Every successful OAuth exchange or only when certain refresh tokens or offline access are also requested? Is there a limit?

---

UPDATE - Feb 9 2015

I checked the User Session Information tab after signing in with OAuth and I can see the newly created OAuth2 session there. The Valid Until definitely seems to be correlated to the 15min Timeout Value set for the account. I am under the impression that this value will expire the requested AccessToken and not the RefreshToken for the user.

I can also confirm that using the RefreshToken after the Valid Until date has passed will reset the Valid Until date and give me a new session valid for 15 more minutes. Also, OAuth2 sessions do not seem to be associated with a parent session.

---

UPDATE - Feb 10 2015

We were finally been able to reproduce the issue but I still do not understand the behavior we're seeing. To reproduce the issue I had to perform 4 consecutive logins using OAuth without performing a request for an AccessToken using the RefreshToken. On the 4th sign in we noticed that the Use Count would drop for some high number (10+ in our case) down to 4. But why 4? Here's what we've been able to deduce.

Requesting an AccessToken/Session using the RefreshToken will always increase the Use Count but will not add a new session row in the Session Management list. You can perform this request as many times as you want. It has no effect on the currently assigned RefreshToken.

Authenticating a user with OAuth seems to always add a new session row in the Session Management list. It will also increase the Use Count up to 4, but no higher. 4 seems to be some sort of magic number here. SFDC seems to create a new session for each successful authentication even if it's for the same user and the previous one hasn't expired yet. Once you pass 4 it seems to invalidate all your previous sessions and tokens.

What's interesting is if you sign in 2 times, then programatically request an AccessToken/Session using the RefreshToken, then sign in an additional 2 more times you don't experience the issue. Using the RefreshToken has some effect on the current outstanding sessions for the user and will give you 4 more successful sign ins.

Is this normal behavior? Should re-authenticating over and over again really create brand new sessions each time for the same user? Could this be because I'm not actually signing out via OAuth for each attempt? Does SFDC think that I'm signing in from different devices and there is a limit of 4 concurrent sessions?

---

UPDATE - Feb 13 2015

It looks like calling the revoke API between each sign in has no effect. I signed in as a user, signed out and called revoke to remove the access token from SF and repeated this 5 times. I can see the OAuth Session disappear from the Session Management list but on the 5th sign in the refresh token once again expired (and the Use Count on the Connected Apps OAuth Usage page once again dropped down to a static 4).

It looks like my only option is to perform a Token Refresh after every single sign in.

Answer 1

I see you've discovered most of this for yourself, but I had this drafted, so I thought I'd post it also, in case it fills in any gaps.

From the docs on connected apps:

An application may be listed more than once. Each time you grant access to an application, it obtains a new access token. Requests for refresh tokens increase the Use Count displayed for the application. You must grant access to your Salesforce data from each device that you use, for example, from both a laptop and a desktop computer. The default limit is five access tokens for each application. Newer applications (using the OAuth 2.0 protocol) are automatically approved for additional devices after you've granted access once. OAuth 2.0 applications can be listed more than once. Each row in the table represents a unique grant, so if an application requests multiple tokens with different scopes, you’ll see the same application multiple times.

A given user may only have 5 access tokens authorized for a given connected app. Since each refresh token can potentially issue an access token, they are counted in that total. The way to think about this is that only the most recent 5 authorizations are valid. When the user goes through login the sixth time, the oldest authorization is invalidated and that refresh token will no longer work.

How do you manage this? Don't ask for a refresh token if you're not going to use it. Don't use the same connected app for interactive and 'batch' operations. If you want to keep a refresh token around, then create a connected app for that purpose, and use a different one for login.

Finally, consider using the JWT Bearer Token flow rather than holding on to a refresh token obtained interactively. It will give you much more predictable behavior.

Answer 2

https://help.salesforce.com/apex/HTViewHelpDoc?id=remoteaccess_request_manage.htm

An application may be listed more than once. Each time you grant access to an application, it obtains a new access token. Requests for refresh tokens increase the Use Count displayed for the application. You must grant access to your Salesforce data from each device that you use, for example, from both a laptop and a desktop computer. The default limit is five access tokens for each application. Newer applications (using the OAuth 2.0 protocol) are automatically approved for additional devices after you've granted access once. OAuth 2.0 applications can be listed more than once. Each row in the table represents a unique grant, so if an application requests multiple tokens with different scopes, you’ll see the same application multiple times.

Even if the connected app tried and failed to access your information because it could not login, the Use Count and Last Used fields are still updated.

It appears that SFDC treats every individual "sign in" as a new device requesting OAuth access via your Connected App. The example they provided about needing to grant access on a laptop and desktop is very misleading because it has absolutely nothing to do with "devices" at all! SFDC merely remembers the last 5 OAuth granted tokens at any given time. (Revoking doesn't help either)

For example, if a user signs in and grants your Connected App access on a desktop website and then later signs in using a mobile app that user will have used up 2 of the 5 devices. If that user simply signs out of either the mobile app or website and and signs in again they will have used 3 of the 5. If the user repeats this sign in process 2 more times then the first device that was granted access will be revoked. If your app had stored the RefreshToken only from that first sign in and never from the subsequent sign ins then your app's token will be invalid and be unable to communicate with SFDC.