7

I have built an export/import tool for Knowledgebase articles using Partner API. Everything works fine, the only functionality I can't produce is the ability to export images, hence obviously I can't import the images.

After much reading on the subject I understand that I need to use HttpClient after giving the access token cookie, but when requesting in that manner the response comes back as 'Page Redirect' instead of returning the image.

Image url example, as it appears inside the rich text area field: https://c.eu0.content.force.com/servlet/rtaImage?eid=ka3200000004MwH&feoid=00N20000008fSIK&refid=0EM20000000TzfR

Note: I'm also using the access token to connect to the partner API, it's works great.

My tool will be hosted on Heroku using servlets, this is a simple code just for trying to fetch the image:

OAuthServlet

public class OAuthServlet extends HttpServlet
{
    private static final long serialVersionUID = 1L;

    private static final String ACCESS_TOKEN = "ACCESS_TOKEN";
    private static final String INSTANCE_URL = "INSTANCE_URL";

    private String clientId = null;
    private String clientSecret = null;
    private String redirectUri = null;
    private String environment = null;
    private String authUrl = null;
    private String tokenUrl = null;

    public void init() throws ServletException
    {
        clientId = System.getenv("SFDC_OAUTH_CLIENT_ID");
        clientSecret = System.getenv("SFDC_OAUTH_CLIENT_SECRET");
        redirectUri = "http://localhost:8080/sfdc/callback.html";
        environment = "https://login.salesforce.com";

        try
        {
            authUrl = environment + "/services/oauth2/authorize?response_type=code&client_id=" + clientId + "&redirect_uri="
                    + URLEncoder.encode(redirectUri, "UTF-8");
        }
        catch (UnsupportedEncodingException e)
        {
            throw new ServletException(e);
        }

        tokenUrl = environment + "/services/oauth2/token";
    }

    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
    {
        String accessToken = (String) request.getSession().getAttribute(ACCESS_TOKEN);

        if (accessToken == null)
        {
            String instanceUrl = null;

            if (request.getRequestURI().endsWith("login"))
            {
                // we need to send the user to authorize
                response.sendRedirect(authUrl);
                return;
            }
            else
            {
                System.out.println("Auth successful - got callback");

                String code = request.getParameter("code");

                HttpClient httpclient = new HttpClient();

                PostMethod post = new PostMethod(tokenUrl);
                post.addParameter("code", code);
                post.addParameter("grant_type", "authorization_code");
                post.addParameter("client_id", clientId);
                post.addParameter("client_secret", clientSecret);
                post.addParameter("redirect_uri", redirectUri);

                try
                {
                    httpclient.executeMethod(post);

                    try
                    {
                        String responseBody = post.getResponseBodyAsString();
                        JSONObject authResponse = new JSONObject(responseBody);
                        System.out.println("Auth response: " + authResponse.toString(2));

                        accessToken = authResponse.getString("access_token");
                        instanceUrl = authResponse.getString("instance_url");

                        System.out.println("Got access token: " + accessToken);

                        Cookie session = new Cookie(ACCESS_TOKEN, accessToken);
                        session.setPath("/");
                        session.setMaxAge(-1); // cookie not persistent,
                                                // destroyed on browser exit
                        response.addCookie(session);
                    }
                    catch (JSONException e)
                    {
                        e.printStackTrace();
                        throw new ServletException(e);
                    }
                }
                finally
                {
                    post.releaseConnection();
                }
            }

            // Set a session attribute so that other servlets can get the access
            // token
            request.getSession().setAttribute(ACCESS_TOKEN, accessToken);

            // We also get the instance URL from the OAuth response, so set it
            // in the session too
            request.getSession().setAttribute(INSTANCE_URL, instanceUrl);

        }
        response.sendRedirect(request.getContextPath() + "/welcome");
    }

WelcomeServlet

public class WelcomeServlet extends HttpServlet
{
    private static final long serialVersionUID = 1L;

    private static final String ACCESS_TOKEN = "ACCESS_TOKEN";

    public void init() throws ServletException { }

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
    {
        // TODO Auto-generated method stub
        String urlString = "https://c.eu0.content.force.com/servlet/rtaImage?eid=ka3200000004MwH&feoid=00N20000008fSIK&refid=0EM20000000TzfR";
        HttpClient client = new HttpClient();
        client.getParams().setParameter("http.protocol.single-cookie-header", true);
        client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);

        GetMethod method = new GetMethod();
        String accessToken = (String) request.getSession().getAttribute(ACCESS_TOKEN);
        method.setRequestHeader("Authorization", "Bearer " + accessToken);
        method.setURI(new URI(urlString, true));
        int returnCode = client.executeMethod(method);
        if (returnCode != HttpStatus.SC_OK)
        {
            System.err.println("Unable to fetch image, status code: " + returnCode);
        }
        InputStream imageData = method.getResponseBodyAsStream();
        OutputStream out = new BufferedOutputStream(new FileOutputStream("test-image.jpg"));
        for (int i; (i = imageData.read()) != -1;)
        {
            out.write(i);
        }
        imageData.close();
        out.close();

        response.sendRedirect("welcome.jsp");
    }
}

Update - 1: I have found something that may help, I have used Live HTTP Header to catch the request when putting the above sample image url in the url path of a browser, after I was login, and I have seen that the request is actually using Cookie as a request header and not Authorization, the request looks like this:

GET /servlet/rtaImage?eid=ka3200000004MwH&feoid=00N20000008fSIK&refid=0EM20000000TzfR HTTP/1.1  
Host: c.eu0.content.force.com:443  
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch 
Accept-Language: en-US,en;q=0.8,he;q=0.6 
Cookie: BrowserId=JH_ibxCpQ56g8RH-T-o8XQ;sid=SESSION_ID;sid_Client=SESSION_CLIENT;clientSrc=CLIENT_SRC_IP; inst=APP2
Referer: https://emea.salesforce.com/_ui/identity/phone/AddPhoneNumber?retURL=%2Fcontent%2Fsession%3Furl%3Dhttps%253A%252F%252Fc.eu0.content.force.com%252Fservlet%252FrtaImage%253Feid%253Dka3200000004MwH%2526feoid%253D00N20000008fSIK%2526refid%253D0EM20000000TzfR&d=m&display=page
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36

Note 1: that the Cookie header is using an sid attribute, I first assumed that this sid is the access token I got from my OAuth 2.0 connection, but when using it in the request header I still got the redirect page, so then I have used the session Id straight from the API when login to the Partner API using user name, password+security token, but yet I got the redirect page, so last I was copied this sid and use it directly in my code and it worked, but I don't know how to get this sid, I don't see it in my Servlet request cookies.

Note 2: in all the above tests I have changed the following line:

method.setRequestHeader("Authorization", "Bearer " + accessToken);

To:

method.setRequestHeader("Cookie", "sid=" + SomeSID);

and again when using directly the sid from the request I got when looking at Live HTTP Headers it worked, the image was downloaded.

Update - 2: In the OAuthServlet.doGet, I try this:

Cookie sid = new Cookie("sid", accessToken);
sid.setPath("/");
sid.setMaxAge(-1);                                          
sid.setSecure(true);
response.addCookie(sid);

Thanks in advanced.

5
+50

Where did you get your accessToken from? If you used OAuth to establish the session then you will need to have the web OAuth scope to use the sessionid for what is essentially a screen scraping web request. Having the api scope would be sufficient for the Partner API, but for for mocking UI requests.

enter image description here

Other than that, as you have found, you need to put the sessionId/accessToken into a sid cookie along with the request.


You can check your OAuth Scopes under

Setup > App Setup > Create > Apps > [App Name].

It should appear next to Selected OAuth Scopes.

When calling /services/oauth2/authorize I'm using response_type=Token. I'm also passing a scope query string parameter that includes web. So I've specified the web scope on both the App and when initiating the OAuth process.

It appears you are using the Web Server OAuth Authentication Flow (based on the response_type=code). The scope parameter is applicable here as well.

scope
Specifies what data your application can access. See “Scope Parameter Values” in the online help for more information.

So you try modifying the authUrl line:

authUrl = environment + "/services/oauth2/authorize?response_type=code&client_id=" + clientId +
 "&redirect_uri=" + URLEncoder.encode(redirectUri, "UTF-8") +
 "&scope=full%20web";

Once you have a sessionId/accessToken, you can verify that it is valid for a web session in a browser using

https://<instanceUrl>/secur/frontdoor.jsp?sid=<accessToken>"

This should bounce you into a browser session with the sid cookie set. If it doesn't work, the session isn't valid for the web, which you will need to screenscrape an image via servlet/rtaImage.

Final solution (By @GoldenAxe), that is only an example to check that the image fetching is working correctly, of-course it can be done much neatly. In the WelcomeServlet.doGet:

String accessToken = (String) request.getSession().getAttribute(ACCESS_TOKEN);
String apiEndPoint = (String) request.getSession().getAttribute(INSTANCE_URL);
CookieManager cookieManager = new CookieManager();
cookieManager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
CookieHandler.setDefault(cookieManager);
URL url = null;
try
{
    url = new URL(apiEndPoint + "/secur/frontdoor.jsp?sid=" + accessToken);
    // open's a connection with the url specified and returns URLConnection object
    URLConnection urlConnection = url.openConnection();
    // get's the contents from this url specifies
    urlConnection.getContent();
}
catch (MalformedURLException e)
{
    // TODO Auto-generated catch block
    e.printStackTrace();
}
catch (IOException e)
{
    e.printStackTrace();
}
String sid = "";
// returns the cookie store(bunch of cookies)
CookieStore cookieStore = cookieManager.getCookieStore();
// getting cookies which returns in the form of List of type HttpCookie
List<HttpCookie> listOfcookies = cookieStore.getCookies();
for (HttpCookie httpCookie : listOfcookies)
{
    System.out.println("Cookie Name : " + httpCookie.getName() + " Cookie Value : " + httpCookie.getValue());
    if (httpCookie.getName().compareTo("sid") == 0)
    {
        sid = httpCookie.getValue();
    }
}
String urlString = "https://c.eu0.content.force.com/servlet/rtaImage?eid=ka3200000004MwH&feoid=00N20000008fSIK&refid=0EM20000000TzfR";
HttpClient client = new HttpClient();
GetMethod method = new GetMethod();
method.setRequestHeader("Cookie", "sid=" + sid);
method.setURI(new URI(urlString, true));
int returnCode = client.executeMethod(method);
if (returnCode != HttpStatus.SC_OK)
{
    System.err.println("Unable to fetch image, status code: " + returnCode);
}
InputStream imageData = method.getResponseBodyAsStream();
OutputStream out = new BufferedOutputStream(new FileOutputStream("test-image.jpg"));
for (int i; (i = imageData.read()) != -1;)
{
    out.write(i);
}
imageData.close();
out.close();
method.releaseConnection();
  • I have full scope (meaning I use all available OAuth Scopes), I have checked it deeper and I see that the sid which used in the image get request is not the user session id, not the API session id and not the OAuth session id, I don't know where to get it... – GoldenAxe Aug 11 '14 at 8:36
  • @GoldenAxe When using the browser it should be coming from the existing sid cookie. Can you use your access token to access the website by replacing the value in the sid cookie? I know full sounds like everything, but for some reason I think you need to explicitly ask for the web OAuth Scope. – Daniel Ballinger Aug 11 '14 at 8:47
  • I have update my question (Update -2) that what I was trying to do with sid cookie getting the access token, but it not seems to work. how do I explicity asks fro the web OAuth Scope? – GoldenAxe Aug 11 '14 at 9:16
  • 1. When using response_type=Token, I don't have any kind of "code" parameter (request.getParameter("code") is null), what should be the name of the parameter (request.getParameter("token") also gives me null)? 2. As I said I already taking all the available scopes on the connected app settings, when using scope parameter the app asks me each time to approve access. Many thanks for the help! – GoldenAxe Aug 12 '14 at 9:18
  • You should be able to keep response_type=code like you currently have. I know it doesn't make much sense, but I would try explicitly asking for the web scope in addition to the full scope. Separate them with a URL encoded space. Once you do have an accessToken, confirm it is a valid web sessionId using the frontdoor.jsp in a browser window. – Daniel Ballinger Aug 12 '14 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.