google indexing

Question

I have salesforce community (salesforce site) and have to index that site by google.

1. How can I index my site by google? Ex:- https://Mywebsite.com/

2. I have to allow few pages only(not all pages), how can I do this ?

3. I have some user profiles which is a public page. it can be with query string parameter. How to index those user profiles by google ? Ex:- https://Mywebsite.com/apex/UserProfile?id=001K000000sLyjhIAC

This id will be changed based on the user profile .

Answer 1

Go to Setup > Develop > Sites. There should be an entry there corresponding to your community (there's a 1-1 relationship between a Site and a Community)

First create a Visualforce page containing (for example) the following:

<apex:page contentType="text/plain">
    User-agent: *
    Allow: /
</apex:page>

Choose this page on the Site Robots.txt attribute of the Site associated with your Community.

Then confirm you can see its contents when you visit your community at https://mywebsite.com/robots.txt

User-agent: *
Allow: /

A bit more reading on how robots.txt works will help you allow/disallow the appropriate pages. Anything behind a login cannot be crawled, and those custom user profile pages must be discoverable via links.

Answer 2

Robots.txt: yes, definitely start with that.

We also used a similar technique to add an XML sitemap. This way, instead of hoping google finds all your pages, you can make a list of them and then submit that list via google Webmaster Tools (set up an acct there if you don't have one already).

SF sites don't require /apex like you do in salesforce.

Final google-oriented comment: look into salesforce's built-in URL-rewriter class. Google prefers domain.com/human-readable-title vs. domain.com/ASF63N3RA73D1D5