Is there a way to modify or change the robots.txt for our public facing KB articles, so I can change which search engines can crawl them, etc?


On force.com sites you can create a custom robots.txt file and modify .

Here are some the links to assist

Help Resource From SFDC

Best Practices For SEO

Setting Up SEO

In nut shell to summarize from these docs ,you will need a visualforce page with below lines of code

<apex:page contentType="text/plain">
 User-agent: *
Disallow: /   # hides everything from ALL bots
Allow: /<path-prefix-1>/s  # add path you want to open to bots
Allow: /<path-prefix-2>/s  # add path you want to open to bots
Sitemap: http://<community_URL>/s/sitemap.xml
Sitemap: http://<community_URL>/<sub_path>/s/sitemap.xml

And then attach this visualforce to site settings using sites record .

| improve this answer | |
  • Thanks Mohith. If I create a visualforce page, how do I associate it with the public KB Articles? – Sidereal Apr 13 '17 at 19:50
  • You associate with the site .Then in that indicate path for articles in the Sitemap variable shown above – Mohith Shrivastava Apr 13 '17 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.