1

Is there a way to modify or change the robots.txt for our public facing KB articles, so I can change which search engines can crawl them, etc?

Improve this question

1 Answer 1

Reset to default
1

On force.com sites you can create a custom robots.txt file and modify .

Here are some the links to assist

Help Resource From SFDC

Best Practices For SEO

Setting Up SEO

In nut shell to summarize from these docs ,you will need a visualforce page with below lines of code

<apex:page contentType="text/plain">
 User-agent: *
Disallow: /   # hides everything from ALL bots
Allow: /<path-prefix-1>/s  # add path you want to open to bots
Allow: /<path-prefix-2>/s  # add path you want to open to bots
Sitemap: http://<community_URL>/s/sitemap.xml
Sitemap: http://<community_URL>/<sub_path>/s/sitemap.xml
</apex:page>

And then attach this visualforce to site settings using sites record .

Improve this answer
2
  • Thanks Mohith. If I create a visualforce page, how do I associate it with the public KB Articles?
    – Sidereal
    Apr 13, 2017 at 19:50
  • You associate with the site .Then in that indicate path for articles in the Sitemap variable shown above
    – Mohith Shrivastava
    Apr 13, 2017 at 20:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .