What are some SEO best practices folks have learned with Salesforce Commerce Cloud (Demandware)? Specifically with configuring URL structures and Robots.txt
This blog post from Salesforce outlines several ecommerce SEO best practices from a business and marketing perspective: https://www.salesforce.com/products/commerce-cloud/resources/your-checklist-for-ecommerce-seo-done-right/#
Implementing this in Salesforce B2C Commerce will require understanding of several areas of Business Manager, and this document has some technical-facing SEO best practices: https://documentation.b2c.commercecloud.salesforce.com/DOC1/topic/com.demandware.dochelp/SearchEngineOptimization/SEOBestPractices.html
SFCC Infocenter has a lot of documents on the topic. Sitemap, URL Rules, and the URL Request Analyzer are especially critical tools.
This is a very broad subject. The documentation has several pages on this.
IMHO, it comes down to common sense: be relevant and be fast.
To your question about robots.txt, here's two bits that were important for me (I'm sure there's more):
- Make sure you announce a (automatically generated) sitemap
- Look at your technical reports, CC Digital offers several reports on bot activity. It may happen they actually crawl too aggressively, impacting site speed for others. So CC support has knowlegdebase articles about limiting them, similar to this post.