Helpcenter +918010117117 https://cdn.storehippo.com/s/573db3149f0d58741f0cc63b/ms.settings/5256837ccc4abf1d39000001/57614ef64256dc6851749879-480x480.png" [email protected] https://www.facebook.com/StoreHippohttps://twitter.com/StoreHippohttps://www.linkedin.com/company/hippoinnovations/https://plus.google.com/+Storehippo/posts
B4,309-10 Spaze iTech Park, Sector 49, Sohna Road, 122001 Gurgaon India
call to replace anchor tags contains '/admin' in href

Robots.txt

Robots.txt is a file that contains the directives for search engines. It tells the search engine crawlers which URLs the crawler cannot access. Most major search engines (including Google, Bing, and Yahoo) recognize and honor robots.txt requests. This is used mainly to avoid overloading your site with requests and it is not a mechanism for keeping a web page out of Google. 

A robots.txt file is automatically generated for your store. This file lives at the root of your site. You can view the robots.txt file by appending "robots.txt" at your store's URL (for example, www.yourstore.com/robots.txt). robots.txt is a plain text file that follows the Robots Exclusion Standard.

The automatically generated robots.txt file looks like as shown below:

User-agent: *
Disallow: /admin
Disallow: /checkout
Disallow: /order
Disallow: /user
Disallow: /account
Disallow: /collections/*+*
Disallow: /cart
Sitemap:https://storehippo.com/sitemap.xml
User-agent: AdsBot-Google
Disallow: /admin
Disallow: /checkout
Disallow: /cart
User-agent: Nutch
Disallow: /

Adding custom robots.txt

In case, if you want to update/edit the custom robots.txt file, you can add the custom robots.txt file. To do so, follow the steps mentioned below:

  • Click the Save & Continue button to apply the changes.

Now, the default robots.txt URL (www.yourstore.com/robots.txt) will serve content from the custom robots.txt file.

Related Topics

2022-02-10T13:53:35.191Z