How to override robots.txt in nopCommerce 3.x

If you want to override robots.txt in nopCommerce you need to setup the robots.custom.txt file.

There is a robots.additions.txt file in the root that looks like it should work.  It does not (as of nopCommerce 3.90).

If you do a search on the below in source code you can see that it is robots.custom.txt that is used.
string robotsFile = System.IO.Path.Combine(_webHelper.MapPath("~/"), "robots.custom.txt"); "

How to...

To override robots.txt (fixes /.well-known/* apple app lookup by google bot)
1. Open your http://yoursite.extension/robots.txt
2. Copy all the contents.
3. Create a file called robots.custom.txt*
5. Paste the contents from robots.txt to robots.custom.txt
6. Add Disallow: /.well-known/* to
7. Copy robots.custom.txt back up to the root of the website.
8. Open http://yoursite.yourextension/robots.txt NOT robots.custom.txt ...verify contents.

*NOTE: There is a file in the root of the website called robots.additions.txt (this file does not work).

...fixes the annoying "/.well-known/*" error that shows up in your log because Google bot is looking for an app link to the Apple iTunes store for a related app for your website.