#Manual sitemap with program from https://wonderwebware.com/sitemap-generator/ Sitemap: https://www.arbiter.com/sitemap.xml #CronJob sitemap Sitemap: https://www.arbiter.com/sitemapG/sitemapG.xml User-agent: * Disallow: /admin/ Disallow: /cgi-bin/ Disallow: /CronJobs/ # Google wanted the /css and /js directory opened for their algorithm # Did as told 28 July 2015 #Disallow: /css/ #Disallow: /js/ # Google had some trouble with this one. No good reason to block it so... #Disallow: /search/ Disallow: /demo/ Disallow: /inc/ Disallow: /php/ Disallow: /resources/sales/rep/ Disallow: /sitemapG/ Disallow: /support/downloads/data/ Disallow: /testing/ Disallow: /vendor_vault/ Disallow: /documentation_vault/ Disallow: /min/ #Disallow: /machform/ Disallow: /resources/sales/ Disallow: /images/amzn/ Disallow: /files/product-attachments/ # Things for the userspice login install Disallow: /users/ Disallow: /usersc/ # Block the .sha256 files # Google complained about blocking them so stopped 20 September 2021 #Disallow: /*.sha256$