Log In Sign Up

isuit.it
robots.txt

Robots Exclusion Standard data for isuit.it

Resource Scan

Scan Details

Site Domain isuit.it
Base Domain isuit.it
Scan Status Ok
Last Scan2025-09-08T21:24:58+00:00
Next Scan 2025-10-08T21:24:58+00:00

Last Scan

Scanned2025-09-08T21:24:58+00:00
URL https://isuit.it/robots.txt
Domain IPs 104.26.2.187, 104.26.3.187, 172.67.73.199, 2606:4700:20::681a:2bb, 2606:4700:20::681a:3bb, 2606:4700:20::ac43:49c7
Response IP 104.26.2.187
Found Yes
Hash 5af6af4e50c9eed30c76cb07a3881a1da3936a85b1ccb15e4c438ca07e6e2836
SimHash 73471850e219

Groups

*

Rule Path
Disallow /catalogsearch/
Disallow /ajaxcart/
Disallow /*color_filter%3D*

facebookexternalhit/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

amazonbot/0.1

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

thesis-research-bot

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://isuit.it/sitemaps/google_sitemap_en.xml
sitemap https://isuit.it/sitemaps/google_sitemap_categories_brand_en.xml
sitemap https://isuit.it/sitemaps/google_sitemap_custom_pages.xml
Back to top

Comments

  • Specific domain robots
  • default isuit.it
  • robots.txt
  • Folder
  • Disallow: /referer/
  • Disallow: /customer/
  • Disallow: /quickview/
  • Parameters
  • Disallow: /*dir=*
  • Disallow: /*order=*
  • Disallow: /*limit=*
  • Disallow: /*currency=*
Back to top