Log In Sign Up

freewebtools.com
robots.txt

Robots Exclusion Standard data for freewebtools.com

Resource Scan

Scan Details

Site Domain freewebtools.com
Base Domain freewebtools.com
Scan Status Ok
Last Scan2025-09-17T17:37:54+00:00
Next Scan 2025-09-24T17:37:54+00:00

Last Scan

Scanned2025-09-17T17:37:54+00:00
URL https://freewebtools.com/robots.txt
Redirect https://www.freewebtools.com/robots.txt/
Redirect Domain www.freewebtools.com
Redirect Base freewebtools.com
Domain IPs 104.21.26.148, 172.67.136.133, 2606:4700:3037::6815:1a94, 2606:4700:3037::ac43:8885
Redirect IPs 104.21.26.148, 172.67.136.133, 2606:4700:3037::6815:1a94, 2606:4700:3037::ac43:8885
Response IP 104.21.26.148
Found Yes
Hash 090f28cd7e565e0a08c18ba6849356d0828843f69cda4d03e76b47860b2fc871
SimHash 5818cdc52fda

Groups

*

Rule Path
Disallow /rimes/*/*/
Disallow /site-review/*
Disallow /*/?nowprocket=1
Disallow /?*

googlebot

Rule Path
Disallow /compteur-youtube/?*
Disallow /rimes/*/*/
Disallow /?*
Disallow /*/?nowprocket=1

bingbot

Rule Path
Disallow /youtube-money-calculator/*
Disallow /?*

yandexbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

dotbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.freewebtools.com/sitemaps/sitemap.xml
sitemap https://www.freewebtools.com/sitemap.xml
sitemap https://www.freewebtools.com/sitemap_index.xml
sitemap https://www.freewebtools.com/sitemaps/sitemapIPS.xml
sitemap https://www.freewebtools.com/sitemaps/sitemapALLIPSS.xml
sitemap https://www.freewebtools.com/sitemaps/sitemapALLIPS.xml
sitemap https://www.freewebtools.com/post-sitemap.xml
Back to top