Log In Sign Up

agentroof.com
robots.txt

Robots Exclusion Standard data for agentroof.com

Resource Scan

Scan Details

Site Domain agentroof.com
Base Domain agentroof.com
Scan Status Ok
Last Scan2024-05-08T05:16:31+00:00
Next Scan 2024-06-07T05:16:31+00:00

Last Scan

Scanned2024-05-08T05:16:31+00:00
URL https://agentroof.com/robots.txt
Domain IPs 104.26.6.108, 104.26.7.108, 172.67.72.173, 2606:4700:20::681a:66c, 2606:4700:20::681a:76c, 2606:4700:20::ac43:48ad
Response IP 172.67.72.173
Found Yes
Hash 8c8dd34832753747b456d2807c12c62dd211503249bfbe79e240adeebdcd275c
SimHash f8015555c795

Groups

*

Rule Path
Disallow /

applebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

slurp

Rule Path
Allow /

twitterbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://agentroof.com/sitemap.xml
Back to top

Comments

  • Notice: Collection of data on this website through automated means is
  • prohibited unless you have express written permission from webmaster
  • and may only be conducted for the limited purpose contained in said
  • permission.
Back to top