ohsemput.com
robots.txt

Robots Exclusion Standard data for ohsemput.com

Resource Scan

Scan Details

Site Domain ohsemput.com
Base Domain ohsemput.com
Scan Status Ok
Last Scan2024-09-25T16:16:53+00:00
Next Scan 2024-10-02T16:16:53+00:00

Last Scan

Scanned2024-09-25T16:16:53+00:00
URL https://ohsemput.com/robots.txt
Domain IPs 104.21.45.115, 172.67.213.148, 2606:4700:3031::6815:2d73, 2606:4700:3037::ac43:d594
Response IP 104.21.45.115
Found Yes
Hash c22d9f1a5f1dda8ec808d658d0a69e978d9bc5b0aa336bb26eb0164d79fbb48f
SimHash 6832d441c5f3

Groups

msnbot

Rule Path
Allow /
Disallow /search/
Disallow /news/
Disallow /category.php

Other Records

Field Value
crawl-delay 120

*

Rule Path
Allow /
Disallow /search/
Disallow /news/
Disallow /category.php

discobot

Rule Path
Disallow /

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 4

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
sitemap http://ohsemput.com/sitemap.xml