ogdirectory.com
robots.txt

Robots Exclusion Standard data for ogdirectory.com

Resource Scan

Scan Details

Site Domain ogdirectory.com
Base Domain ogdirectory.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan3/12/2025, 10:55:33 PM
Next Scan 6/10/2025, 10:55:33 PM

Last Successful Scan

Scanned5/17/2024, 8:17:47 AM
URL https://ogdirectory.com/robots.txt
Redirect https://www.ogdirectory.com/robots.txt
Redirect Domain www.ogdirectory.com
Redirect Base ogdirectory.com
Domain IPs 104.21.84.220, 172.67.197.99, 2606:4700:3035::6815:54dc, 2606:4700:3035::ac43:c563
Redirect IPs 104.21.84.220, 172.67.197.99, 2606:4700:3035::6815:54dc, 2606:4700:3035::ac43:c563
Response IP 172.67.197.99
Found Yes
Hash 6039c91671fa0e0fbb32a50380b8cd7451f7832926f7430f3f660ce7645a6ed3
SimHash 4015c0468257

Groups

*

Rule Path
Disallow /custom/domain*
Disallow /custom/log
Disallow /custom/profile
Disallow /custom/tmp
Disallow /layout
Disallow /includes
Disallow /search/map/summary
Allow /custom/domain*/sitemap*
Allow /custom/domain*/tmp*.js
Allow /custom/domain*/tmp*.css
Allow /custom/domain*/theme*
Allow /custom/domain*/image_files/*
Allow /custom/domain*/content_files/*
Allow /

Other Records

Field Value
crawl-delay 10