acog.org
robots.txt

Robots Exclusion Standard data for acog.org

Resource Scan

Scan Details

Site Domain acog.org
Base Domain acog.org
Scan Status Ok
Last Scan2024-11-15T12:09:03+00:00
Next Scan 2024-12-15T12:09:03+00:00

Last Scan

Scanned2024-11-15T12:09:03+00:00
URL https://www.acog.org/robots.txt
Domain IPs 104.22.32.116, 104.22.33.116, 172.67.38.235, 2606:4700:10::6816:2074, 2606:4700:10::6816:2174, 2606:4700:10::ac43:26eb
Response IP 172.67.38.235
Found Yes
Hash bc2e853b54f326b50a94b2c8ad5f3349a300123ac87a34d0bc129a8195453327
SimHash ba059b658445

Groups

*

Rule Path
Disallow /js/
Disallow /css/
Disallow /xsl/
Disallow /data/
Disallow /temp/
Disallow /Assets/
Disallow /images/
Disallow /upload/
Disallow /scripts/
Disallow /-/media/
Disallow /sitecore/
Disallow /App_Data/
Disallow /App_Config/
Disallow /App_Browsers/
Disallow /sitecore_files/
Disallow /womens-health/find-an-ob-gyn/

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

typhoeus

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

grapeshot

Rule Path
Allow /
Disallow /js/
Disallow /css/
Disallow /xsl/
Disallow /data/
Disallow /temp/
Disallow /Assets/
Disallow /images/
Disallow /upload/
Disallow /scripts/
Disallow /-/media/
Disallow /sitecore/
Disallow /App_Data/
Disallow /App_Config/
Disallow /App_Browsers/
Disallow /sitecore_files/
Disallow /womens-health/find-an-ob-gyn/

facebookbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

sogou pic agent

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

sogou pic spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou head spider

Rule Path
Disallow /

sogou-test-spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.acog.org/sitemap.xml
sitemap https://www.acog.org/acogsitemap.xml

Warnings

  • 4 invalid lines.