bigbeachmarathon.com
robots.txt

Robots Exclusion Standard data for bigbeachmarathon.com

Resource Scan

Scan Details

Site Domain bigbeachmarathon.com
Base Domain bigbeachmarathon.com
Scan Status Ok
Last Scan2024-09-30T11:26:39+00:00
Next Scan 2024-10-30T11:26:39+00:00

Last Scan

Scanned2024-09-30T11:26:39+00:00
URL https://bigbeachmarathon.com/robots.txt
Redirect https://www.bigbeachmarathon.com/robots.txt
Redirect Domain www.bigbeachmarathon.com
Redirect Base bigbeachmarathon.com
Domain IPs 3.210.12.204
Redirect IPs 3.210.12.204, 3.210.169.147
Response IP 3.210.12.204
Found Yes
Hash cfa58cef8b34a75d279d5fe3c74743fda12316d828d1fd1915bc719021095ef6
SimHash 6c1eef406edb

Groups

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /Race/NJ/Moorestown/InYourFaceScrapper
Disallow /race/nj/moorestown/inyourfacescrapper
Disallow /captcha
Disallow /em/
Disallow /Logout
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /Race/Photos/ViewPhoto/
Disallow /Race/Results/Simple/
Disallow /Race/Results/*/FinishersCert
Disallow /Race/Results/*/FinishersCertImg
Disallow /*?*PHPSESSID*
Disallow /*?*embedId2*
Disallow /*?*embedToken*
Disallow /*?*regToken*
Disallow /*?*RefCode=*
Disallow /*?*err=*
Disallow /*?*smsg=*
Disallow /*?*redirect=*
Disallow /*?*autoLogin*
Disallow /*?*remMeAttempt*

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://runsignup.com/sitemap.xml

Comments

  • Place wildcards last since some robots don't know how to parse them