runsignup.com
robots.txt

Robots Exclusion Standard data for runsignup.com

Resource Scan

Scan Details

Site Domain runsignup.com
Base Domain runsignup.com
Scan Status Ok
Last Scan2024-09-28T17:37:50+00:00
Next Scan 2024-10-28T17:37:50+00:00

Last Scan

Scanned2024-09-28T17:37:50+00:00
URL https://runsignup.com/robots.txt
Domain IPs 3.222.216.205, 34.194.61.140, 54.165.155.11
Response IP 54.165.155.11
Found Yes
Hash cfa58cef8b34a75d279d5fe3c74743fda12316d828d1fd1915bc719021095ef6
SimHash 6c1eef406edb

Groups

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /Race/NJ/Moorestown/InYourFaceScrapper
Disallow /race/nj/moorestown/inyourfacescrapper
Disallow /captcha
Disallow /em/
Disallow /Logout
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /Race/Photos/ViewPhoto/
Disallow /Race/Results/Simple/
Disallow /Race/Results/*/FinishersCert
Disallow /Race/Results/*/FinishersCertImg
Disallow /*?*PHPSESSID*
Disallow /*?*embedId2*
Disallow /*?*embedToken*
Disallow /*?*regToken*
Disallow /*?*RefCode=*
Disallow /*?*err=*
Disallow /*?*smsg=*
Disallow /*?*redirect=*
Disallow /*?*autoLogin*
Disallow /*?*remMeAttempt*

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://runsignup.com/sitemap.xml

Comments

  • Place wildcards last since some robots don't know how to parse them