rsupartner.com
robots.txt

Robots Exclusion Standard data for rsupartner.com

Resource Scan

Scan Details

Site Domain rsupartner.com
Base Domain rsupartner.com
Scan Status Ok
Last Scan2024-10-02T15:21:44+00:00
Next Scan 2024-11-01T15:21:44+00:00

Last Scan

Scanned2024-10-02T15:21:44+00:00
URL https://www.rsupartner.com/robots.txt
Redirect https://runsignup.com/robots.txt
Redirect Domain runsignup.com
Redirect Base runsignup.com
Domain IPs 3.210.12.204, 3.210.169.147
Redirect IPs 3.222.216.205, 34.194.61.140, 54.165.155.11
Response IP 54.165.155.11
Found Yes
Hash cfa58cef8b34a75d279d5fe3c74743fda12316d828d1fd1915bc719021095ef6
SimHash 6c1eef406edb

Groups

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /Race/NJ/Moorestown/InYourFaceScrapper
Disallow /race/nj/moorestown/inyourfacescrapper
Disallow /captcha
Disallow /em/
Disallow /Logout
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /Race/Photos/ViewPhoto/
Disallow /Race/Results/Simple/
Disallow /Race/Results/*/FinishersCert
Disallow /Race/Results/*/FinishersCertImg
Disallow /*?*PHPSESSID*
Disallow /*?*embedId2*
Disallow /*?*embedToken*
Disallow /*?*regToken*
Disallow /*?*RefCode=*
Disallow /*?*err=*
Disallow /*?*smsg=*
Disallow /*?*redirect=*
Disallow /*?*autoLogin*
Disallow /*?*remMeAttempt*

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://runsignup.com/sitemap.xml

Comments

  • Place wildcards last since some robots don't know how to parse them