Log In Sign Up

betsyburke.net
robots.txt

Robots Exclusion Standard data for betsyburke.net

Resource Scan

Scan Details

Site Domain betsyburke.net
Base Domain betsyburke.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-12-26T20:10:03+00:00
Next Scan 2026-03-26T20:10:03+00:00

Last Successful Scan

Scanned2023-11-15T19:52:50+00:00
URL http://betsyburke.net/robots.txt
Domain IPs 199.34.233.6
Response IP 199.34.233.6
Found Yes
Hash 83c37895927d982edf5f7d6e5644b62c65e559cce59bdac0d3bf775e04816a2c
SimHash 9e5d9369d171

Groups

ahrefsbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow

googlebot

Rule Path
Disallow /*.aspx$
Disallow /index.asp?*
Disallow /?p=findagent.asp*
Disallow /?p=findoffice.asp*
Disallow /modules/internet/*
Disallow /modules/officegroup/*
Disallow /ThirdParty/
Disallow /clientdashboard/*
Disallow /?p=buildsavedsearch.asp*
Disallow /widgets/*

*

Rule Path
Disallow /modules/internet/logging/logwebhit.js
Disallow /modules/internet/logging/LogWebHit.aspx
Disallow /modules/internet/logging/LogAdHit.aspx
Disallow /modules/internet/logging/LogSearch.aspx
Disallow /modules/internet/logging/LogSearchResultsHistory.aspx
Disallow /modules/internet/logging/inc_loghit.asp
Disallow /modules/global/LogWebHit.aspx
Disallow /*?s=
Disallow /*?w=
Disallow /*sortbyid%3D
Disallow /*searchID%3D
Disallow /*searchid%3D
Disallow /*results%3D
Disallow /*selected%3D
Disallow /*sub%3D
Disallow /*pso%3D
Disallow /*referrer%3D
Disallow /home-details/*
Disallow /Listings/*

*

Rule Path
Disallow /389ccec4-85c4-4aa2-86f3-916bc9e768ec/admin
Back to top

Other Records

Field Value
sitemap https://betsyburke.net/sitemap.xml
Back to top

Comments

  • Backlink Analysis
  • Yandex
  • Google AdBot
  • Plead with robots/scrapers to not cause page views that Agents would see in their analytics.
  • Per Corby, if any bot hits this URL they are abusing robots.txt and should be blacklisted
Back to top