Log In Sign Up

baxtel.com
robots.txt

Robots Exclusion Standard data for baxtel.com

Resource Scan

Scan Details

Site Domain baxtel.com
Base Domain baxtel.com
Scan Status Ok
Last Scan2024-11-03T22:19:50+00:00
Next Scan 2024-12-03T22:19:50+00:00

Last Scan

Scanned2024-11-03T22:19:50+00:00
URL https://baxtel.com/robots.txt
Domain IPs 216.24.57.1
Response IP 216.24.57.1
Found Yes
Hash 2d9eb050befe4f1a56b6e2499d05cba58a9e3c4c40c35bee3eb8dbb6f25c5209
SimHash 6a8eca8577c4

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

googlebot-video

Rule Path
Disallow

googlebot-smartphone

Rule Path
Disallow

google-inspectiontool

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow

bingbot

Rule Path
Disallow

duckduckbot

Rule Path
Disallow

slurp

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

*

Rule Path
Disallow /
Allow /$
Allow /news
Allow /about-us
Allow /contacts/new
Allow /events
Allow /advertise
Allow /quotes/new

*

Rule Path
Disallow /map/temp18707
Back to top

Other Records

Field Value
sitemap https://baxtel.com/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
  • Allow Googlebot full access
  • Allow Googlebot-Image full access
  • Allow Googlebot-News full access
  • Allow Googlebot-Video full access
  • Allow Googlebot-Smartphone full access
  • Allow Google-InspectionTool full access
  • Allow AhrefsBot full access
  • Allow Bingbot full access
  • Allow DuckDuckBot full access
  • Allow Yahoo full access
  • Allow Web Archive full access
  • Block all other bots
  • In-Development Map
Back to top