Log In Sign Up

onestop-webshop.co.uk
robots.txt

Robots Exclusion Standard data for onestop-webshop.co.uk

Resource Scan

Scan Details

Site Domain onestop-webshop.co.uk
Base Domain onestop-webshop.co.uk
Scan Status Ok
Last Scan3/27/2025, 3:57:42 AM
Next Scan 4/26/2025, 3:57:42 AM

Last Scan

Scanned3/27/2025, 3:57:42 AM
URL https://onestop-webshop.co.uk/robots.txt
Domain IPs 62.233.107.64
Response IP 62.233.107.64
Found Yes
Hash 266fe1b55ef5d76f650cfe2476c88aa3addae71948dd936f15c343ad984242c0
SimHash 9a178e5b43d4

Groups

bingbot
googlebot
ia_archiver
msnbot
robozilla
slurp
dotbot
mj12bot
teoma

Product Comment
bingbot Bing
googlebot Google
ia_archiver Internet Archive "wayback" machine
msnbot Microsoft
robozilla DMOZ
slurp Yahoo!
dotbot Moz - https://moz.com/researchtools/ose/dotbot
mj12bot Majestic - https://majestic.com/support/faq
teoma Ask Jeeves
Rule Path Comment
Disallow /admin/ No Need To Crawl This
Disallow /blog/wp-admin/ No Need to Crawl This
Disallow /clients/ No Need To Crawl This
Disallow /gybcndlmx.php No need to Crawl This
Disallow /error_log.txt No Need To Crawl This

*

Rule Path
Disallow /
Back to top

Other Records

Field Value Comment
sitemap https://www.onestop-webshop.co.uk/sitemap.xml Advise bots where to locate the sitemap
Back to top

Comments

  • Onestop Webshop robots.txt file
  • Well-behaved robots understand "Allow" directives.
  • See http://www.robotstxt.org/db.html for full list
  • Well-behaved robots understand "Allow" directives.
  • Disallow all other robots
Back to top