xenoconcept.com
robots.txt

Robots Exclusion Standard data for xenoconcept.com

Resource Scan

Scan Details

Site Domain xenoconcept.com
Base Domain xenoconcept.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-11-05T01:13:25+00:00
Next Scan 2026-02-03T01:13:25+00:00

Last Successful Scan

Scanned2025-07-07T18:04:00+00:00
URL https://xenoconcept.com/robots.txt
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.112.1
Found Yes
Hash a69a26047eb03e31af63cf6fe35eda5268389491ac386bc901a09fc816bfe4dd
SimHash d82bc9620e80

Groups

*

Rule Path
Disallow /*add-to-cart%3D*
Disallow /wp-admin/
Disallow /*warenkorb*
Disallow /agb
Disallow /kasse
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /wp-content/themes
Disallow /feed/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 120

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

proximic

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

sitelockspider

Rule Path
Disallow /

zgrab

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

turnitin

Rule Path
Disallow /