Log In Sign Up

egling.defirmenkataloge.com
robots.txt

Robots Exclusion Standard data for egling.defirmenkataloge.com

Resource Scan

Scan Details

Site Domain egling.defirmenkataloge.com
Base Domain defirmenkataloge.com
Scan Status Ok
Last Scan2025-10-18T05:59:06+00:00
Next Scan 2025-11-17T05:59:06+00:00

Last Scan

Scanned2025-10-18T05:59:06+00:00
URL https://egling.defirmenkataloge.com/robots.txt
Domain IPs 104.21.29.62, 172.67.148.131, 2606:4700:3032::ac43:9483, 2606:4700:3036::6815:1d3e
Response IP 172.67.148.131
Found Yes
Hash d4feb45d475d9d7b7908a3bd8edde44d943a550d0975b6cc7dd5c448f307a18b
SimHash 6934ba3263b3

Groups

*

Rule Path
Disallow /add/
Disallow /admin/
Disallow /reviews/add/
Disallow /company/edit/

Other Records

Field Value
crawl-delay 1

gptbot

Rule Path
Disallow /add/
Disallow /admin/
Disallow /reviews/add/
Disallow /company/edit/

Other Records

Field Value
crawl-delay 1

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /
Back to top

Comments

  • www.robotstxt.org/
  • Allow crawling of all content
Back to top