isbn.nu
robots.txt

Robots Exclusion Standard data for isbn.nu

Resource Scan

Scan Details

Site Domain isbn.nu
Base Domain isbn.nu
Scan Status Ok
Last Scan2025-05-13T21:26:36+00:00
Next Scan 2025-05-20T21:26:36+00:00

Last Scan

Scanned2025-05-13T21:26:36+00:00
URL https://isbn.nu/robots.txt
Domain IPs 45.79.66.15
Response IP 45.79.66.15
Found Yes
Hash 9122b31abb535ca274f50a54f7dff83eff69ae5d0d064de63c14096c9a2cd466
SimHash f919b962e9d8

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /tmp/
Disallow /dont.index/
Disallow /temp/
Disallow /data/
Disallow /scripts/
Disallow /bin/

mozilla/5.0 (compatible;contxbot/1.0)

Rule Path
Disallow

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meta

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

Comments

  • everybody else is ok, but restrict directories