nylons-pantyhose.com
robots.txt

Robots Exclusion Standard data for nylons-pantyhose.com

Resource Scan

Scan Details

Site Domain nylons-pantyhose.com
Base Domain nylons-pantyhose.com
Scan Status Ok
Last Scan2025-12-24T00:53:17+00:00
Next Scan 2026-01-23T00:53:17+00:00

Last Scan

Scanned2025-12-24T00:53:17+00:00
URL https://nylons-pantyhose.com/robots.txt
Domain IPs 104.21.53.147, 172.67.213.140, 2606:4700:3037::6815:3593, 2606:4700:3037::ac43:d58c
Response IP 172.67.213.140
Found Yes
Hash d9f8314c80c196c4ceb15685da1f3722decc879b173a7ceb82e2c2e0ca87b7e1
SimHash 6650ce22d505

Groups

amazonbot

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

applebot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

bingbot

Rule Path
Allow /

facebot

Rule Path
Allow /

googlebot

Rule Path
Allow /

msnbot

Rule Path
Allow /

naverbot

Rule Path
Allow /

seznambot

Rule Path
Allow /

slurp

Rule Path
Allow /

teoma

Rule Path
Allow /

claudebot

Rule Path
Allow /

google-extended

Rule Path
Allow /

gptbot

Rule Path
Allow /

petalbot

Rule Path
Allow /

uptimerobot

Rule Path
Allow /

viberbot

Rule Path
Allow /

yak

Rule Path
Allow /

twitterbot

Rule Path
Allow /

yeti

Rule Path
Allow /

yandex

Rule Path
Allow /

*

Rule Path
Disallow /
Disallow /register.html
Disallow /admin.php
Disallow /birthdays/
Disallow /find-new/
Disallow /forums/*/?direction
Disallow /forums/*/?order
Disallow /forums/*/page-*?order
Disallow /help/
Disallow /login/
Disallow /lost-password/
Disallow /misc/
Disallow /online/
Disallow /search/
Disallow /search.php
Disallow /threads/*/poll
Disallow /WhoHasVisited/
Disallow /recent-activity/
Disallow /advstats/
Disallow /BanList/
Disallow /goto/
Disallow /conversations/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://nylons-pantyhose.com/sitemap.xml

Warnings

  • `host` is not a known field.