wwwserver1.com
robots.txt

Robots Exclusion Standard data for wwwserver1.com

Resource Scan

Scan Details

Site Domain wwwserver1.com
Base Domain wwwserver1.com
Scan Status Ok
Last Scan2026-02-13T14:45:40+00:00
Next Scan 2026-03-15T14:45:40+00:00

Last Scan

Scanned2026-02-13T14:45:40+00:00
URL https://www.wwwserver1.com/robots.txt
Domain IPs 172.66.44.156, 172.66.47.100, 2606:4700:310c::ac42:2c9c, 2606:4700:310c::ac42:2f64
Response IP 172.66.47.100
Found Yes
Hash 2edf49ffea22f6adf727815594d1ba9f47f934e906719cbab3ba8b8996b4bf58
SimHash 602f55f4de1d

Groups

semrushbot

Rule Path
Disallow /*

mj12bot

Rule Path
Disallow /*

ahrefsbot

Rule Path
Disallow /*

blexbot

Rule Path
Disallow /*

baiduspider

Rule Path
Disallow /*

gnowitnewsbot

Rule Path
Disallow /*

petalbot

Rule Path
Disallow /*

aspiegelbot

Rule Path
Disallow /*

mauibot

Rule Path
Disallow /*

wwwserver1.com_monitor

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

*

Rule Path
Allow /index.php
Disallow /*.php
Disallow /cgi-bin/
Disallow /kaka/
Disallow /wp-admin/
Disallow /wp-json/
Disallow /wp-content/cache/supercache/

Other Records

Field Value
sitemap https://wwwserver1.com/sitemap.xml