texwiller.org
robots.txt

Robots Exclusion Standard data for texwiller.org

Resource Scan

Scan Details

Site Domain texwiller.org
Base Domain texwiller.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-14T05:22:34+00:00
Next Scan 2024-11-13T05:22:34+00:00

Last Successful Scan

Scanned2024-07-23T02:50:35+00:00
URL http://texwiller.org/robots.txt
Redirect http://tex-willer.over-blog.com/robots.txt
Redirect Domain tex-willer.over-blog.com
Redirect Base over-blog.com
Domain IPs 217.70.184.38
Redirect IPs 185.128.239.39, 185.128.239.66
Response IP 185.128.239.66
Found Yes
Hash 2412162417c46b916de73ba0c5e072765c7a40c764c7a845057d547a5cd5f686
SimHash 6b00d455c733

Groups

*

Rule Path
Allow /
Disallow /contact
Disallow /mail/subscribe
Disallow /mail/valid-*

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

spbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap http://tex-willer.over-blog.com/sitemap-news.xml
sitemap http://tex-willer.over-blog.com/sitemap.xml