spainbuddy.com
robots.txt

Robots Exclusion Standard data for spainbuddy.com

Resource Scan

Scan Details

Site Domain spainbuddy.com
Base Domain spainbuddy.com
Scan Status Ok
Last Scan2024-09-21T11:38:40+00:00
Next Scan 2024-09-28T11:38:40+00:00

Last Scan

Scanned2024-09-21T11:38:40+00:00
URL https://spainbuddy.com/robots.txt
Domain IPs 35.214.151.129
Response IP 35.214.151.129
Found Yes
Hash 51499f7c8ef5cb1e6b4ccda0753e1aaae90fe19ed3f8708d7634ce330268c364
SimHash 4511f541c6e8

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /classifieds/
Disallow /property/property/*
Disallow /property/*
Disallow /events/*
Disallow /business-directory/*/feed
Disallow /?attachment_id=*
Disallow /property/FCKeditor/userimages/*
Disallow /facebook_event_*
Disallow /regions-of-spain/
Disallow /regions-of-spain/*

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Warnings

  • 1 invalid line.