Log In Sign Up

wecasa.fr
robots.txt

Robots Exclusion Standard data for wecasa.fr

Resource Scan

Scan Details

Site Domain wecasa.fr
Base Domain wecasa.fr
Scan Status Ok
Last Scan2024-11-16T12:32:12+00:00
Next Scan 2024-11-30T12:32:12+00:00

Last Scan

Scanned2024-11-16T12:32:12+00:00
URL https://wecasa.fr/robots.txt
Redirect https://www.wecasa.fr/robots.txt
Redirect Domain www.wecasa.fr
Redirect Base wecasa.fr
Domain IPs 104.26.0.102, 104.26.1.102, 172.67.74.187, 2606:4700:20::681a:166, 2606:4700:20::681a:66, 2606:4700:20::ac43:4abb
Redirect IPs 104.26.0.102, 104.26.1.102, 172.67.74.187, 2606:4700:20::681a:166, 2606:4700:20::681a:66, 2606:4700:20::ac43:4abb
Response IP 104.26.1.102
Found Yes
Hash 9e61b5d548e3fdb9551f711b92c2f53057e7d90ba28d49eee1f0815c4a835efd
SimHash 9a949d85f440

Groups

facebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookexternalhit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

indeedbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.wecasa.fr/sitemaps/fr/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
  • Facebook bot crawlers
  • Facebook bot crawlers
  • Indeed bot crawlers
  • Commercial SEO Professional Crawler
  • Commercial SEO Professional Crawler
Back to top