foodforward.org
robots.txt

Robots Exclusion Standard data for foodforward.org

Resource Scan

Scan Details

Site Domain foodforward.org
Base Domain foodforward.org
Scan Status Ok
Last Scan2024-05-23T14:55:35+00:00
Next Scan 2024-06-22T14:55:35+00:00

Last Scan

Scanned2024-05-23T14:55:35+00:00
URL https://foodforward.org/robots.txt
Domain IPs 104.21.1.48, 172.67.128.136, 2606:4700:3036::ac43:8088, 2606:4700:3037::6815:130
Response IP 172.67.128.136
Found Yes
Hash ea385281c424f6035524d6aadc4b0db9f3169fc1a442da740c696bff2b2fa33a
SimHash 232edb32aaa6

Groups

*

Rule Path
Disallow /cms/wp-admin/
Disallow /wp-admin/
Allow /cms/wp-admin/admin-ajax.php
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 1

powermapper

Rule Path
Allow /

Comments

  • Default WordPress robots.txt
  • Plus a small crawl delay to mitigate risk the risk of a site
  • being deluged with multiple requests per second