parallax.com
robots.txt

Robots Exclusion Standard data for parallax.com

Resource Scan

Scan Details

Site Domain parallax.com
Base Domain parallax.com
Scan Status Ok
Last Scan2024-09-30T09:57:03+00:00
Next Scan 2024-10-30T09:57:03+00:00

Last Scan

Scanned2024-09-30T09:57:03+00:00
URL https://parallax.com/robots.txt
Redirect https://www.parallax.com/robots.txt
Redirect Domain www.parallax.com
Redirect Base parallax.com
Domain IPs 18.216.88.167, 3.141.87.22
Redirect IPs 18.216.88.167, 3.141.87.22
Response IP 18.216.88.167
Found Yes
Hash 750b0ca71690a2fd4a3a205d156a0b641d93c482840b8b1af63c6b08daa4765c
SimHash 6b0adaa0e1a3

Groups

bingbot

Rule Path
Disallow /download/
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 120

duckduckbot
googlebot
slurp

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.parallax.com/sitemap_index.xml

Comments

  • Web Crawler configuration