pbi.ai
robots.txt

Robots Exclusion Standard data for pbi.ai

Resource Scan

Scan Details

Site Domain pbi.ai
Base Domain pbi.ai
Scan Status Ok
Last Scan2025-10-13T21:18:35+00:00
Next Scan 2025-11-12T21:18:35+00:00

Last Scan

Scanned2025-10-13T21:18:35+00:00
URL https://pbi.ai/robots.txt
Redirect https://www.pbi.ai/robots.txt
Redirect Domain www.pbi.ai
Redirect Base pbi.ai
Domain IPs 2a00:4b40:aaaa:2005::7, 37.9.175.165
Redirect IPs 2a00:4b40:aaaa:2005::7, 37.9.175.165
Response IP 37.9.175.165
Found Yes
Hash a33e87cd54019fd663741cd5162d90b74a310ba327df3b7da24d43312a1fb447
SimHash 6d0c1c46c393

Groups

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap http://pbi.ai/sitemap.xml

Comments

  • Request-rate: 18/1m 2100-0529
  • Request-rate: 6/1m 0530-2059