freight.cargo.site
robots.txt

Robots Exclusion Standard data for freight.cargo.site

Resource Scan

Scan Details

Site Domain freight.cargo.site
Base Domain cargo.site
Scan Status Ok
Last Scan2024-09-18T21:26:26+00:00
Next Scan 2024-10-18T21:26:26+00:00

Last Scan

Scanned2024-09-18T21:26:26+00:00
URL https://freight.cargo.site/robots.txt
Domain IPs 108.156.133.19, 108.156.133.32, 108.156.133.38, 108.156.133.78, 2600:9000:2755:5e00:1d:a440:23c0:93a1, 2600:9000:2755:6400:1d:a440:23c0:93a1, 2600:9000:2755:6800:1d:a440:23c0:93a1, 2600:9000:2755:6a00:1d:a440:23c0:93a1, 2600:9000:2755:9800:1d:a440:23c0:93a1, 2600:9000:2755:c400:1d:a440:23c0:93a1, 2600:9000:2755:d000:1d:a440:23c0:93a1, 2600:9000:2755:da00:1d:a440:23c0:93a1
Response IP 108.156.133.38
Found Yes
Hash 955883bea0ebf31dd6475053374505fa27e0453a9540bcdc9e0ede1bd48470c5
SimHash 4510c0d24335

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3