wagecalculator.net
robots.txt

Robots Exclusion Standard data for wagecalculator.net

Resource Scan

Scan Details

Site Domain wagecalculator.net
Base Domain wagecalculator.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-03-24T21:29:00+00:00
Next Scan 2025-06-22T21:29:00+00:00

Last Successful Scan

Scanned2024-02-29T21:08:59+00:00
URL https://wagecalculator.net/robots.txt
Domain IPs 104.21.56.80, 172.67.181.109, 2606:4700:3033::ac43:b56d, 2606:4700:3034::6815:3850
Response IP 104.21.56.80
Found Yes
Hash 88a435dd27950c450fd14388fbe26efbbf8cb01af699d896a59fcb28fc47de8c
SimHash 6555cc146764

Groups

*
bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookexternalhit/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookcatalog/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10