Log In Sign Up

ow.ly
robots.txt

Robots Exclusion Standard data for ow.ly

Resource Scan

Scan Details

Site Domain ow.ly
Base Domain ow.ly
Scan Status Ok
Last Scan2024-09-16T02:46:03+00:00
Next Scan 2024-10-16T02:46:03+00:00

Last Scan

Scanned2024-09-16T02:46:03+00:00
URL https://ow.ly/robots.txt
Domain IPs 34.203.101.66, 34.235.31.221, 54.146.209.227
Response IP 54.146.209.227
Found Yes
Hash acfd3625b17915652c1fa28d3fddafd9df4bc5bb06ace4837092db8deb858ff4
SimHash b80d7930dd94

Groups

googlebot
*

Rule Path
Disallow /login
Disallow /auth
Disallow /logout
Disallow /ajax/*
Disallow /docs
Disallow /d
Back to top

Other Records

Field Value
sitemap http://ow.ly/sitemap.xml
Back to top

Comments

  • tells Google not to include these URLs in the index even if they have already been crawled
  • tells all engines not to crawl these URLs
Back to top

Warnings

  • `noindex` is not a known field.
Back to top