Log In Sign Up

pdf-awesome.com
robots.txt

Robots Exclusion Standard data for pdf-awesome.com

Resource Scan

Scan Details

Site Domain pdf-awesome.com
Base Domain pdf-awesome.com
Scan Status Ok
Last Scan2025-11-15T17:56:55+00:00
Next Scan 2025-12-15T17:56:55+00:00

Last Scan

Scanned2025-11-15T17:56:55+00:00
URL https://pdf-awesome.com/robots.txt
Domain IPs 13.33.45.119, 13.33.45.12, 13.33.45.4, 13.33.45.71, 2600:9000:229f:3000:6:a61b:c5c0:93a1, 2600:9000:229f:7a00:6:a61b:c5c0:93a1, 2600:9000:229f:7c00:6:a61b:c5c0:93a1, 2600:9000:229f:8800:6:a61b:c5c0:93a1, 2600:9000:229f:8a00:6:a61b:c5c0:93a1, 2600:9000:229f:8c00:6:a61b:c5c0:93a1, 2600:9000:229f:b600:6:a61b:c5c0:93a1, 2600:9000:229f:e00:6:a61b:c5c0:93a1
Response IP 13.33.45.119
Found Yes
Hash 9b19054d9fc31517a60bd272435e3e6eade987a42235e73cd38d1c8e9f26f721
SimHash 84000840cf15

Groups

*

Rule Path
Disallow /account
Disallow /archive
Disallow /history
Disallow /inbox
Allow /
Back to top

Other Records

Field Value
sitemap https://pdf-awesome.com/api/sitemap
Back to top

Comments

  • Specify the host
  • Sitemap location
  • Disallow crawlers from accessing user-specific pages
  • Allow everything else
Back to top

Warnings

  • `host` is not a known field.
Back to top