pullrequest.com
robots.txt

Robots Exclusion Standard data for pullrequest.com

Resource Scan

Scan Details

Site Domain pullrequest.com
Base Domain pullrequest.com
Scan Status Ok
Last Scan2024-05-23T06:22:44+00:00
Next Scan 2024-06-22T06:22:44+00:00

Last Scan

Scanned2024-05-23T06:22:44+00:00
URL https://pullrequest.com/robots.txt
Redirect https://www.pullrequest.com/robots.txt
Redirect Domain www.pullrequest.com
Redirect Base pullrequest.com
Domain IPs 75.2.60.5
Redirect IPs 2406:da18:880:3802::c8, 2406:da18:b3d:e202::64, 46.137.195.11, 52.74.166.77
Response IP 13.215.144.61
Found Yes
Hash 9e531ea9a4609cd12c10822e5fc5b30dbe77304108251bc6aa3185b793a8009f
SimHash 1c719909cd96

Groups

*

Rule Path
Disallow /press/2018-04-13-techcrunch/
Disallow /about/investors/gradient/
Disallow /about/team/lyal-avery/
Disallow /about/investors/fika/
Disallow /about/team/tyler-mann/
Disallow /about/investors/ycombinator/
Disallow /about/team/dan-mateer/
Disallow /about/investors/lynett/
Disallow /about/investors/fundersclub/
Disallow /about/investors/liquid2/
Disallow /about/investors/slack/
Disallow /about/investors/defy/
Disallow /l/code-review/
Disallow /press/2018-06-06-templarbit/
Disallow /press/2018-05-24-austin-chamber/
Disallow /press/2018-05-22-venturebeat/
Disallow /press/2018-04-18-global-corporate-venturing/
Disallow /press/2018-04-16-axios/
Disallow /press/2018-04-16-fortune/
Disallow /press/2018-04-13-austin-inno/
Disallow /press/2018-04-13-austin-business-journal/
Disallow /press/2018-04-13-built-in-austin/
Disallow /press/2018-04-12-techvibes/
Disallow /press/2018-01-16-built-in-austin/
Disallow /press/2018-01-04-ft-reporter/
Disallow /press/2017-12-12-fortune/
Disallow /press/2017-12-11-austin-business-journal/
Disallow /press/2017-12-08-finsmes/
Disallow /press/2017-12-07-silicon-angle/
Disallow /press/2017-12-07-austin-inno/
Disallow /press/2017-12-07-techcrunch/
Disallow /press/2017-08-29-techvibes/
Disallow /press/2017-08-22-techcrunch/
Disallow /press/2017-08-21-techcrunch/
Disallow /press/2017-08-09-techcrunch/
Disallow /authors/
Disallow /product/
Disallow /reviewers/
Disallow /tags/

Other Records

Field Value
sitemap https://www.pullrequest.com/sitemap.xml

Comments

  • robotstxt.org - if ENV production variable is false robots will be disallowed.