hjl.me
robots.txt

Robots Exclusion Standard data for hjl.me

Resource Scan

Scan Details

Site Domain hjl.me
Base Domain hjl.me
Scan Status Ok
Last Scan2025-12-14T05:09:53+00:00
Next Scan 2025-12-21T05:09:53+00:00

Last Scan

Scanned2025-12-14T05:09:53+00:00
URL https://hjl.me/robots.txt
Domain IPs 2a02:4780:84:84e8:d369:d7d2:72e1:7820, 2a02:4780:84:8eaf:28bb:2dfa:777d:828, 84.32.84.128, 84.32.84.183
Response IP 93.127.187.209
Found Yes
Hash a2f0d7783107a96014e9c2241e246f7a5d8debeb92880829301bd5753bfded34
SimHash 6f071873e331

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /cgi-bin/
Disallow /includes/
Disallow /tmp/
Disallow /private/
Disallow /config/
Disallow /backup/
Disallow /logs/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-json/
Disallow /xmlrpc.php
Disallow /readme.html
Disallow /license.txt
Disallow /wp-config.php
Disallow /error_log
Allow /pdf-to-image
Allow /images-converter
Allow /privacy-policy.html
Allow /terms-of-service.html
Allow /report-problem.html
Disallow /*?*
Disallow /*.php$
Disallow /*.js$
Disallow /*.css$
Allow /images/
Allow /css/
Allow /js/

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://hjl.me/sitemap.xml

Warnings

  • `host` is not a known field.