resocia.jp
robots.txt

Robots Exclusion Standard data for resocia.jp

Resource Scan

Scan Details

Site Domain resocia.jp
Base Domain resocia.jp
Scan Status Ok
Last Scan2024-05-24T15:07:50+00:00
Next Scan 2024-06-23T15:07:50+00:00

Last Scan

Scanned2024-05-24T15:07:50+00:00
URL https://resocia.jp/robots.txt
Redirect https://haken.resocia.jp/robots.txt
Redirect Domain haken.resocia.jp
Redirect Base resocia.jp
Domain IPs 52.199.156.210
Redirect IPs 3.114.134.104, 52.69.98.197
Response IP 52.69.98.197
Found Yes
Hash 180703a550e88d4d11f5e0f45a1be96f0f8669e5e86c7f8a1feedc7958d542ce
SimHash 7564dc75dd90

Groups

*

Rule Path
Disallow /keyword/
Disallow /job/search
Disallow /campaign/fin/
Disallow /lp/fin/

googlebot

Rule Path
Disallow

twitterbot

Rule Path
Disallow

y!j-brw

Rule Path
Disallow

yandexbot

Rule Path
Disallow /

yetibot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mozilla/5.0 ()

Rule Path
Disallow /

google-http-java-client/1.17.0-rc (gzip)

Rule Path
Disallow /

google-http-java-client

Rule Path
Disallow /

metauri api/2.0 +metauri.com

Rule Path
Disallow /

metauri

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

slurp

Rule Path
Disallow

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap http://resocia.jp/sitemap.xml
sitemap http://resocia.jp/joblist.xml