jobcolo.com
robots.txt

Robots Exclusion Standard data for jobcolo.com

Resource Scan

Scan Details

Site Domain jobcolo.com
Base Domain jobcolo.com
Scan Status Ok
Last Scan2024-09-03T00:22:31+00:00
Next Scan 2024-10-03T00:22:31+00:00

Last Scan

Scanned2024-09-03T00:22:31+00:00
URL https://jobcolo.com/robots.txt
Domain IPs 13.33.88.119, 13.33.88.2, 13.33.88.54, 13.33.88.7, 2600:9000:223b:2c00:9:8d4b:2b80:93a1, 2600:9000:223b:3200:9:8d4b:2b80:93a1, 2600:9000:223b:3800:9:8d4b:2b80:93a1, 2600:9000:223b:6800:9:8d4b:2b80:93a1, 2600:9000:223b:8200:9:8d4b:2b80:93a1, 2600:9000:223b:ae00:9:8d4b:2b80:93a1, 2600:9000:223b:c800:9:8d4b:2b80:93a1, 2600:9000:223b:fc00:9:8d4b:2b80:93a1
Response IP 13.33.88.54
Found Yes
Hash f924bd3d34328a5f487596ff30cd6f2ac2b94243aa3f7c611797134093766a0c
SimHash c154cf509992

Groups

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

adsbot-google
adsbot-google-mobile

Rule Path
Disallow /*historykeep*
Disallow /*/jobs/page$
Disallow /*cookierewrite
Disallow /*trafficcount
Disallow /*userhistory
Disallow /*utm-ajax
Disallow /*?t_a=*&t_e=*
Disallow /*?osa=*&tsa=*

*

Rule Path
Disallow /*historykeep*
Disallow /*/jobs/page$
Disallow /*cookierewrite
Disallow /*trafficcount
Disallow /*userhistory
Disallow /*utm-ajax
Disallow /*?t_a=*&t_e=*
Disallow /*?osa=*&tsa=*

Other Records

Field Value
sitemap https://jobcolo.com/sitemap-xml/sitemap.xml

Comments

  • グローバルユーザエージェント(*)は無視される
  • https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers?hl=ja