city.kanzaki.saga.jp
robots.txt

Robots Exclusion Standard data for city.kanzaki.saga.jp

Resource Scan

Scan Details

Site Domain city.kanzaki.saga.jp
Base Domain city.kanzaki.saga.jp
Scan Status Ok
Last Scan2024-11-13T15:33:51+00:00
Next Scan 2024-12-13T15:33:51+00:00

Last Scan

Scanned2024-11-13T15:33:51+00:00
URL https://www.city.kanzaki.saga.jp/robots.txt
Domain IPs 96.17.96.11, 96.17.96.31
Response IP 23.52.40.130
Found Yes
Hash 43d394fef3dc8d66c658ab74b3f5c1fa616dada535b6931d9d5c1082db56258b
SimHash 684cc010f458

Groups

*

Rule Path
Disallow /manage/
Disallow /rss/
Disallow /main.php*
Disallow /main/main/*
Disallow /otassha_map/manage/
Disallow /main/82.html*
Allow /main/82.html$

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mappy

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

slackbot

Rule Path
Disallow /

bbot

Rule Path
Disallow /

brands-bot

Rule Path
Disallow /

brands-bot-logo

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dnbcrawler-analytics

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

linespider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

thesis-research-bot

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

my-tiny-bot

Rule Path
Disallow /

bedrockbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

h-bot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

icc-crawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Warnings

  • 2 invalid lines.