city.kanzaki.saga.jp
robots.txt

Robots Exclusion Standard data for city.kanzaki.saga.jp

Resource Scan

Scan Details

Site Domain city.kanzaki.saga.jp
Base Domain city.kanzaki.saga.jp
Scan Status Ok
Last Scan2024-09-14T15:32:52+00:00
Next Scan 2024-10-14T15:32:52+00:00

Last Scan

Scanned2024-09-14T15:32:52+00:00
URL https://www.city.kanzaki.saga.jp/robots.txt
Domain IPs 96.17.96.11, 96.17.96.31
Response IP 23.44.5.17
Found Yes
Hash 5c24553c713be597b6f0318b0c0c16d4b6a7aab2b8fd01f2f3d83fecda792424
SimHash 684ec018f438

Groups

*

Rule Path
Disallow /manage/
Disallow /rss/
Disallow /main.php*
Disallow /main/main/*
Disallow /otassha_map/manage/
Disallow /main/82.html*
Allow /main/82.html$

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mappy

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

slackbot

Rule Path
Disallow /

bbot

Rule Path
Disallow /

brands-bot

Rule Path
Disallow /

brands-bot-logo

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dnbcrawler-analytics

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

linespider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

thesis-research-bot

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

my-tiny-bot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Warnings

  • 2 invalid lines.