makassar.terkini.id
robots.txt

Robots Exclusion Standard data for makassar.terkini.id

Resource Scan

Scan Details

Site Domain makassar.terkini.id
Base Domain terkini.id
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-28T21:09:41+00:00
Next Scan 2024-11-27T21:09:41+00:00

Last Successful Scan

Scanned2024-09-06T20:47:56+00:00
URL https://makassar.terkini.id/robots.txt
Domain IPs 104.26.10.178, 104.26.11.178, 172.67.73.155, 2606:4700:20::681a:ab2, 2606:4700:20::681a:bb2, 2606:4700:20::ac43:499b
Response IP 104.26.11.178
Found Yes
Hash 195607b4058744059002ba021d291e758d5e6124df9f7731a16c015881c78ea3
SimHash ce03ea309cb1

Groups

*

Rule Path
Disallow /search$
Disallow /search?*
Disallow /wp-admin/
Disallow */komentar$
Disallow */komentar?*
Disallow */komentar/
Allow /wp-admin/admin-ajax.php

chatgpt-user

Rule Path
Disallow /

openai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

nuclei

Rule Path
Disallow /

wikido

Rule Path
Disallow /

riddler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

node/simplecrawler

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://makassar.terkini.id/sitemap.xml