Log In Sign Up

carmudi.co.id
robots.txt

Robots Exclusion Standard data for carmudi.co.id

Resource Scan

Scan Details

Site Domain carmudi.co.id
Base Domain carmudi.co.id
Scan Status Ok
Last Scan2024-11-09T09:01:47+00:00
Next Scan 2024-11-16T09:01:47+00:00

Last Scan

Scanned2024-11-09T09:01:47+00:00
URL https://carmudi.co.id/robots.txt
Redirect https://www.carmudi.co.id/robots.txt
Redirect Domain www.carmudi.co.id
Redirect Base carmudi.co.id
Domain IPs 104.21.55.5, 172.67.170.5, 2606:4700:3032::6815:3705, 2606:4700:3033::ac43:aa05
Redirect IPs 104.21.55.5, 172.67.170.5, 2606:4700:3032::6815:3705, 2606:4700:3033::ac43:aa05
Response IP 172.67.170.5
Found Yes
Hash 8a042bd99650cd2b74779021d440d9d34d72dd6046ca476966540f6b2e87fa06
SimHash 6b14d8d5e391

Groups

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

serpstatbot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

teracent-feed-processing

Rule Path
Disallow /

*

Rule Path
Disallow /ajax/
Back to top

Other Records

Field Value
sitemap https://www.carmudi.co.id/sitemap.xml
Back to top