popmelody.ir
robots.txt

Robots Exclusion Standard data for popmelody.ir

Resource Scan

Scan Details

Site Domain popmelody.ir
Base Domain popmelody.ir
Scan Status Ok
Last Scan2024-09-09T12:45:09+00:00
Next Scan 2024-10-09T12:45:09+00:00

Last Scan

Scanned2024-09-09T12:45:09+00:00
URL https://popmelody.ir/robots.txt
Domain IPs 178.216.251.231
Response IP 178.216.251.231
Found Yes
Hash 8cffcb1ef02d9f516a320190ffa52bd5a523839c5651c4c9541f02c0ad2c0532
SimHash 6200c23167b4

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

crawler4j

Rule Path
Disallow /

wespe.de spider

Rule Path
Disallow /

cimri.com spider 1.x

Rule Path
Disallow /

pricebot

Rule Path
Disallow /

rytebot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

zumibot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

tineye-bot

Rule Path
Disallow /

intersearch.de spider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

exabot-thumbnails

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5