Log In Sign Up

calciomatome.net
robots.txt

Robots Exclusion Standard data for calciomatome.net

Resource Scan

Scan Details

Site Domain calciomatome.net
Base Domain calciomatome.net
Scan Status Ok
Last Scan2024-06-15T02:47:28+00:00
Next Scan 2024-06-22T02:47:28+00:00

Last Scan

Scanned2024-06-15T02:47:28+00:00
URL https://calciomatome.net/robots.txt
Redirect http://www.calciomatome.net/robots.txt
Redirect Domain www.calciomatome.net
Redirect Base calciomatome.net
Domain IPs 104.21.61.83, 172.67.207.178, 2606:4700:3030::6815:3d53, 2606:4700:3033::ac43:cfb2
Redirect IPs 104.21.61.83, 172.67.207.178, 2606:4700:3030::6815:3d53, 2606:4700:3033::ac43:cfb2
Response IP 104.21.61.83
Found Yes
Hash 7298f69e9189adb5822d64c127bf0847961196226509e7738bb07368b75285f4
SimHash 234d58c0e178

Groups

*

Rule Path
Disallow /tag/
Disallow /upload/

bingbot

Rule Path
Disallow /archives/
Disallow /upload/
Disallow /tag/
Disallow /comment
Disallow /list_comments/
Disallow /post_comment/

Other Records

Field Value
crawl-delay 60

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

gptbot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

criteobot/0.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

y!j-asr/0.1 crawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

seokicks-robot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

hypefactors

Rule Path
Disallow /

linespider

Rule Path
Disallow /archives/
Disallow /upload/
Disallow /comment

googlebot

Rule Path
Disallow /archives/
Disallow /upload/
Disallow /tag/
Disallow /comment
Disallow /list_comments/
Disallow /post_comment/
Back to top