modelfol.io
robots.txt

Robots Exclusion Standard data for modelfol.io

Resource Scan

Scan Details

Site Domain modelfol.io
Base Domain modelfol.io
Scan Status Ok
Last Scan2024-06-24T12:15:38+00:00
Next Scan 2024-07-01T12:15:38+00:00

Last Scan

Scanned2024-06-24T12:15:38+00:00
URL https://modelfol.io/robots.txt
Domain IPs 104.26.14.145, 104.26.15.145, 172.67.71.186, 2606:4700:20::681a:e91, 2606:4700:20::681a:f91, 2606:4700:20::ac43:47ba
Response IP 104.26.15.145
Found Yes
Hash 0c71c82c903d5c8fec4b77db522eae61d714c58dcfca07e9220c9c5cff2c8b77
SimHash ce190460f730

Groups

*

Rule Path
Disallow /advert.php
Disallow /ajax/ViewedPhotoLoad.php
Disallow /admin.php
Disallow /ControlPanel.php
Disallow /NewControlPanel.php
Disallow /worksafe.php
Disallow /ViewerSwitch.php
Disallow /LazyLoad.php
Disallow /hideMedia.php
Disallow /verification_images/
Disallow /ajax/memberHeaderLoad.php

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

barkrowler

Rule Path
Disallow /

Comments

  • https://megaindex.com/crawler