worldtravelserver.com
robots.txt

Robots Exclusion Standard data for worldtravelserver.com

Resource Scan

Scan Details

Site Domain worldtravelserver.com
Base Domain worldtravelserver.com
Scan Status Ok
Last Scan6/19/2025, 1:59:22 AM
Next Scan 6/26/2025, 1:59:22 AM

Last Scan

Scanned6/19/2025, 1:59:22 AM
URL https://worldtravelserver.com/robots.txt
Domain IPs 104.21.57.98, 172.67.162.237, 2606:4700:3031::6815:3962, 2606:4700:3032::ac43:a2ed
Response IP 104.21.57.98
Found Yes
Hash 851c467b28a1966de77f992defc01ce5f14ec5d09ee2f4122aefdc9def6b4a09
SimHash 53ad4672ba82

Groups

googlebot

Rule Path
Disallow /*?country=*
Disallow /*?country=*&city=*
Disallow /*?newtopic=*
Disallow /*?edit=*&textid=*
Disallow /*?edit=*&categid=*
Disallow /images/protoGallery.swf
Disallow /gallery.php
Disallow /swfscripts/infocity.php
Disallow /swfscripts/infocountry.php
Disallow /swfscripts/infofile.php
Disallow /world/
Disallow /popup/
Disallow /orderforms/
Disallow /orderform/
Disallow /orderbanner/
Disallow /orderbanner3/
Disallow /ordergallery/
Disallow /friend/
Disallow /addphoto/
Disallow /review/
Disallow /adv/
Disallow /travel/fr/

*

Rule Path
Disallow /world/
Disallow /popup/
Disallow /orderforms/
Disallow /orderform/
Disallow /orderbanner/
Disallow /orderbanner3/
Disallow /ordergallery/
Disallow /friend/
Disallow /addphoto/
Disallow /review/
Disallow /adv/

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

urlmetrics

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Warnings

  • 2 invalid lines.