forums.internationalskeptics.com
robots.txt

Robots Exclusion Standard data for forums.internationalskeptics.com

Resource Scan

Scan Details

Site Domain forums.internationalskeptics.com
Base Domain internationalskeptics.com
Scan Status Ok
Last Scan2024-09-18T03:16:53+00:00
Next Scan 2024-10-18T03:16:53+00:00

Last Scan

Scanned2024-09-18T03:16:53+00:00
URL http://forums.internationalskeptics.com/robots.txt
Redirect https://www.internationalskeptics.com/forums/robots.txt
Redirect Domain www.internationalskeptics.com
Redirect Base internationalskeptics.com
Domain IPs 104.131.54.4
Redirect IPs 104.131.54.4
Response IP 104.131.54.4
Found Yes
Hash b13cbdea0ce863e6ceebeaba101154612a75d8f93d3b141d53e2e61ec4243c3c
SimHash 22881b4186d5

Groups

www.integromedb.org/crawler

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

umbot-fc

Rule Path
Disallow /

wesee

Rule Path
Disallow /

newslebot

Rule Path
Disallow /

flr-bot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

*

Rule Path
Disallow /*.php$
Disallow /admincp/
Disallow /clientscript/
Disallow /customavatars/
Disallow /customprofilepics/
Disallow /designs/
Disallow /modcp/
Disallow /vbseo_sitemap/
Disallow /includes/
Disallow /archive/archive.css
Disallow /archive/global.php
Disallow /cpstyles/
Disallow /images/
Disallow /signaturepics/
Disallow /ajax.php
Disallow /album.php
Disallow /calender.php
Disallow /clear.gif
Disallow /converse.php
Disallow /cron.php
Disallow /editpost.php
Disallow /external.php
Disallow /faq.php
Disallow /global.php
Disallow /group_inlinemod.php
Disallow /image.php
Disallow /infraction.php
Disallow /inlinemod.php
Disallow /joinrequests.php
Disallow /login.php
Disallow /member.php
Disallow /member_inlinemod.php
Disallow /memberlist.php
Disallow /misc.php
Disallow /moderation.php
Disallow /moderator.php
Disallow /newattachment.php
Disallow /newreply.php
Disallow /newthread.php
Disallow /online.php
Disallow /payment_gateway.php
Disallow /payments.php
Disallow /picture.php
Disallow /picture_inlinemod.php
Disallow /picturecomment.php
Disallow /poll.php
Disallow /posthistory.php
Disallow /postings.php
Disallow /printthread.php
Disallow /private.php
Disallow /profile.php
Disallow /register.php
Disallow /report.php
Disallow /reputation.php
Disallow /search.php
Disallow /sendmessage.php
Disallow /showgroups.php
Disallow /subscription.php
Disallow /tags.php
Disallow /threadrate.php
Disallow /threadtag.php
Disallow /usercp.php
Disallow /usernote.php
Disallow /visitormessage.php

Other Records

Field Value
crawl-delay 30

Comments

  • User-agent: Googlebot