italianseduction.club
robots.txt

Robots Exclusion Standard data for italianseduction.club

Resource Scan

Scan Details

Site Domain italianseduction.club
Base Domain italianseduction.club
Scan Status Ok
Last Scan2024-06-14T09:01:02+00:00
Next Scan 2024-06-21T09:01:02+00:00

Last Scan

Scanned2024-06-14T09:01:02+00:00
URL https://italianseduction.club/robots.txt
Domain IPs 104.26.0.45, 104.26.1.45, 172.67.75.157, 2606:4700:20::681a:12d, 2606:4700:20::681a:2d, 2606:4700:20::ac43:4b9d
Response IP 104.26.0.45
Found Yes
Hash 1b39bdb57354c8a18ad7936ce7894382b6ce63672d9b1fb057f3598772ebbd4c
SimHash d14009a6c100

Groups

*

Rule Path
Disallow /forum/startTopic/
Disallow /forum/*?do=add
Disallow /forum/*?do=submit
Disallow /forum/discover/unread/
Disallow /forum/markallread/
Disallow /forum/staff/
Disallow /forum/online/
Disallow /forum/discover/
Disallow /forum/leaderboard/
Disallow /forum/search/
Disallow /forum/*?advancedSearchForm=
Disallow /forum/register/
Disallow /forum/lostpassword/
Disallow /forum/login/
Disallow /wp-content/plugins/
Disallow /wp-admin/
Disallow /readme.html
Disallow /refer/
Disallow /forum/*?sortby=
Disallow /forum/*?filter=
Disallow /forum/*?tab=comments
Disallow /forum/*?do=email
Disallow /forum/*?do=findComment
Disallow /forum/*?do=getLastComment
Disallow /forum/*?do=getNewComment
Disallow /forum/profile/

Other Records

Field Value
sitemap https://www.italianseduction.club/forum/sitemap.php

Comments

  • Rules for Invision Community (https://invisioncommunity.com)
  • Block pages with no unique content
  • Block faceted pages and 301 redirect pages
  • Block profile pages as these have little unique value, consume a lot of crawl time and contain hundreds of 301 links
  • Sitemap URL