thelist.group
robots.txt

Robots Exclusion Standard data for thelist.group

Resource Scan

Scan Details

Site Domain thelist.group
Base Domain thelist.group
Scan Status Ok
Last Scan2025-10-26T19:54:42+00:00
Next Scan 2025-11-02T19:54:42+00:00

Last Scan

Scanned2025-10-26T19:54:42+00:00
URL https://thelist.group/robots.txt
Domain IPs 104.21.82.243, 172.67.209.138, 2606:4700:3033::ac43:d18a, 2606:4700:3035::6815:52f3
Response IP 172.67.209.138
Found Yes
Hash 286af52c3adfec3d147bcd0f6a5635cbfb43e529485ad495a5b45d479e322bbb
SimHash 6115cb02d793

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login/
Disallow /*?comments
Disallow /script-comments.js
Disallow /wp-comments-post.php
Disallow /wp-content/plugins/
Disallow /cgi-bin/
Disallow /cdn-cgi/
Allow /wp-admin/admin-ajax.php

googlebot-image

Rule Path
Allow /

mediapartners-google*

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

ia_archiver

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /