soapsspoilers.com
robots.txt

Robots Exclusion Standard data for soapsspoilers.com

Resource Scan

Scan Details

Site Domain soapsspoilers.com
Base Domain soapsspoilers.com
Scan Status Ok
Last Scan2024-09-19T23:53:26+00:00
Next Scan 2024-09-26T23:53:26+00:00

Last Scan

Scanned2024-09-19T23:53:26+00:00
URL https://soapsspoilers.com/robots.txt
Domain IPs 104.26.10.32, 104.26.11.32, 172.67.72.104, 2606:4700:20::681a:a20, 2606:4700:20::681a:b20, 2606:4700:20::ac43:4868
Response IP 172.67.72.104
Found Yes
Hash ec112490d00cac3150996475dcd3a694d0425f537f94b0f3465f0a195776c4d8
SimHash 7010d943a4a6

Groups

*

Rule Path
Disallow /?s=
Disallow /page/*/?s=
Disallow /search/
Disallow /wp-json/
Disallow /?rest_route=
Disallow /wp-admin/
Disallow /trackback/
Disallow /xmlrpc.php
Disallow /feed/
Allow /wp-admin/admin-ajax.php

amazonbot
anthropic-ai
applebot-extended
bytespider
ccbot
chatgpt-user
claudebot
claude-web
cohere-ai
diffbot
facebookbot
friendlycrawler
google-extended
gptbot
img2dataset
omgili
omgilibot
peer39_crawler
peer39_crawler/1.0
perplexitybot
youbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://yandrrecaps.com/sitemap_index.xml