thuvienanime.com
robots.txt

Robots Exclusion Standard data for thuvienanime.com

Resource Scan

Scan Details

Site Domain thuvienanime.com
Base Domain thuvienanime.com
Scan Status Ok
Last Scan2024-11-12T13:41:26+00:00
Next Scan 2024-11-19T13:41:26+00:00

Last Scan

Scanned2024-11-12T13:41:26+00:00
URL https://thuvienanime.com/robots.txt
Domain IPs 172.93.219.72
Response IP 172.93.219.72
Found Yes
Hash 2176aec19a0ced093275700d51a3a0a2c08e7672e64caab65402a2ea7bfe4a91
SimHash 403c0f52ac13

Groups

*

Rule Path
Disallow /feed/$
Disallow /search/
Disallow /?s=
Disallow /?__hstc=
Disallow /p%3D*
Disallow /wp-admin/
Disallow /component/*
Disallow /wp-login.php*
Disallow /cdn-cgi/
Disallow /?author=
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

mj12bot
rogerbot
backlinkcrawler
semrushbot
ahrefsbot
seomoz
rogerbot
dotbot
semrush
majestic12
majestic-seo
claudebot
google-extended
gptbot
ccbot
anthropic-ai
chatgpt-user
perplexitybot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://thuvienanime.com/sitemap_index.xml