totalactivehub.com
robots.txt

Robots Exclusion Standard data for totalactivehub.com

Resource Scan

Scan Details

Site Domain totalactivehub.com
Base Domain totalactivehub.com
Scan Status Ok
Last Scan2025-10-08T22:47:23+00:00
Next Scan 2025-11-07T22:47:23+00:00

Last Scan

Scanned2025-10-08T22:47:23+00:00
URL https://totalactivehub.com/robots.txt
Redirect https://www.totalactivehub.com/robots.txt
Redirect Domain www.totalactivehub.com
Redirect Base totalactivehub.com
Domain IPs 13.33.88.114, 13.33.88.23, 13.33.88.72, 13.33.88.87
Redirect IPs 15.197.253.240, 3.33.249.164, 52.223.53.203, 99.83.185.157
Response IP 15.197.253.240
Found Yes
Hash 49bc73d6ff614f3c3e4f9e67e23895bc9dac917f3863b06911971460497542d2
SimHash 7c244160f620

Groups

*

Rule Path
Disallow /login$
Disallow /login/
Disallow /login?
Disallow /forgottenpassword$
Disallow /forgottenpassword/
Disallow /forgottenpassword?
Disallow /register$
Disallow /register/
Disallow /register?
Disallow /ssologin$
Disallow /ssologin/
Disallow /ssologin?

magpie-crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.totalactivehub.com/sitemap-20241206.xml