Log In Sign Up

next-tech.com
robots.txt

Robots Exclusion Standard data for next-tech.com

Resource Scan

Scan Details

Site Domain next-tech.com
Base Domain next-tech.com
Scan Status Ok
Last Scan2025-12-21T06:13:26+00:00
Next Scan 2026-01-20T06:13:26+00:00

Last Scan

Scanned2025-12-21T06:13:26+00:00
URL https://next-tech.com/robots.txt
Domain IPs 162.159.134.42
Response IP 162.159.134.42
Found Yes
Hash b1f4bf4c43c5e1006276e7dee67214fe23c83667a2fa5a0bb6174ef33a5d1b3a
SimHash 704c3952a6a5

Groups

gptbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

claude-web

Rule Path
Allow /

claude-user

Rule Path
Allow /

claude-searchbot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

google-extended

Rule Path
Allow /

deepseekbot

Rule Path
Allow /

facebookbot

Rule Path
Allow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

peer39_crawler/1.0

Rule Path
Disallow /

*

Rule Path
Disallow */?*
Disallow */?s=*
Disallow */?post_type=*
Disallow */?__hstc*
Disallow */feed/
Disallow */author/
Allow /wp-admin/admin-ajax.php
Back to top

Other Records

Field Value
sitemap https://next-tech.com/sitemap_index.xml
Back to top

Warnings

  • 3 invalid lines.
Back to top