tonga-soa.com
robots.txt

Robots Exclusion Standard data for tonga-soa.com

Resource Scan

Scan Details

Site Domain tonga-soa.com
Base Domain tonga-soa.com
Scan Status Ok
Last Scan4/20/2025, 5:36:27 AM
Next Scan 4/27/2025, 5:36:27 AM

Last Scan

Scanned4/20/2025, 5:36:27 AM
URL https://tonga-soa.com/robots.txt
Redirect https://www.tonga-soa.com/robots.txt
Redirect Domain www.tonga-soa.com
Redirect Base tonga-soa.com
Domain IPs 46.105.204.2
Redirect IPs 46.105.204.2
Response IP 46.105.204.2
Found Yes
Hash f78c8041f1a6f97fc753d257aa6059ad80e10322fbcde54a80de752dd63c0078
SimHash 470c9273c651

Groups

mediapartners-google*

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

semrushbot-sa

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

semrushbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

dataforseobot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

yandex

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

sitecheckerbotcrawler

Rule Path
Allow /

*

Rule Path Comment
Allow / -
Disallow /db/ sans interet
Disallow /chat*/ sans interet
Disallow /file/out/ sans interet
Disallow /index2.html sans interet
Disallow /dnl/ sans interet
Disallow /cgibin/ sans interet
Disallow /cgi-bin2/ sans interet
Disallow /pag/req/adm/ sans interet
Disallow /pag/too/ sans interet

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.tonga-soa.com/sitemap_index.xml

Comments

  • Disallow: /file/ # sans interet
  • Disallow: /rencontres/ # sans interet
  • Disallow: /upl/user/ # sans interet
  • Disallow: /upl/usern/ # sans interet
  • Disallow: /membres/ # sans interet
  • slow down Yahoo
  • slow down all bot