urlgalleries.net
robots.txt

Robots Exclusion Standard data for urlgalleries.net

Resource Scan

Scan Details

Site Domain urlgalleries.net
Base Domain urlgalleries.net
Scan Status Ok
Last Scan2024-09-20T10:29:47+00:00
Next Scan 2024-09-27T10:29:47+00:00

Last Scan

Scanned2024-09-20T10:29:47+00:00
URL https://urlgalleries.net/robots.txt
Domain IPs 104.21.1.243, 172.67.152.155, 2606:4700:3030::ac43:989b, 2606:4700:3033::6815:1f3
Response IP 104.21.1.243
Found Yes
Hash 9cf4cbc11fccc6002f79aa55c3c8ae8de286882a9abc8c32848e327820f10f6e
SimHash 8078dd276dd0

Groups

*

Rule Path
Disallow /blog_report.php
Disallow /image_new.php
Disallow /image.php
Disallow /out.php
Disallow /trade/
Disallow /login_form.php
Disallow /static.php
Disallow /reset_password.php
Disallow /trade.php
Disallow /out/
Disallow /contact
Disallow /static/
Disallow /maillist/
Disallow /bcontact/
Disallow /blog_trade.php
Disallow /signup.php
Disallow /contact.php
Disallow /report.php
Disallow /blog_maillist.php
Disallow /blog_contact.php
Disallow /advertisers/
Disallow /advertisersback/
Disallow /report
Disallow

ia_archiver

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Comments

  • User-Agent: bingbot
  • Disallow: /
  • semrush spam
  • ahrefs spam