scenebeta.com
robots.txt
Robots Exclusion Standard data for scenebeta.com
Resource Scan
Scan Details
| Site Domain | scenebeta.com |
| Base Domain | scenebeta.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a server error. |
| Last Scan | 2025-07-30T07:52:54+00:00 |
| Next Scan | 2025-10-28T07:52:54+00:00 |
Last Successful Scan
| Scanned | 2025-04-01T21:29:52+00:00 |
| URL | https://scenebeta.com/robots.txt |
| Redirect | http://www.scenebeta.com/robots.txt |
| Redirect Domain | www.scenebeta.com |
| Redirect Base | scenebeta.com |
| Domain IPs | 104.21.94.22, 172.67.218.104, 2606:4700:3035::6815:5e16, 2606:4700:3037::ac43:da68 |
| Redirect IPs | 104.21.94.22, 172.67.218.104, 2606:4700:3035::6815:5e16, 2606:4700:3037::ac43:da68 |
| Response IP | 104.21.94.22 |
| Found | Yes |
| Hash | a0e03a2a6e0b57e9f8f962613d37e3e60039ef54e79b086dcfe20ea750506b6c |
| SimHash | bc941d08ce74 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /includes/ |
| Disallow | /misc/ |
| Disallow | /modules/ |
| Disallow | /profiles/ |
| Disallow | /scripts/ |
| Disallow | /sites/ |
| Disallow | /themes/ |
| Disallow | /CHANGELOG.txt |
| Disallow | /cron.php |
| Disallow | /INSTALL.mysql.txt |
| Disallow | /INSTALL.pgsql.txt |
| Disallow | /install.php |
| Disallow | /INSTALL.txt |
| Disallow | /LICENSE.txt |
| Disallow | /MAINTAINERS.txt |
| Disallow | /update.php |
| Disallow | /UPGRADE.txt |
| Disallow | /xmlrpc.php |
| Disallow | /admin/ |
| Disallow | /comment/reply/ |
| Disallow | /contact/ |
| Disallow | /logout/ |
| Disallow | /node/add/ |
| Disallow | /search/ |
| Disallow | /user/register/ |
| Disallow | /user/password/ |
| Disallow | /user/login/ |
| Disallow | /?q=admin%2F |
| Disallow | /?q=comment%2Freply%2F |
| Disallow | /?q=contact%2F |
| Disallow | /?q=logout%2F |
| Disallow | /?q=node%2Fadd%2F |
| Disallow | /?q=search%2F |
| Disallow | /?q=user%2Fpassword%2F |
| Disallow | /?q=user%2Fregister%2F |
| Disallow | /?q=user%2Flogin%2F |
| Disallow | /pub/ |
| Disallow | /pub2/ |
| Disallow | /mini/ |
| Disallow | /system/files/ |
| Disallow | /user/ |
| Disallow | /?q=user |
| Disallow | /favorite_nodes/ |
| Disallow | /userpoints |
| Disallow | /userpoints/ |
| Disallow | /myuserpoints/ |
| Disallow | /privatemsg/ |
| Disallow | /abuse/ |
| Disallow | /abuse |
| Disallow | /legal |
| Disallow | /feedback |
| Disallow | /members/staff |
| Disallow | /hof |
| Disallow | /hof/ |
| Disallow | /noticia/*page%3D |
| Disallow | /tutorial/*page%3D |
| Disallow | /aggregator |
| Disallow | /contact |
| Disallow | /logout |
| Disallow | /node/add |
| Disallow | /user/register |
| Disallow | /user/password |
| Disallow | /user/login |
| Disallow | /*sort%3D |
| Disallow | /*/feed$ |
| Disallow | /search$ |
| Disallow | /tracker? |
| Disallow | /*/track$ |
| Disallow | /user2userpoints |
| Disallow | /user/register? |
| Disallow | /user/login? |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Comments