moddb.com
robots.txt
Robots Exclusion Standard data for moddb.com
Resource Scan
Scan Details
| Site Domain | moddb.com |
| Base Domain | moddb.com |
| Scan Status | Ok |
| Last Scan | 2024-11-14T18:31:57+00:00 |
| Next Scan | 2024-11-21T18:31:57+00:00 |
Last Scan
| Scanned | 2024-11-14T18:31:57+00:00 |
| URL | https://moddb.com/robots.txt |
| Redirect | https://www.moddb.com/robots.txt |
| Redirect Domain | www.moddb.com |
| Redirect Base | moddb.com |
| Domain IPs | 104.26.12.171, 104.26.13.171, 172.67.70.78, 2606:4700:20::681a:cab, 2606:4700:20::681a:dab, 2606:4700:20::ac43:464e |
| Redirect IPs | 104.26.12.171, 104.26.13.171, 172.67.70.78, 2606:4700:20::681a:cab, 2606:4700:20::681a:dab, 2606:4700:20::ac43:464e |
| Response IP | 172.67.70.78 |
| Found | Yes |
| Hash | 52259adb906db8e0a1d1ff2609129c8752959eae5393b8f9c1feb5342cca9397 |
| SimHash | 67dcd2525ec0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*? |
| Disallow | /*/ajax/ |
| Disallow | /*/blog/ |
| Disallow | /*/comments/ |
| Disallow | /*/add/ |
| Disallow | /*/edit/ |
| Disallow | /*/images/archive |
| Disallow | /*/page/ |
| Disallow | /*/stats/ |
| Disallow | /*/videos/archive |
| Disallow | /platforms/*/set/ |
| Disallow | /addons/start/ |
| Disallow | /admin |
| Disallow | /articles/ajax/ |
| Disallow | /cart |
| Disallow | /comment |
| Disallow | /company/ajax/ |
| Disallow | /cutoff/html/ |
| Disallow | /downloads/ajax/ |
| Disallow | /downloads/mirror/ |
| Disallow | /downloads/start/ |
| Disallow | /downloads/timeout/ |
| Disallow | /external |
| Disallow | /features/ajax/ |
| Disallow | /forum/ajax/ |
| Disallow | /forum/shortcut/ |
| Disallow | /forum/search |
| Disallow | /games/ajax/ |
| Disallow | /giveaways |
| Disallow | /groups/ajax/ |
| Disallow | /groups/gamefront/ |
| Disallow | /groups/indiedb/ |
| Disallow | /groups/moddb/ |
| Disallow | /groups/slidedb/ |
| Disallow | /groups/vrdb/ |
| Disallow | /home |
| Disallow | /hardware |
| Disallow | /media |
| Disallow | /members/ajax/ |
| Disallow | /members/logout/ |
| Disallow | /members/password/ |
| Disallow | /members/permission/ |
| Disallow | /messages |
| Disallow | /mods/ajax/ |
| Disallow | /newsletter/ajax/ |
| Disallow | /profile |
| Disallow | /reviews/ajax/ |
| Disallow | /search |
| Disallow | /software |
| Disallow | /staff |
| Disallow | /statistics |
| Disallow | /tags/ajax/ |
| Disallow | /tutorials/ajax/ |
mj12bot
ahrefsbot
semrushbot
dotbot
mauibot
petalbot
| Rule | Path |
|---|---|
| Disallow | /*? |
| Disallow | /*/ajax/ |
| Disallow | /*/blog/ |
| Disallow | /*/comments/ |
| Disallow | /*/add/ |
| Disallow | /*/edit/ |
| Disallow | /*/images/archive |
| Disallow | /*/page/ |
| Disallow | /*/stats/ |
| Disallow | /*/videos/archive |
| Disallow | /platforms/*/set/ |
| Disallow | /addons/start/ |
| Disallow | /admin |
| Disallow | /articles/ajax/ |
| Disallow | /cart |
| Disallow | /comment |
| Disallow | /company/ajax/ |
| Disallow | /cutoff/html/ |
| Disallow | /downloads/ajax/ |
| Disallow | /downloads/mirror/ |
| Disallow | /downloads/start/ |
| Disallow | /downloads/timeout/ |
| Disallow | /external |
| Disallow | /features/ajax/ |
| Disallow | /forum/ajax/ |
| Disallow | /forum/shortcut/ |
| Disallow | /forum/search |
| Disallow | /games/ajax/ |
| Disallow | /giveaways |
| Disallow | /groups/ajax/ |
| Disallow | /groups/gamefront/ |
| Disallow | /groups/indiedb/ |
| Disallow | /groups/moddb/ |
| Disallow | /groups/slidedb/ |
| Disallow | /groups/vrdb/ |
| Disallow | /home |
| Disallow | /hardware |
| Disallow | /media |
| Disallow | /members |
| Disallow | /messages |
| Disallow | /mods/ajax/ |
| Disallow | /newsletter/ajax/ |
| Disallow | /profile |
| Disallow | /reviews/ajax/ |
| Disallow | /search |
| Disallow | /software |
| Disallow | /staff |
| Disallow | /statistics |
| Disallow | /tags/ajax/ |
| Disallow | /tutorials/ajax/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.moddb.com/api/siteindex |
| sitemap | https://www.moddb.com/api/siteindex |