Log In Sign Up

phonedog.com
robots.txt

Robots Exclusion Standard data for phonedog.com

Resource Scan

Scan Details

Site Domain phonedog.com
Base Domain phonedog.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-19T22:03:51+00:00
Next Scan 2024-12-18T22:03:51+00:00

Last Successful Scan

Scanned2024-05-23T12:19:02+00:00
URL https://phonedog.com/robots.txt
Redirect https://news.wirefly.com/robots.txt
Redirect Domain news.wirefly.com
Redirect Base wirefly.com
Domain IPs 147.203.61.14
Redirect IPs 216.105.42.83
Response IP 216.105.42.83
Found Yes
Hash 32d128bdf01673888c332e38775321ab5b9a770c6f037dfb6c6b3ad00c242fdb
SimHash 38941559c272

Groups

*

Rule Path
Allow /misc/*.css$
Allow /misc/*.css?
Allow /misc/*.js$
Allow /misc/*.js?
Allow /misc/*.gif
Allow /misc/*.jpg
Allow /misc/*.jpeg
Allow /misc/*.png
Allow /modules/*.css$
Allow /modules/*.css?
Allow /modules/*.js$
Allow /modules/*.js?
Allow /modules/*.gif
Allow /modules/*.jpg
Allow /modules/*.jpeg
Allow /modules/*.png
Allow /profiles/*.css$
Allow /profiles/*.css?
Allow /profiles/*.js$
Allow /profiles/*.js?
Allow /profiles/*.gif
Allow /profiles/*.jpg
Allow /profiles/*.jpeg
Allow /profiles/*.png
Allow /themes/*.css$
Allow /themes/*.css?
Allow /themes/*.js$
Allow /themes/*.js?
Allow /themes/*.gif
Allow /themes/*.jpg
Allow /themes/*.jpeg
Allow /themes/*.png
Disallow /includes
Disallow /misc
Disallow /modules
Disallow /profiles
Disallow /scripts
Disallow /themes
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /INSTALL.sqlite.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Disallow /admin
Disallow /comment/reply
Disallow /filter/tips
Disallow /node/add
Disallow /search
Disallow /user/register
Disallow /user/password
Disallow /user/login
Disallow /user/logout
Disallow /?q=admin
Disallow /?q=comment%2Freply
Disallow /?q=filter%2Ftips
Disallow /?q=node%2Fadd
Disallow /?q=search
Disallow /?q=user%2Fpassword
Disallow /?q=user%2Fregister
Disallow /?q=user%2Flogin
Disallow /?q=user%2Flogout
Disallow /~r/
Disallow /41014381/PhoneDog
Disallow /phone/feed
Disallow /user/simple-fb-connect
Disallow /?q=filter%2Ftips

Other Records

Field Value
crawl-delay 10

megaindex

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.phonedog.com/sitemap.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
  • CSS, JS, Images
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Custom stuff
Back to top