edit.dailyherald.com
robots.txt

Robots Exclusion Standard data for edit.dailyherald.com

Resource Scan

Scan Details

Site Domain edit.dailyherald.com
Base Domain dailyherald.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-04T02:12:49+00:00
Next Scan 2024-12-03T02:12:49+00:00

Last Successful Scan

Scanned2024-05-08T01:57:27+00:00
URL http://edit.dailyherald.com/robots.txt
Domain IPs 52.70.179.129, 54.82.140.154
Response IP 54.82.140.154
Found Yes
Hash d36b6cee7e0e86350ce5fb5b5ed7d6194f7f411c1921d43bb7eceee62c3a8b3a
SimHash cc605c252981

Groups

*

Rule Path
Disallow /apps/pbcs.dll/classifieds
Disallow /apps/pbcs.dll/events
Disallow /apps/pbcs.dll/index
Disallow /apps/pbcs.dll/temaoversikt
Disallow /apps/pbcs.dll/related
Disallow /apps/pbcs.dll/misc
Disallow /apps/pbcs.dll/error
Disallow /apps/pbcs.dll/search
Disallow /apps/pbcs.dll/netguest
Disallow /apps/pbcs.dll/ptshowguide
Disallow /apps/pbcs.dll/ptshowguideitem
Disallow /apps/pbcsad.dll
Disallow /apps/rub.dll
Disallow /tmp/
Disallow /logs/
Disallow /.cache/
Disallow /mal/
Disallow /templates/
Disallow /dev/
Disallow /forsaxo/
Disallow /section/mostread?template=ovr.json&mime=json

googlebot

Rule Path
Disallow /article/*/print/$

Other Records

Field Value
crawl-delay 10

Comments

  • EditRobots.txt
  • Be nice.
  • Sitemap: /sitemap_general.xml