southwest.newsnetmedia.com
robots.txt

Robots Exclusion Standard data for southwest.newsnetmedia.com

Resource Scan

Scan Details

Site Domain southwest.newsnetmedia.com
Base Domain newsnetmedia.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-29T18:20:31+00:00
Next Scan 2024-12-28T18:20:31+00:00

Last Successful Scan

Scanned2023-03-10T16:23:54+00:00
URL https://southwest.newsnetmedia.com/robots.txt
Domain IPs 104.18.30.13, 104.18.31.13, 2606:4700::6812:1e0d, 2606:4700::6812:1f0d
Response IP 104.18.30.13
Found Yes
Hash 76af8109e9bfe8f718f3d430968256245157860483357cf273c92e6f06fb2144
SimHash 45bd574c1bd3

Groups

*

Rule Path
Disallow /ads/
Disallow /global/tools/
Disallow /global/interfaces/
Disallow /global/images/
Disallow /global/include/
Disallow /global/applications/
Disallow /global/pm/
Disallow /global/utilities/
Disallow /global/reports/
Disallow /global/video/
Disallow /applications/
Disallow /cgi-bin/
Disallow /classifieds/
Disallow /default_files/
Disallow /images/
Disallow /include/
Disallow /incoming/
Disallow /reports/
Disallow /professionalservices/
Disallow /search
Disallow /temp/
Disallow /trafficcam/
Disallow /traffic/
Disallow /contentmgmt/
Disallow /link/
Disallow /register
Disallow /login
Disallow /forgot-password
Disallow /reset-password
Disallow /profile

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://southwest.newsnetmedia.com/sitemap.xml.gz
sitemap https://southwest.newsnetmedia.com/sitemap-pages.xml.gz
sitemap https://southwest.newsnetmedia.com/newssitemap.xml.gz
sitemap https://southwest.newsnetmedia.com/videositemap.xml.gz