nextshark.com
robots.txt

Robots Exclusion Standard data for nextshark.com

Resource Scan

Scan Details

Site Domain nextshark.com
Base Domain nextshark.com
Scan Status Ok
Last Scan2024-11-14T12:12:06+00:00
Next Scan 2024-11-21T12:12:06+00:00

Last Scan

Scanned2024-11-14T12:12:06+00:00
URL https://nextshark.com/robots.txt
Domain IPs 104.22.66.85, 104.22.67.85, 172.67.42.173, 2606:4700:10::6816:4255, 2606:4700:10::6816:4355, 2606:4700:10::ac43:2aad
Response IP 172.67.42.173
Found Yes
Hash ea7e4193fdc4244b1b6922e398d480497f483f55c91b9e9cdbf31f2d9160c074
SimHash 671a55544187

Groups

amazonadbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

Other Records

Field Value
sitemap https://nextshark.com/sitemap_index.xml