vaeret1.com
robots.txt

Robots Exclusion Standard data for vaeret1.com

Resource Scan

Scan Details

Site Domain vaeret1.com
Base Domain vaeret1.com
Scan Status Ok
Last Scan2024-04-24T12:10:47+00:00
Next Scan 2024-05-01T12:10:47+00:00

Last Scan

Scanned2024-04-24T12:10:47+00:00
URL https://vaeret1.com/robots.txt
Redirect https://www.vaeret1.com/robots.txt
Redirect Domain www.vaeret1.com
Redirect Base vaeret1.com
Domain IPs 104.21.73.181, 172.67.165.61, 2606:4700:3031::6815:49b5, 2606:4700:3031::ac43:a53d
Redirect IPs 104.21.73.181, 172.67.165.61, 2606:4700:3031::6815:49b5, 2606:4700:3031::ac43:a53d
Response IP 172.67.165.61
Found Yes
Hash eac4a2d680d75a4a1cb2b1c912d38e23975975eeccc64ddb4017eee1416543d8
SimHash c3228c9a576b

Groups

*

Rule Path
Disallow /africa/ethiopia/lake-tana%20
Disallow *%20*
Disallow /africa*
Disallow /amp/africa*
Disallow /asia*
Disallow /amp/asia*
Disallow /north-america*
Disallow /amp/north-america*
Disallow /south-america*
Disallow /amp/south-america*
Disallow /oceania*
Disallow /amp/oceania*
Disallow /europe/armenia*
Disallow /europe/albania*
Disallow /europe/bosnia*
Disallow /europe/bulgaria*
Disallow /europe/andorra*
Disallow /europe/estonia*
Allow /asia/thailand*
Allow /amp/asia/thailand*
Allow /africa/morocco*
Allow /amp/africa/morocco*
Allow /asia/malaysia*
Allow /amp/asia/malaysia*
Allow /asia/saudi-arabia*
Allow /amp/asia/saudi-arabia*
Allow /asia/seychelles*
Allow /amp/asia/seychelles*
Allow /asia/indonesia/jakarta-raya/jakarta*
Allow /amp/asia/indonesia/jakarta-raya/jakarta*
Allow /asia/sri-lanka*
Allow /amp/asia/sri-lanka*
Allow /africa/egypt*
Allow /amp/africa/egypt*
Allow /africa/tanzania*
Allow /amp/africa/tanzania*
Allow /asia/singapore*
Allow /amp/asia/singapore*
Allow /africa/mauritius*
Allow /amp/africa/mauritius*

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 99999999

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 6

Other Records

Field Value
sitemap http://www.vaeret1.com/sitemap.xml