terminix.com
robots.txt

Robots Exclusion Standard data for terminix.com

Resource Scan

Scan Details

Site Domain terminix.com
Base Domain terminix.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-21T16:34:11+00:00
Next Scan 2024-09-19T16:34:11+00:00

Last Successful Scan

Scanned2024-01-31T14:09:07+00:00
URL https://terminix.com/robots.txt
Redirect https://www.terminix.com/robots.txt
Redirect Domain www.terminix.com
Redirect Base terminix.com
Domain IPs 20.75.32.102
Redirect IPs 20.75.32.102
Response IP 20.75.32.102
Found Yes
Hash 6c8a8a586c68d91a882e0224799f5b17e2d38400b71793c1fb1beba01c87655d
SimHash 46dccf15a790

Groups

*

Rule Path
Disallow /FAQ.aspx
Disallow /buyonline*
Disallow /EditorPage*
Disallow /blog/search
Disallow /purchase/cart
Disallow /www.terminix.com
Disallow /home-blog/article
Disallow /blog/pest-control
Disallow /exterminators/*/%5C
Disallow /home-blog/category/
Disallow /blog/termite-control
Disallow /exterminators/search
Disallow /exterminators/*/index.html
Disallow /home-disinfecting-service/

buck

Rule Path
Disallow /

wget

Rule Path
Disallow /

yeti

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

fever

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

jersey

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

blp_bbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

gobuster

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

jobkicks

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

seekport

Rule Path
Disallow /

cyencebot

Rule Path
Disallow /

lumtelbot

Rule Path
Disallow /

magus bot

Rule Path
Disallow /

onpagebot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

nl-crawler

Rule Path
Disallow /

extlinksbot

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

siteimprove

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

lua-resty-http

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

yunsecuritybot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

brokenlinkcheck.com

Rule Path
Disallow /

dow jones searchbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; msie 10.0; windows nt 6.1; trident/6.0) linkcheck by siteimprove.com

Rule Path
Disallow /

mozilla/5.0 (compatible; msie 10.0; windows nt 6.1; trident/6.0) sitecheck-sitecrawl by siteimprove.com

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.terminix.com/sitemap_index.xml