Actually that is something akin to how it works in Israel, while they won't necessarily kill a whole terrorists' family, they will confiscate all their family's property, bulldoze their house (and often times their neighbors' houses too in the process), and anyone who knows or was connected can be imprisoned or such. Taking away everything the family has and throwing them out on the street is actually more effective when fighting the muslims since killing them would only make them be seen as martyrs.
Taking away their property, on the other hand, affects their family for generations to come since in the arab / muslim world it is near-impossible to 'move up' or 'get rich' by hard work alone. Birth, inheritence, or marraige (for women) determines everything as far as how you and your family live. If the government confiscates property, the family will be poor for many generations and the terrorist will be remembered as the cause of it all.
Of course it's easier to do this in occupied territories, while most terrorist suicide bombers in Israel have their families in the west bank or gaza strip, it's much harder to do in a place like Britian b'cos most of the terrorists' families probably aren't in britian, they're in Syria, Pakistan, Saudi Arabia, etc.
There's no question people have to go on the offensive against these people, but knowing who to attack, when, and where; that is the great difficulty. If all you do is defend, you might stop about half the attacks but the other half still gets through.