I'm not saying we should continue to use it and I'm not saying that it's perfectly safe. But if you were to ask regular people about asbestos, I would wager that most of them think that if you are exposed to any sort of asbestos, you're going to get lung cancer immediately. Scaring people to the point where people are misinformed from word-of-mouth doesn't make it any safer.
For example, many houses in the Bay Area have asbestos in it, so it's not like having it in your house is going to kill you. You need to deal with it intelligently, but just having asbestos in your home isn't going to kill you. And even then, unless you're breathing it in all the time, you're not automatically going to get lung cancer, just like if you get a CT scan with a radiation exposure equivalent to hundreds of X-rays, you're not automatically going to get cancer. Proper education is important. Sure, lots of people tragically got lung cancer due to heavy exposure, but some exposure won't guarantee anything bad, but it certainly feels like you will based on how people talk about it.
Most contractors, if they know or are unsure will not touch certain jobs with a 10ft pole for fear of asbestos. In older houses in the Bay Area, likely culprits are old tile, the glue used, or old ducting.
With care, most of this can be removed with little risk. However, as you point out, caution is needed and most DIYers may not think about things. The other extreme is calling in the men in bunny suits that will run multiple thousands for even small issues due to permits, precautions, and disposal.
Statistically, the DIYers need not really think about it, either. If you work with it at most a few days of your life. you can be the unlucky guy who breathes in a lot of asbestos dust and dies, but chances are you won't (if you don't go berserk when demolishing a building, and are smart enough to wear a mouth/nose cap when working inside in dust)
Professionals in the industry, on the other hand, may work way more carefully, but still run a much larger risk of adverse effects due to their longer exposure time. That's another reason the men work in bunny suits.
For example, many houses in the Bay Area have asbestos in it, so it's not like having it in your house is going to kill you. You need to deal with it intelligently, but just having asbestos in your home isn't going to kill you. And even then, unless you're breathing it in all the time, you're not automatically going to get lung cancer, just like if you get a CT scan with a radiation exposure equivalent to hundreds of X-rays, you're not automatically going to get cancer. Proper education is important. Sure, lots of people tragically got lung cancer due to heavy exposure, but some exposure won't guarantee anything bad, but it certainly feels like you will based on how people talk about it.