I'm not.
First - it's not asbestos - and hasn't been for a nuber of years. It's an organic friction material that is highly tolerant of heat, but it's not asbestos. They stopped selling asbestos brakes shortly after they stopped selling asbestos insulation. Properly, you'd just call it "friction material" if you don't know what sort it is.
Second - brake friction is not only subject to a great deal of stress and pressure - and heat! - it's also subject to it intermittently, and not all at the same time (when you have the park brake on, the material is compressed, but isn't heated and isn't under shear load. When you stop, it's got all three. After you've stopped and you're starting again, it's hot, but not stressed.) This will result, eventually, in cracking of the friction material. With the bonded material - even if it falls to bits it should stay in place long enough to get a couple stops - or one good stop. Rivetted is sometimes also bonded, but the advantage of rivetted friction shoes is that heat is transferred from the friction (heat source) to the metal shoe (heat sink,) and can get transferred to the axle faster to reduce failure.
As long as you don't have any heavy cracks (you can't slip a piece of heavy paper into them,) and you don't have any chunks missing, they're nothing to worry about. It's common to notice heat cracks or heat checking in brake material - both friction material, and cast drum & rotor surfaces. Again, as long as they aren't too wide or deep - or you note a straw-coloured or bluish discolouration, you should be find (if the metal is discoloured, replace it - don't get it turned, since the discolouration will indicate a hard spot in the surface of the rotor/drum.)
Hope this helps.
5-90