Literally, being on a tree limb is risky because it requires good balance and careful footing in order to keep from falling. You are hanging out over space without much in the way of support. Similarly, metaphorically being out on a limb describes a situation in which one screw-up can send you crashing to the metahporical ground, which hurts just as much as, if not more, than actual real-life ground.