Simple, yet oddly confusing.
First off, average dice are not the dice you use to play Yahtzee and Monopoly. A normal die will have its sides marked with pips representing the numbers 1, 2, 3, 4, 5, and 6. An average die will have the sides marked with the numbers 2, 3, 3, 4, 4, and 5. Average dice traditionally are marked with printed numbers rather than pips, although pipped average dice are available.
Average dice are useful in gaming, and particularly in war games, because they will average the same as a standard d6 (a 'normal' die), but do not allow for the extremes of a one or a six. This allows for some units to be more stable (with the average dice), while others are more unpredictable (regular d6). In war games the units that are rolled with normal d6's are termed irregulars, while those rolled with average dice are called regulars. Thus, average dice are sometimes called regular dice.
Average dice are hard to find in America, but apparently aren't uncommon in England. You can easily order them on-line, and in a pinch you can buy blank dice and label them yourself with a marker. If you are really lazy, you can just use your regular dice, and remember that a six is a four, and a one is a three.
No die collection is complete without one.