Everything Statistics - September 29, 2001

"There are three kinds of lies: lies, damned lies, and statistics"

This quote, attributed to Benjamin Disraeli should grant me immunity for what I am about to present; Last Saturday, JayBonci pulled off a gigantic datafile from the E2 server. After some post-processing, I was able to do a statistical analysis on all of the node and noder data. These statistics are now available on the E2 server.

The analysis sheds some light on frequently reoccurring questions such as:

• How many active noders does E2 have?
• What's a good average node reputation? How does my noding compare to that of other noders?
• What is "good" XP or Node-Fu?

The document also gives some insight in the Level Advancement System. As the statistics show, (and most noders already know this), it is much more difficult to meet the writeup requirements than the XP requirements in order to level-up. This suggests that it may be time to raise the XP requirements, in order to promote higher quality writing. Or perhaps the Level Advancement System needs to be reworked completely to a system based on writeup reputation, instead of XP.

I have talked to dem bones and several other gods, and changes in the Level Advancement system are on the horizon. Two fundamental problems stand in our way:

• 'Change frightens us'
• database strain / lag - this is our number one concern

Whether the change is based on XP, MNFP, or otherwise is still up in the air. These statistics may serve as a resource for this issue. The statistical analysis can be found at:

http://everything2.com/stats

For a better understanding of some of the terms used in the document, please read my writeups on Median Node-Fu, and Median Node-Fu Product. In case you're not too familiar with statistical terms, the mean and median writeups are also useful.

To keep the E2 frontpage clean, I adressed the numerous comments in Everything Statistics - September 29, 2001 (2). Also see my October 5, 2001 writeup.

Professor Pi's system of using median node-fu product (MNFP) thresholds as a level advancement system resembles something I had been developing in my scratch pad. A system based on reputation would have some advantages: it would discourage use of "Don't display in New Writeups" autonoded bulls*** and would encourage users to vote tag each other.

Median has flaws

However, raw MNFP as a level advancement system has one major flaw: discontinuity. For example, if a user has 249 writeups <= 2 rep and 250 writeups >= 3 rep, the user has median rep 3, and MNFP = (3+1)*499 = 1996. If this user submits two writeups that go unnoticed in New Writeups and do not quite reach 3 rep by the end of the day, the user will then have 251 writeups <= 2 rep and 250 writeups >= 3 rep, putting the median rep down to 2 and the MNFP down to (2+1)*501 = 1503. A loss of 493 MNFP seems like a pretty harsh penalty for adding two writeups. Such a system would also exact less-than-fair penalties on victims of drownvoting. 1010011010's writeup below explains more about the dangers of the median.

Prof. Pi wrote in response: "adding those 2 wus should not change matters until median is recalculated. If the 2 wus remain below 3 rep by the end of the day... then the median will change. but then he has more than 50% below median=3, so that's fair. He's borderline and should work on increasing his rep in future wus." There's a difference between being borderline and having hardly anybody awake to read a writeup submitted outside of United States waking hours; I've had some good work fall out of New Writeups at (+0/-0).

To solve this, I propose using interquartile node-fu product (IQNFP) instead. Take the mean reputation of a user's writeups between the 25th and 75th percentile, add 1, and multiply by the number of writeups. This has the same property of rejecting outliers as MNFP without the harsh discontinuity.

m_turner gladly let me analyze his sample data and present my analysis:

• number of writeups: 586
• mean rep: 8.66
• median rep: 6
• first quartile rep: 4
• third quartile rep: 12
• total rep of writeups in interquartile range: 1882
• interquartile node-fu: 6.423
• IQNFP = (IQNF + 1) * nwriteups = 4,350
• (interquartile total rep * 2) + nwriteups = 4,350

Writeups that fall within interquartile range:

rep  count 'tile
 <=3   146  146    -- first quartile
  3     45  191
  4     39  230
  5     42  272
  6     39  311
  7     33  344
  8     23  367
  9     22  389
 10     21  410
 11     24  434
 12      5  439    -- third quartile
>=12   147  586

This sample data shows that the distribution is a bit skewed, making IQNF slightly larger on average than the MNF but not by much. It also shows the near-impossibility of a single vote changing the IQNFP by more than 2 points.

So why would anybody ever vote?

Another problem exists of how to reward voting. Prof. Pi has assured me that there will still be XP (so users can get a number about which they can boast), but it probably won't contribute much to the level. The new code can even multiply it by j (the square root of -1) to help users see that it really is an imaginary number. In fact, the format "Experience: $MNFP + $XPj" would show just how complex things can get.

This was supposed to be an email to dem bones and Professor Pi, but bones told me to node my comments instead.

Remove content, and get rewarded

The proposed system has another major flaw in that it is much easier to "game the system", than under the old system. Particularly in negative ways. As it stand right now I would lose 2 levels (whine, whine, suck it up I know). But I could instead only lose a single level, simply by nuke requesting all my negative rep nodes (which there is largely nothing wrong with). So I could be rewarded by removing valid content? That is not good. Doing some math now, I could remove all of my negative and zero rep nodes and be only a handful of rep points away from not losing a level at all. I ask you, is this what we want? Update, by removing all of my 1 rep and below nodes I will not lose any levels at all.

A troll a day keeps the levels away

With the new system serial downvoters/trolls can actually effect someone. Say I decide to take a disliking to some random 10th level noder. With a weeks worth of votes, I could make him lose a level. A high level noder could vote someone down a level in a single day. You might be thinking "aw, no one would do that". Yes they would, show a troll that he can be effective, and he will put three times the effort into his trolling.

Rise to the top, with political pot smoking lesbian daylogs

The new system can be very effectively gamed by only writing about subjects that will assuredly get massive upvotes. That is, daylogs, politics, drugs, and sex. Do we really need even more of this content than we are already getting now? And if we do need more of this, should the people writing it be lifted above factual noders in the pecking order? I ask you?

Why should I vote?

A lot of E2 users log on every day simply to use up their votes. I would go so far as to say that over 60 percent of the votes cast are voting for XP. Take away the reward for voting, and you take away the votes.

How to win friends and influence people

Another thing is certain noders are, well, popular. Anything they write is going to be voted to the sky. One of my favorite E2 people (I won't mention his name) has a median rep of about 8. But a good 70 percent of his nodes are cut and paste works. So, should being friendly and well liked, mean that you should get free levels? Once again, I ask you?

Solutions

Solutions are simple. First, set a number of upvotes and downvotes per day. (I am 9th level, perhaps I would get 90 upvotes and 15 downvotes). With no one getting any downvotes at all until 3rd level. This effectively removes the ability for a single noder to "vote someone off the island". Why no downvotes for second level you might ask. Simple, it is too easy to make an alternate account, and get it up to second level in a matter of days. With this change, bad nodes will still get voted down, while almost eliminating serial downvoting. (You might ask, should gods and editors get more downvotes? My answer is no. You have the nuke button, if it is that bad, you should use it.)

Secondly, set a cap for the median number. To prevent people from getting to the top from daylogs and blowjob nodes. The exact number for the cap would have to be discussed. This would also very effectively minimize the popularity bias.

Third and final. Provide an alternative means to gain levels based on much higher XP totals. This is mainly so factual noders will still have a way to gain levels, without resorting to rep whoring. This also still give people a "reward" for voting, that they would otherwise lose.

"Using median size as a reference it's perfectly possible to fit four ping-pong balls and two blue whales in a rowboat."

Median is an accurate representation of only the most ideal samples.
The main selling point of Median Node-Fu and Median Node-Fu Product is that you could check this for any user at the cost of, at most, two votes. This selling point is irrelevant to a behind the scenes process as it is not limited in the number of write-ups it can check for reputation.

Here is my recommendation:
Divide the reputation sorted list of someone's write-ups into N sections. Find the reputation of the write-up in the middle of each section. Mutiply the value of these midpoints by 1/Nth the number of write-ups. Add these products together. The resulting sum is someone's contribution to E2.
Sound familar? It's an algorithm for calculating the area under a curve.
As you may recall from calculus, the accuracy of this measure increases with N (specifically in this case accuracy increases as N approaches #WU)
Node-fu would be calculated by averaging the the values of the midpoints. These methods can be refered to as MpNF for Midpoint Node-Fu and MpNFP for sum E2 contribution.

Some interesting things you may not have noticed

1. When N=1 this method gives you MedianNFP and MedianNF
2. When N=#WU this method gives you MeanNF (I.E WNF uncorrected for C!)

Advantages:

• As has been noted MNF and MNFP are generally more acccurate for users with large numbers of write-ups (due to a more ideal distribution of reputations). This is also true for MpNF and MpNFP. However, the expected accuracy of MpNF and MpNFP increase as N-->#WU... therefor noders with fewer WU can expect higher accuracy. The overall result is that MpNF(P) is generally accurate for all users
• The value of N can be the same for all users. Thus there is no greater computational overhead for figuring the MpNF of a Lvl10 noder v. a Lvl3 noder.
• The value of N can be adjusted for different levels. The tendency for expected accuracy to decrease can be corrected by increasing the value of N for higher level users. Because the number of users drops off as level increases this would also not produce a significant change in computational overhead between user groups.
• As with MNF and MNFP, skillfull selection of the value of N will allow radically up- and downvoted write-ups to be ignored in the calculations.

Random other considerations:

Two write-ups both have reputations of 1. One has (+35/-34). The other has (+1/0). Have they contributed equally? Do they both have equal Node-fu?

When the value of recieving C! or the number of votes alotted is changed should XP be recalculated to reflect the changes?

How do you encourage voting but not vote dumping?