Keyper droprates from 4969 runs

#21

Aigh, I’ll let him to know to come here

#22

sorry, I dont have discord, so i cant. Also im not sure how helpful I can actually be since i havent done this type of statistics in a while

#23

Lmao but it is too bad you got banned

#24

At least I finally manged to quit this game c:

#25

So trolling with an in-game item is enough to get perma’d, but using a literal hacked client gets you 2 weeks?

#26

Exactly, they make actually broken items (9<s of invicibility) then “OMG HOW U DO THAT”

#27

I would be happy if they were all permabanned first offence, but I can understand why deliberately killing other players is more serious than using a client to just help yourself.

#28

Hi, bot creator here. I will answer the questions that have cropped up in this thread now.

  1. 99% is used because of the number of “rates” we are dealing with. Our work is analogous to 600-ish published journal articles or experiments each, so overall, the less strict the CI, the more results are likely to be wrong. One person might be concerned with the drop rates from one dungeon - 10-50 rates. A 95% CI makes it much more likely that several of these rates are wrong, rather than one or two at most. Does that make sense? I may have misunderstood CIs here.
  2. (I am going to write this answer with the assumption that you are interested in the hard math.) Error margins are included because the drop rate information does not have any easy mathematical translation to what you can say in the drop rates. 1 drop out of 5000 returns a median drop rate of higher than 1/5000, because we are calculating these margins with the cumulative Beta distribution (inverse of the Binomial distribution, which RotMG and other games’ drop rates follow). The 99.5%-ile and 0.5%-ile drop rates are not symmetric from the median, either. For instance, think about how 1 drop out of 1 does not imply a 100% drop rate. The special cases of the Beta distribution which you may find interesting are that for 0 or 1 drops out of 1, and 0 drops out of 0, for more intuition. (Beta(1, 1) and Beta(2, 1)).
#29

It may be very slightly affected by number of people at runs at very small numbers. The conditions which affect drop rates, excluding clovers and events (which are controlled for), are averaged over the many runs and so, happily, the final result should be an average over a varied gameplay experience, albeit biased slightly towards larger (discord) runs, which does not have a massive effect.

#30

Hi, see my answers to Skalding. The confidence interval is based off the Beta distribution (it spans the middle area of the distribution, excluding tails of area 0.005), which is a mathematically accurate (not an approximation in any sense except “quality” of data - it is derived directly as the inverse of the binomial distribution which RotMG loot usually closely takes) way of getting “probabilities of probabilities” so to speak. Do let me know more about this - CIs are not something I have learned formally and would love to hear your thoughts.

#31

Hi, see my answers to Univoid and Skalding. The Beta distribution the bot uses to calculate the error margins/CIs is not symmetrical, nor is the median centred on the experimental drop rate x/n, which is indeed really counter-intuitive given that the Beta distribution is the mathematical inverse of the binomial distribution that the game’s loot follows closely. However, to my knowledge, it is correct. Some counterexamples: 0/0 is undefined, but when you have 0 drops over 0 runs, you can still say that p (drop rate) is likely to be anywhere from 0 to 1 with uniform probability density. Further, if you get 1 drop from 1 run, the drop rate is not definitely 1; the beta PDF takes the form of PDF = 2p where p is the drop rate you think might be the real one. And so on.

#32

The main idea is for a website to present the information in a more easy-to-understand way.

#33

The beta distribution can be the conjugate prior of the binomial distribution. That is, for a binomial distribution f(X|p) = Bin(n, p) you can decide p as a random variable to get f(p|X=k)= beta dist(a,b) (essentially Bayes’ rule applied to the original function and a new constant, too unwieldy to describe here) for various observations of X and new parameters (no longer n and p - these new parameters are what you generate when you input data from players). So when we are unsure about the probability itself, we can try to ‘teach’ or ‘update’ our binomial distribution by using a beta distribution. That is probably why the bot uses the beta distribution for loot, which is naturally a binomial distribution unless specifically coded otherwise. The beta distribution also only allows fractional values, which is great since we are looking to get probabilities for… probabilities, like you said. My basic statistics courses only had beta dist. as a fringe topic so most of my knowledge is from self study.

On the other hand, a confidence interval is largely a qualitative decision that has quantitative underpinnings in terms of application. The way a confidence interval is applied will vary based on the type of distribution - but the confidence interval by definition must give you the ‘concept’ (the range of values over which there is a [ci]% chance of a parameter existing) regardless of symmetry. This is not ‘snipping’ the upper 2.5% and lower 2.5% values for a 95% confidence interval - what happens is that you generate a new likelihood function with the mean you obtained from your beta distribution as a parameter. The decision about what confidence interval to use is still a qualitative/philosophical one. The reason I think 95% is okay is because trying to be 99% certain will generate a very unhelpful range without a very very large number of observations that adhere to a level of methodical standards that you can’t expect in this experiment.

Finally, I want to point out that there is room for qualitative interpretation here that could be more helpful than the math. This is not a response to your work, but a thought your work elicits for me: Unlike the probabilities generated in natural systems, the probabilities of loot are set decisively by people. I think if you can generate a smaller range of mean p, one can wonder about what values of p are likely to be set by a person. One assumption might be that they are (or very close to) rounded values that are comfortable to humans (eg. in a range of 1/13 to 1/21 - I would imagine 1/15 and 1/20 as loot rates a person might actually implement). Of course, this is an assumption and has flaws/could be wrong. I just wanted to give an example. In the same thought experiment, you could wonder, ‘what process is a human likely using to set/update loot probabilities?’. Perhaps a developer tries to set a probability by trying to control the number of times a certain item drops per week. This could encourage one to look at the number of active players, dungeon runs etc. to see how many times a loot drop table for a boss will be called and find probabilities that result in various outcomes for the number of times a certain white (or whites in general - but that is a another line of reasoning) is dropped. By trying to predict what number the dev is trying to control. you can double down on certain probabilities that exist within your mean interval. To clarify, this whole paragraph is just to show how there is room for qualitative interpretation and judgement when it comes to the ‘math’, and not to suggest that the ideas used in the example are what I necessarily believe.

#34

That is a very thought out and good set of ideas. I had not previously considered moving from MOEs to a CI rather than the other way around before. We found that the display of data got a bit cluttered even when including uncertainties before. I am trying to set up a website to display the data and I hope to incorporate that kind of feature (e.g. demanding drop rates with a particular MOE) within that website or natively in the bot at a later date.

I also very much like the consideration of dev drop rate setups. I have seen the XMLs that were used in Build 7 for drop rates before (from a pserver wiki) and those were set on a decimal basis - not by 1 in x settings, preferring nice round numbers like 0.008 or 0.02 - that was my original line of thinking for determining “human” drop rates within our error margins, and what we found seemed to corroborate that (e.g. going on the human-preference hypothesis, I might say that since we know the drop rates for life and mana from MBC are in the 19-21% range, it’s likely to be set as a perfect 1/5, and going on abyss of demons data we know that the drop rates are centred roughly on 1/4).

I humbly invite your exploration into the server here: AR8Gpr8 . We have a large corpus of data (~26,000 runs now) and your insight would be very valuable.

#35

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.