Austarlia Possible Ddos? [likely to be Golden Dagger bug in Library]


#21

Why did this not get caught in testing? Going through all the dialogue choices is one of the first things that should have been tested.


#22

We went through a -lot- in public testing (upwards of 150 responses). Even then, we weren’t able to catch it.

There were 250 responses (253, with the 3 added when it released to prod?) the Realm Eye had. It’s simply unfeasible to have someone test all 250, especially when it was just a simple copy-paste job to fill in each response (so you would assume that if the first 150 worked fine as the public testers discovered, the other 100 are likely fine; what no one would reasonably expect is that not only did a typo happen with the copy-paste, but also the server really, really not liking the resulting empty string).


#23

It’s simply unfeasible to have someone test all 250

No it’s not. Aren’t there like 30 testers? You could get someone to do all daggers, someone else does all swords, that sort of thing. There’s no reason to have 1 person go through all 250 responses, but they do have to get tested for the exact reasons of having a typo. Assumptions are dangerous when testing, because that’s how you get things like realm crashes on demand, and it’s the exact thing that testing exists to prevent in the first place.
As much as I know you guys like to think of yourselves as early access, that’s not your job.


#24

What bothers me is the fact that Deca implemented the patch on Monday. I understand not working on weekends, but there should be some emergency coverage if something like this happens.


#25

It was surprising, after it had been established only one line of code needed fixing, that it wasn’t done immediately.


#26

it was fine in testing its when toasterzs was moving it to prod did the problem appear


#27

You’re assuming that all 30 testers have the time to spare to do a bunch of responses (we don’t even know how many private testers are active; yes, if they aren’t active enough to do so, then you could say they should be replaced, but I think it’s debatable whether or not failing to sit down and go through 80 responses, with no suspicion that a typo in them would actually cause a game-breaking bug beforehand, is worth demoting someone from the role of private tester).

Hindsight is 20/20. Toastrz himself stated he didn’t expect the empty string to cause a server crash; it’s unfortunate and unforeseen that it did. Why on earth would someone want to manually go through 250 copy-pasted responses when the worst that could happen is some phrases might get switched in order, or one of the phrases happens to be blank (which, once again, no one had any reason to suspect would cause major problems).

If anything, this should have been debugged and caught long before (“what happens if we set the text to an empty string?”). 250 actual responses intended to be used on prod are not proper test cases. An unintentional error in one of the responses still does not make the set of responses a set of proper test cases. I think it’s absurd to have testers waste their time going through 250 responses when such a bug can be easily caught with potential <10 test responses by the developers themselves.

Like, I really want to stress this. You don’t need 250 test cases. If you have 250 test cases for a bit of code with almost all of them being functionally the same (a string with letters, maybe some numbers, I guess they have differing lengths), dear god am I going to judge you and your code. What I’m trying to get at is no group should be expected to test 250 responses like that, yes testing is for catching bugs but your QA/beta testing group should not be the replacement for making sure your code actually runs and works…

(I’m saying this assuming a lot of stuff, but I also don’t want to just blame Deca for all of this without knowing what really went on; it’s genuinely possible that with how everything interweaves and all that, stuff happens and they couldn’t have foreseen this either, or they did test it but something did happen, it’s complicated and I really don’t mean to play the hot potato blame game)

Additionally, I think your response is disrespectful to the private testers. The bit about “early access” is completely unwarranted and I’ve seen nothing from any private testers that indicates they treat private testing like that. It sounds like you’re saying this just because this bug made it to prod; even if it did happen to be completely the private testers’ fault (which, as I have stated above, I think is ridiculous to blame them for), I think painting them all as just wanting “early access” to the game as opposed to being actual testers is rude.

The public testers also, as I mentioned above, did upwards of 150 responses. That was a lot of time and effort spent by people who loved the new dungeon and the lore, completely for free, without any obligation to get it done right then and there in public testing. I think ideally, we’d have gotten even more responses before public testing closed, but that would probably require more people to have helped out.

tl;dr blaming testers for not doing things that no sane devs would expect them to do is ???


it was fine in testing its when toasterzs was moving it to prod did the problem appear

Source? No one tried the golden dagger prompt in public testing.


#28

I work in a different sector and I don’t know how things are done in software, but to me it doesn’t seem unreasonable to test all the prompts. The person who organises testing and QC/QA should have a list of actions that need to be performed at a minimum and this should have included testing the prompts.
I appreciate all the unpaid work the testers and UGC group do, and in my opinion this is on Deca, they should have reacted faster and directed the testing better.


#29

Maybe someone did, disconnected, and didn’t think anything of it because people disconnect all the time…


#30

I think that the golden dagger glitch was meant to be there