Why did this not get caught in testing? Going through all the dialogue choices is one of the first things that should have been tested.
Austarlia Possible Ddos? [likely to be Golden Dagger bug in Library]
We went through a -lot- in public testing (upwards of 150 responses). Even then, we werenât able to catch it.
There were 250 responses (253, with the 3 added when it released to prod?) the Realm Eye had. Itâs simply unfeasible to have someone test all 250, especially when it was just a simple copy-paste job to fill in each response (so you would assume that if the first 150 worked fine as the public testers discovered, the other 100 are likely fine; what no one would reasonably expect is that not only did a typo happen with the copy-paste, but also the server really, really not liking the resulting empty string).
Itâs simply unfeasible to have someone test all 250
No itâs not. Arenât there like 30 testers? You could get someone to do all daggers, someone else does all swords, that sort of thing. Thereâs no reason to have 1 person go through all 250 responses, but they do have to get tested for the exact reasons of having a typo. Assumptions are dangerous when testing, because thatâs how you get things like realm crashes on demand, and itâs the exact thing that testing exists to prevent in the first place.
As much as I know you guys like to think of yourselves as early access, thatâs not your job.
What bothers me is the fact that Deca implemented the patch on Monday. I understand not working on weekends, but there should be some emergency coverage if something like this happens.
It was surprising, after it had been established only one line of code needed fixing, that it wasnât done immediately.
it was fine in testing its when toasterzs was moving it to prod did the problem appear
Youâre assuming that all 30 testers have the time to spare to do a bunch of responses (we donât even know how many private testers are active; yes, if they arenât active enough to do so, then you could say they should be replaced, but I think itâs debatable whether or not failing to sit down and go through 80 responses, with no suspicion that a typo in them would actually cause a game-breaking bug beforehand, is worth demoting someone from the role of private tester).
Hindsight is 20/20. Toastrz himself stated he didnât expect the empty string to cause a server crash; itâs unfortunate and unforeseen that it did. Why on earth would someone want to manually go through 250 copy-pasted responses when the worst that could happen is some phrases might get switched in order, or one of the phrases happens to be blank (which, once again, no one had any reason to suspect would cause major problems).
If anything, this should have been debugged and caught long before (âwhat happens if we set the text to an empty string?â). 250 actual responses intended to be used on prod are not proper test cases. An unintentional error in one of the responses still does not make the set of responses a set of proper test cases. I think itâs absurd to have testers waste their time going through 250 responses when such a bug can be easily caught with potential <10 test responses by the developers themselves.
Like, I really want to stress this. You donât need 250 test cases. If you have 250 test cases for a bit of code with almost all of them being functionally the same (a string with letters, maybe some numbers, I guess they have differing lengths), dear god am I going to judge you and your code. What Iâm trying to get at is no group should be expected to test 250 responses like that, yes testing is for catching bugs but your QA/beta testing group should not be the replacement for making sure your code actually runs and worksâŚ
(Iâm saying this assuming a lot of stuff, but I also donât want to just blame Deca for all of this without knowing what really went on; itâs genuinely possible that with how everything interweaves and all that, stuff happens and they couldnât have foreseen this either, or they did test it but something did happen, itâs complicated and I really donât mean to play the hot potato blame game)
Additionally, I think your response is disrespectful to the private testers. The bit about âearly accessâ is completely unwarranted and Iâve seen nothing from any private testers that indicates they treat private testing like that. It sounds like youâre saying this just because this bug made it to prod; even if it did happen to be completely the private testersâ fault (which, as I have stated above, I think is ridiculous to blame them for), I think painting them all as just wanting âearly accessâ to the game as opposed to being actual testers is rude.
The public testers also, as I mentioned above, did upwards of 150 responses. That was a lot of time and effort spent by people who loved the new dungeon and the lore, completely for free, without any obligation to get it done right then and there in public testing. I think ideally, weâd have gotten even more responses before public testing closed, but that would probably require more people to have helped out.
tl;dr blaming testers for not doing things that no sane devs would expect them to do is ???
it was fine in testing its when toasterzs was moving it to prod did the problem appear
Source? No one tried the golden dagger prompt in public testing.
I work in a different sector and I donât know how things are done in software, but to me it doesnât seem unreasonable to test all the prompts. The person who organises testing and QC/QA should have a list of actions that need to be performed at a minimum and this should have included testing the prompts.
I appreciate all the unpaid work the testers and UGC group do, and in my opinion this is on Deca, they should have reacted faster and directed the testing better.
Maybe someone did, disconnected, and didnât think anything of it because people disconnect all the timeâŚ
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.