Jakcodex/Muledump v9.4.0 - New Character Sorting and Export UIs, storage compression, and more

Jakisaurus · 2018-08-06 09:29:06 UTC

Hello everyone,

I am pleased to announce a new version of Jakcodex/Muledump. This is a smaller update with improved UIs, a handful of small new features, and some bug fixes.

Jakcodex/Muledump - current version 9.4.0

Please see the Installation and Setup Guide to get started.

The online web version is updated and available on Github at Muledump Online.

Releases can be downloaded on Github for Muledump Local

You are welcome to join our Discord server - https://discord.gg/JFS5fqW

v9.4.0 Changelog Summary

New Features

Text exports can be uploaded to Jakcodex/Paste for linking
SetupTools Compression can be optionally enabled to increase the amount of data your browser can store
Accounts.js users can now set their own SetupTools client configuration settings
Clock correction now available as an optional feature

Improvements

Character Sorting UI rewritten with a visual drag and drop display and new options
Muledump Exporter UI created to handle exporting text and images of Muledump data
Default export mode and totals export image width added as new settings

Bug Fixes

Dead characters not cleaned up properly from custom character sorting lists
MuleQueue fails when accounts get disabled while in queue
Disabling SetupTools can cause mules to not parse properly

News

New Character Sorting UI

No more lists of numbers! The new UI is fully visual and is sorted by drag and drop. Changes are saved automatically. Optionally display non-selected characters after the main list with a handful of sorting modes.

Muledump Exporter

The major change here is the introduction of Jakcodex/Paste support. Generate short urls with your exports data to link to users.

Accounts.js Users and SetupTools Client Configuration

A good number of users still prefer to utilize accounts.js files instead of SetupTools. One drawback to this has been no means to easily change settings only available in SetupTools.

Accounts.js users can now fully configure SetupTools, Character Lists, and Totals Config Sets.

See Accounts.js and Settings in the wiki for more information.

Storage Compression

Many users have found themselves reaching the limit for storage space the browser allows Muledump to use. This release introduces storage compression as an optional feature. You can read more about this at Storage Compression in the wiki.

Thanks

Thank you to everyone who has helped contribute to Muledump. Your feedback, bug reports, ideas, testing, input, and skills have been invaluable.

This is a follow up to the previous release post.

Skandling · 2018-08-27 12:28:46 UTC

As I noted in another thread, this release is not working for me. In particular:

Every time it opens nothing displays: I am presented with a black window with just the menus visible. To get anything to display I need to change something. e.g. if I change the sort order of the totals they display. I then need to change the sort order back to the one I am used to. The same for the characters although there I can just hide/show the account name which I don’t care about.

The second problem is reloading no longer works. If I choose ‘reload accounts’ from the menu (it used to be a button, no idea why this was changed) nothing happens. I need to use the menu again, go into a dialog, then for each account tell it to load immediately from yet another menu, one for each account.

For now I have gone back to the previous version, 9.3.2, which works fine.

My details: on a Mac, using Google Chrome. Muledump is hosted on the same machine with a http://localhost address. To update Muledump I just download the zip, delete the previous files and copy the files into the same directory which already contains my accounts.js

Sorry for not posting this sooner, but I was unaware of this thread – I am notified of updates from the notices within Muledump.

Skandling · 2018-08-27 15:10:42 UTC

One more thing that’s missing that I‘ve rediscovered in 9.3.2: the ability to click on an item in e.g. the totals and it’s highlighted there and everywhere you have it. Useful for tracking down things when you’ve forgotten where they are. This feature seems not to be working in 9.4

Jakisaurus · 2018-08-28 07:23:09 UTC

This is a unique report meaning that nobody else has reported problems like this. So there’s likely a cause local to your end causing this.

After loading Muledump or trying to reload accounts can you hit F12 in your browser and go to console. There may be errors recorded there. If so I’d like to see them. If so be sure you clean them of any sensitive account info before sharing them with us here.

Skandling · 2018-08-28 07:54:18 UTC

Here it the log in full. BTW F12 does something different on the Mac, but the console is available from a menu

SetupTools/Init - Bypassing
app.js:620 SetupTools/Init - Client config importing 2 keys
2app.js:620 Muledump_TotalsCounter/Import mode 1
app.js:620 Muledump/Update_Totals mode 0
totals.js:118 Uncaught TypeError: Cannot read property 'indexOf' of undefined
    at isbad (totals.js:118)
    at upd2 (totals.js:108)
    at update_totals (totals.js:165)
    at muledump.js:134

I should add: that is what I see at startup, without using any of my workarounds or trying to reload data.

Jakisaurus · 2018-08-30 09:30:26 UTC

So I notice that you are an accounts.js user and that it says that it imported two client config keys. Did you add anything to your accounts.js in way of Totals configuration sets or anything other than the accounts? Could you possibly share the non-accounts portion of your accounts.js? Or message it to me?

Skandling · 2018-08-30 09:58:28 UTC

Here it is in full, with the personal details obsfucated. I have not changed it in a long time (over 3 years according to the timestamp on the file), I just leave it in place and replace everything else in the folder when updating Muledump.

// editor with syntax hilighting is recommended (for example, notepad++ or textmate)

accounts = {

// Put your data here as shown below. Don't forget the commas and quotes!
// If your emails or passwords contain single quotes,
// escape them with backslashes (\) like this: 'pass\'word'


'kongregate:xxxxxx': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
'xxxxxxxxxxxxxx@xxx.xxx' : 'xxxxxxxx',


}// don't delete this line!


// how many characters are displayed in each row (within one account)
rowlength = 7

// change to 1 to switch to testing
testing = 0

// change to 1 to enable price display in tooltips
prices = 0

// change to 1 to enable one-click login (run lib/mulelogin.au3 first)
mulelogin = 0

// 0 = use smart layout (fill empty spaces)
// 1 = show account boxes row by row
nomasonry = 0

Jakisaurus · 2018-08-30 19:36:20 UTC

Aha, this is a new bug. Interesting that it didn’t turn up during testing.

As a temporary fix, let’s have you modify your accounts.js a bit. Muledump v9.4 introduced a way for users who wish to remain on accounts.js to take advantage of the SetupTools-specific features. But this is not working as intended for you.

First, in your browser console could you type JSON.stringify(userConfiguration, null, 5); and show the output here?

Then try replacing the rowlength, testing, prices, muledump, and nomasonry blocks in accounts.js with this instead:

userConfiguration.config = {
    rowlength: 7,
    testing: 0,
    mulelogin: 0,
    nomasonry: 0
}

The way you have it is supposed to work, but as I said thar be a bug.

Skandling · 2018-08-30 20:58:47 UTC

The output is

"{
     "config": {},
     "muledump": {
          "chsortcustom": {},
          "totals": {
               "configSets": {}
          }
     }
}"

Replacing the lines with the block you give seems not to make any difference; it still does not load and still gives exactly the same error even after restarting + forcing reload.

Jakisaurus · 2018-08-31 21:49:46 UTC

Hoping this isn’t a red herring. Can you show me the console output of this command:

(new SeaSalt_Hashing(JSON.stringify(userConfiguration))).toString()

Then change the addition to accounts.js to look like this instead:

userConfiguration = {};
userConfiguration.config = {
    rowlength: 7,
    testing: 0,
    mulelogin: 0,
    nomasonry: 0
}

If it still isn’t working, then that bug is unrelated to your issue. If that’s the case, can you give the output of:

JSON.stringify(setuptools.tmp.totalsSecondaryFilter, null, 5);

Thanks for diving into the technical details of all this and being patient

Skandling · 2018-08-31 22:00:19 UTC

gives “a43a04fc1c912e1d40319549e7f3e4f78a2953d2d1587ce01b64fe23596f6b88”

The new addition to my accounts.js made no difference - same error in log. After which

gives

undefined

Jakisaurus · 2018-08-31 22:13:26 UTC

Ahh, I think I understand now. So for currently unverified reasons (likely due to differences in our OS) that string is entirely different from what the code expects.

There’s a circuit breaker we can use that should work. Change it to read:

userConfiguration = false;

With a bit of cooperation from the browser this should stop the offending code from running. If not I’ll help you disable the offending code and then push a fix with the next release.

Now for the technical thoughts on the problem which one could choose to not read.

I think the userConfiguration import it is trying is causing the Totals Configuration Sets feature to error which is causing your runtime error. This is what the code looks like and the string it expects, which as you can see is not the same one you got back.

if (
    typeof userConfiguration === 'object' &&
    (new SeaSalt_Hashing(JSON.stringify(userConfiguration))).toString() !== '7df52aba88bafe6ffdcdd8cf2e0d358be3bcb08ae5a4f4991940938c45f8ce96'
) {

I guess it was a bad assumption on my part that a complex structure like that would be consistent across operating systems. The circuit breaker here is that if we change userConfiguration to be false it will not match the first requirement for processing which should prevent breaking Totals ConfigSets.

Skandling · 2018-08-31 22:38:34 UTC

The OS should make no difference. Some more detail: I am using personal web sharing, which has been part of Mac OS X since the first version. But this is nothing Mac specific as under the hood it’s just unix and it uses unix for this, in particular Apache HTTP server.

So it serves up the files as HTML, CSS and JS like any other Apache server. I have not had much to do with web severs for a long time but Apache is about as vanilla as you get. In theory if I opened up ports on my internet router and gave you the IP you could access the server yourself. It’s just a small server running in the background on my Mac, just for Muledump at the moment.

It’s the Mac version of Chrome but that too is cross platform. In fact for HTML, CSS and JS handling you have a hard time telling the difference between all modern browsers, at least on Mac. They all work identically. Differences tend to be in arcane things like media support, Flash and other plugin support, Firefox’s odd multilingual support. Loading web pages, including ones with lots of CSS and complex JS, works identically.

Jakisaurus · 2018-08-31 22:43:27 UTC

I’m just referring to the fact that when your browser JSON.stringify’s that default userConfiguration object that it returns a value different from what my browser returns. It could be a padding/nullbyte issue or something. But something changes its hash, which is causing it to be imported erroneously. By setting that variable to false it should stop trying to load it.

Skandling · 2018-08-31 23:08:16 UTC

My point was it should be serving up the files identically to anywhere else they might be hosted. They are just text files after all.

Chrome on Mac might be doing something weird, but it seems unlikely – I would expect it to perform identically to Chrome on Windows for any sort of HTML, CSS + JS. It is basically the same code.

I can’t see anything odd about the file such as null/padding chars (I use a text editor which will display all invisible chars as an option). I tried changing the line endings from CR/LF to LF and saved it out but no change. I would think the web server will do this automatically anyway, take file with different sorts of line endings and serve them consistently. And it’s JS so line endings and spacings between elements should not matter – I could strip out all comments and line feeds and it would still be valid JS.

Jakisaurus · 2018-08-31 23:16:07 UTC

We’re not talking about the files tho. We’re talking about your browser turning that object into a JSON string and then hashing it with SHA256. Your browser does it differently than Chrome or Firefox do on Windows.

Either way, did you try setting it to false?

Skandling · 2018-08-31 23:23:04 UTC

I see, so why is it doing that, and why is it important it’s consistent across browsers?

Setting it to false does not work. The same error appears though with different text before:

SetupTools/Init - Bypassing
totals.js:118 Uncaught TypeError: Cannot read property 'indexOf' of undefined
    at isbad (totals.js:118)
    at upd2 (totals.js:108)
    at update_totals (totals.js:165)
    at muledump.js:134

Jakisaurus · 2018-08-31 23:27:33 UTC

Ack, what… so it fixed the first bug but not the second.

Regarding the first bug, the why is likely due to the character encoding of the JSON string. Browsers/OS’s are fairly consistent, but the character encodings and nullbytes vary (like in Windows newline is \r\n whereas in Linux it is \n).

I’m going to try and replicate this problem again using the info you’ve provided. I’ll get back to you tonight or tomorrow night.

Edit: Sorry for a delay in getting back to you on this. Have been caught up in the evenings recently. I’ll try again soon

Jakisaurus · 2018-09-22 08:35:23 UTC

Welp, I’ve tried several times to replicate this issue and I just can’t do it. A last ditch effort could be clicking Setup > Settings > System > Erase All Data. Or manually erasing the local storage data. Perhaps there’s bad data somewhere. Sorry it took so long to reply.

Skandling · 2018-09-22 10:07:19 UTC

Clearing local storage, through the developer tools, made no difference – it’s doing the same with the same error in the console.

No idea myself either, you have my(obfuscated) accounts.js and that should be all that’s different/special about my config. I don’t have any other way to test it as it depends on the Cors extension which requires Chrome (I’d much rather use another browser as this is the only thing I need Chrome for and would uninstall it otherwise).