The Enigma Machine is a fairly well-known cipher, so I’d like to think that whatever The Glitch’s message is encoded in is also something the layman might think of when it comes to the topic of cryptography, but we could also be just thrown a big curveball as it could be something fairly obscure.
It’s almost certainly not a simple substitution cipher, either (which does lend support to your idea of it being a specific setup of the Enigma Machine).
edit: Realm Eye Responses page has been edited to add the 3 responses, including as accurate of detail as possible for deciphering The Machine and Null (in case someone in the future might be able to piece together a variant that can decipher the message for The Glitch, and also because it’s good to have these sorts of details down somewhere concrete)