TotalFreedom Development Update - March 13th, 2023 [TFM Build: Caladrius v2023.03]

  • Hello again TotalFreedom community! We've been hard at work behind the scenes trying to improve our systems in a way that is more in line with current standards. As you all know, we are using an 11-year-old plugin called "TotalFreedomMod". In the past, this was a plugin engineering marvel, because of the things which it accomplished.
    Recently, it has become very clear to us on the development team that pursuing the "True OP" prospect is no longer a viable option.

    Consider the following bullet points:
    - True OP is dangerous as it gives people unadulterated administrative access to root processes such as manipulating NBT data beyond the allowed scope, or utilizing native commands or features that otherwise should not be available to the regular player.

    - We've already circumvented this by taking away a large portion of these privileges, which arguably also removes the idea of "True OP" as you now are imposed restrictions, with still a hierarchy of individuals overseeing your actions.

    - Effectively, in the current day and age, we can have the same exact setup without all the game-breaking bugs, simply by converting our systems to permissions.

    As a result, we've decided, to migrate TFM entirely to a permissions-based system. What does this mean?
    For starters, there is no more "True OP". As previously stated, this system is full of issues, some minor and some severe, but are issues none-the-less. Instead, we are using permissions, which are registered against the Bukkit permissions system and as a result are compatible with any off-the-shelf permission management plugins.
    Don't worry though! For you as a user, everything should remain exactly the same, albeit some features will no longer be accessible. This will ideally prevent things like the armor stand bugs, the sign bugs, the item frame bugs, the painting bugs, the entity nbt bugs, etc...

    This may interfere with the player's ability to create hacked / death / troll potions. On one side, this is good, but on the other, it may be an issue for some of the more beneficial hacked potions that players tend to use. If this is the case, we will find a way to implement those features back (with restrictions).

    Additionally, we have redesigned our discord implementation. It is functionally identical to our current one, however it uses a completely different backend api to accomplish the same tasks. It is also now a completely separate plugin, although it still depends on TFM as we provide some context to the plugin that otherwise would require a lot more development effort to reasonably accomplish. This was done exclusively to improve our implementation's performance, alongside some additional security features which JDA (our current implementation) does not provide.

    We have also migrated the shop out to a separate plugin dependent on TFM, though it is deprecated and slated for complete removal in the future.

    We have also spent time on introducing some QoL changes across the board. Since there's quite a bit to cover on this topic, we will be including the feature changes on our release notes, which will be appended to the end of this forum post when the release candidate is cut.

    Another thing I would like to bring to our attention is our worlds, and why we need to drop them before we can upgrade to 1.19.

    I have explained this in the freedom server before, but I will explain it here, so it is accessible to everyone.

    Here's an explanation of why we need to wipe flatlands for 1.19.3 and why the world sizes increased: With 1.19, Mojang introduced a negative y value for their world generation. First, let's imagine how java processes an integer. Integers by default can take up to 32 bits of space. We can divide this into 8 numbers, appearing as such: 0000 0000 0000 0000. The number 1, for example, is 0000 0000 0000 0001. This can be stored negligibly as 1 bit as it only needs the 1, as all the prefixed 0s are unneeded. Through something called the Most Significant Bit (MSB), we determine how much storage is necessary to store that particular value. With the number 1, our most significant bit is 1. This means we can store a that number as a singular bit in our storage files. However, when you introduce the negative number, our MSB changes. -1 is represented as 1000 0000 0000 0001. This means our MSB is the first 1 in the sequence, which means we need to store all the bits to memory.

    Now, in the context of our world files, this means that for every single y increment of -1 that the world generates, it stores the 32 bits of data in comparison to its minimum. Across thousands of chunks with around 25 million blocks squared across the map (might be larger, either way not good), not including our Y coordinate, this is exponentially larger than previously where integers were strictly positive. This means that world sizes nearly double due to the new Y value rules, and this causes major issues especially with larger world files. For comparison, our flatlands file is around 300GB. If we update to 1.19, it will increase to about 534GB. The total amount of storage space all of the worlds occupy is about 580 gb, currently. If we update without wiping, this will increase to over 1TB of space, which is absolutely outrageous.

    Without a doubt, this is a critical issue and we must wipe our worlds before we can officially update our network to be 1.19.3 across the board.

    I hope this provides valuable insight to the current state of development. If you have any questions please feel free to comment on this post, or dm me [on here or discord].

  • [...] manipulating NBT data beyond the allowed scope [...]. We've already circumvented this by taking away a large portion of these privileges

    I'm not sure I agree with revoking the ability to mess around with NBT. Frankly, I believe this will harm the community more than it will help it because a large portion of this community focuses on the idea of custom or unique items, usually created by messing around with NBT beyond what is normally possibly within the limits of the game.

    This will ideally prevent things like the armor stand bugs, the sign bugs, the item frame bugs, the painting bugs, the entity nbt bugs, etc...

    You see, that's what makes TF so unique though. The unrestricted ability to mess around with items to do things that were never intended by Mojang and would have been blocked by other servers is one of the things that makes TF feel more free than a generic creative server. I don't think it's wise to take away this freedom to push Minecraft to its limits.


    Here's an explanation of why we need to wipe flatlands for 1.19.3 and why the world sizes increased: With 1.19, Mojang introduced a negative y value for their world generation. First, let's imagine how java processes an integer. Integers by default can take up to 32 bits of space. We can divide this into 8 numbers, appearing as such: 0000 0000 0000 0000. The number 1, for example, is 0000 0000 0000 0001. This can be stored negligibly as 1 bit as it only needs the 1, as all the prefixed 0s are unneeded. Through something called the Most Significant Bit (MSB), we determine how much storage is necessary to store that particular value. With the number 1, our most significant bit is 1. This means we can store a that number as a singular bit in our storage files. However, when you introduce the negative number, our MSB changes. -1 is represented as 1000 0000 0000 0001. This means our MSB is the first 1 in the sequence, which means we need to store all the bits to memory.

    Now, in the context of our world files, this means that for every single y increment of -1 that the world generates, it stores the 32 bits of data in comparison to its minimum. Across thousands of chunks with around 25 million blocks squared across the map (might be larger, either way not good), not including our Y coordinate, this is exponentially larger than previously where integers were strictly positive. This means that world sizes nearly double due to the new Y value rules, and this causes major issues especially with larger world files. For comparison, our flatlands file is around 300GB. If we update to 1.19, it will increase to about 534GB. The total amount of storage space all of the worlds occupy is about 580 gb, currently. If we update without wiping, this will increase to over 1TB of space, which is absolutely outrageous.

    TL;DR: If we were to upgrade to 1.19.3 with the current worlds we have now, the size of the worlds would grow to a point where it would be beyond our control due to the differences in how Minecraft stores world data in 1.18+.

    image.png

  • "Beyond the allowed scope" is meant for code restriction not the default numbers that minecraft typically allows; there's a limit to the amount of data we can assign especially to numerical objects such as longs and integers. If users make items that then result minecraft exceeding its bounds then it will crash the server. My intention is not to take away the ability to freely modify and manipulate NBT data, but rather prevent things that could crash the server or otherwise impact the performance or experiences of others in a negative way. I completely agree that free expression through NBT manipulation is one of the things that sets TF apart, and I only want to make sure that we can grant these features to our community without risk of exploitation.

  • I'm not sure I agree with revoking the ability to mess around with NBT. Frankly, I believe this will harm the community more than it will help it because a large portion of this community focuses on the idea of custom or unique items, usually created by messing around with NBT beyond what is normally possibly within the limits of the game.

    You see, that's what makes TF so unique though. The unrestricted ability to mess around with items to do things that were never intended by Mojang and would have been blocked by other servers is one of the things that makes TF feel more free than a generic creative server. I don't think it's wise to take away this freedom to push Minecraft to its limits.


    TL;DR: If we were to upgrade to 1.19.3 with the current worlds we have now, the size of the worlds would grow to a point where it would be beyond our control due to the differences in how Minecraft stores world data in 1.18+.

    "Beyond the allowed scope" is meant for code restriction not the default numbers that minecraft typically allows; there's a limit to the amount of data we can assign especially to numerical objects such as longs and integers. If users make items that then result minecraft exceeding its bounds then it will crash the server. My intention is not to take away the ability to freely modify and manipulate NBT data, but rather prevent things that could crash the server or otherwise impact the performance or experiences of others in a negative way. I completely agree that free expression through NBT manipulation is one of the things that sets TF apart, and I only want to make sure that we can grant these features to our community without risk of exploitation.

    yea ok this will 100% have to be in scissors lol

  • didn't y'all like, already burn that bridge?

    no and even if we did TFM burned that bridge before us

    fairly sure ashaz burnt whatever was left of it when plex annihilated the network for a few weeks. not blaming plex given it was a networkmanager fuckup but i’m confident in assuming NM would take priority over plex.

    52-CEF3-CF-C4-FF-4798-8469-4-BDCA5-D35247.jpg

  • Omg this update has been in the works for years I'm glad you guys finally made good on your plans and managed to convert the systems. I understand that this is for sure going to be a controversial update, as the whole Free OP deal was our whole thing, but I think it's a step in the right direction. By having more control over the permissions and by overall making things less jank, hopefully we can actually *increase* the freedom players have now as we should have finer control over restrictions.

    Now have the ranks been moved to network manager yet 🤪