Server Size

I've updated with data as of today.

The good news is the work we're doing to down-size the CoreProtect Database on Freedom-01 seems to be working. the bad news is we've still got a lot of data to delete before it'll be back at a manageable size.

For anyone interested, I've plotted the data on to a chart. This one shows a forecast of what the storage sizes would look like over the coming months unless we do something.

And this shows the total storage use by month to date.

The reason for doing this now, is because we're finding that Freedom-01 is now so large, it breaks the backup solution which means it's iffy when TF does / does not get backed up... That's why we're keen to start looking at ways to split the worlds out and hopefully prevent this from getting too much worse, along with purging CoreProtect data from the server and ageing it off, which I don't like, but unfortunately needs to be done.

@Erin#18285 Some of it was, but it didn't really make a dent. I'm currently working through it 70000000 rows at a time, which tends at the moment to be about a couple of days. Current plan is to remove everything from before April iirc.

For those interested, I've saved the spreadsheet and will be keeping that up to date. I will update this thread as well, but it is already getting a bit hard to follow.

For those interested the spreadsheet can be seen here - https://atlasmediadev-my.sharepoint.com/:x:/g/personal…Hl7iCQ?e=7j4tdC

@StevenNL2000#18309 Partially, it's a combination of factors.

Because Freedom-01 and the DB server are so large, server capacity is already tighter than would be ideal (We tend to sit around 85% disk usage on the underlying host with everything else running, and a significant chunk of that is TF).

The backups work by effectively cloning the server and then tarballing the clone, so you need the headroom to run effectively a duplicate Freedom-01 server all the time, which we generally have, but you then also need enough space for the tarball, which is where the issue I think starts to come in, and because it then hangs like that all day until the next backup cron runs, it ends up filling up the entire usable disk.

By splitting the servers out and hopefully significantly shrinking the DB size we should be back in a position where it's easier to backup, and also easier to restore... Restoring Freedom-01 to pull a sub-set of the VPS out is a PITA right now and it a multi-day activity, which it really need not be.

@videogamesm12#18347 I could, but that isn't how the SN backup system works, and I'd rather not be building bespoke backup solutions for a single VPS, when in reality the sizing is just broken...

If it were a client server it'd be different, I'd need to sort something out to make it right, but the bottom line is TF uses up over half of SN's sellable space, which just is not a sustainable business model at the moment, we need to find a way to reduce the amount of storage space we're using because this is just silly tbh.

I'll also add, the reason for splitting the server out into multiple servers was never originally about size, but more because the merging of gamemodes has effectively meant a lot of the time we end up with the worst of all the gamemodes... Like the plot world could have a lot less restrictions because of the nature of plots, but because that doesn't work outside of the plot world it's a lot harder to manage. The dream is to enable the plot world first at least, to have a lot more plugins which can be managed using normal permissions, and just set them up to be pretty much as permissive as we can get them, that way we don't have to deal with the current rubbish that is modifying plugins to deal with OP perms.

Data added as of today as I suspect I will forget to do it at the start of June, and thought it would be interesting for folks to see the rate of increases / decreases.

@mattlawn#19053 would absolutely tank the performance of the server... That's the issue.

@Folfy_Blue#19058 It would make core protect unusable. We had major issues when the database was on a hdd not SSD, basically meant the plugin would hang and roll backs would take hours. Putting it onto some sort of external cloud storage would make it considerably worse to the point of it being better to have just deleted the data entirely.

@mattlawn#19069 nope. It'd break the world's or the databases.