Do not ever push to production without testing things first. I went and moved us to the beta branch because this commit caught my eye and I wanted us to have the emoji picker fixed: https://github.com/LemmyNet/lemmy-ui/commit/ae4c37ed4450b194719859413983d6ec651e9609

The beta branch was on dockerhub so I thought that it was at least minimally tested. I was sorely mistaken. Lemmy (the backend) itself would load, but lemmy-ui (the actual website that renders everything) would keep crashing when attempting to load. I couldn’t roll back because the database was migrated to this beta version and it couldn’t migrate back to the old version when attempting to launch said old version.

I had no choice but to restore from backup. We’ve lost a whole day’s worth of posts (anything after 3AM CST.) I’m really really sorry… blobcatsadpleading

I was just so excited to be able to unveil this, I didn’t take my time with actually testing it.

  • Burger@burggit.moeOPM
    2·
    1 year ago

    Right. I thought I was restoring the most recent backup, but it turned out that the order for the archive list was in ascending order. I picked the top one thinking it was the latest. So that’s why you saw 8 days when Burggit was back up for a split second.

    My server automatically takes a backup once daily in the early AM, and keeps (I think? I’d need to check) 7 on hand. Any older than that, and they’re pruned.

    • CookieJarObserver@burggit.moe
      2·
      1 year ago

      You should pick one day a week that is stored longer, if there is a repetitive issue it might corrupt over more than a few days.

      Man i thought i did something wrong when the posts from the last week where gone

        • CookieJarObserver@burggit.moe
          2·
          1 year ago

          Yeah had a problem once where a backup (made one once a day and keep 14) was overwritten by a corrupted data set and i basically lost most data, would be sad if that happens here if its avoidable👍