• 0 Posts
  • 5 Comments
Joined 1 year ago
cake
Cake day: December 14th, 2023

help-circle


  • and it turns out simply making models bigger does not lead to better outputs.

    I’d say that’s debatable though, as what we have seen so far could just be that scaling with the current “low quality” data might not be enough. So, just like R1 might have been impossible earlier before there was enough high quality data for RL to work we might still be a ways of of having good enough data for huge models.

    If that was the case that is kinda of a plateau, but a temporary one that could be raised once other things are improved enough. Who knows for sure though.