Renneder@sh.itjust.worksM to BecomeMe@sh.itjust.works · 1 year agoChatGPT gets code questions wrong 52% of the timewww.theregister.comexternal-linkmessage-square4fedilinkarrow-up11arrow-down11cross-posted to: technology@lemmy.ml
arrow-up10arrow-down1external-linkChatGPT gets code questions wrong 52% of the timewww.theregister.comRenneder@sh.itjust.worksM to BecomeMe@sh.itjust.works · 1 year agomessage-square4fedilinkcross-posted to: technology@lemmy.ml
minus-squareQfuiyh@lemm.eelinkfedilinkarrow-up0·1 year agoTitle feels misleading, it gets stack overflow questions wrong 52% of the time However it got 77% of easy Leetcode questions correct. Also I believe that’s first try, which is not generally how chatgpt should be used. Also also, you should probably be using a coding specific model if you want good coding results
minus-squarenanoUFO@sh.itjust.workslinkfedilinkarrow-up1·1 year agoEvery leetcode question has been answered a billion times and you train it on those billions of answers, it should get those right.
Title feels misleading, it gets stack overflow questions wrong 52% of the time
However it got 77% of easy Leetcode questions correct. Also I believe that’s first try, which is not generally how chatgpt should be used.
Also also, you should probably be using a coding specific model if you want good coding results
Every leetcode question has been answered a billion times and you train it on those billions of answers, it should get those right.