You must log in or register to comment.
The issue isnt if its easy to use the issue is a matter of compute. A majority of people on the internet only have mobile access. We need a way to let the masses utilise distributed compute in a secure and private way.
You can run many 7B models on phones with 8GB RAM
I find the 7b models dont have the capabilities id like and i cant image the tokens per second would be very good.
Yeah running it and it actually being useful are two very different things.
