No problem! Thanks for the insight regarding its capability and costs

I appreciate the insight. I didn't know that it would be that expensive!

So you're saying that even if magically somehow OpenAI were to close shop, I could still just jump ship and use another API and I'll probably only need to very slightly modify the code accessing it since they should be able to handle the same prompts?

They become self replicating.

That is good.

Unfortunately I have not enough knowledge to answer that question.

Your laptop will not begin to suffice, not for inference and especially not for fine tuning. You would need something like an A100 GPU in a server that handles requests. And in the end, the results will be much worse than GPT-3. If you dont already have an AI infrastructure, go with an API, it will save you more than a bit of money (unless you are certain you will use it at scale long-term). If you are worried about OpenAI, there are some other companies that serve LMs.

So if I was handling something like 2000-10000+ requests per day for my business, locally isn't going to cut it?

I'm not a professional, but I can answer some of your questions as my personal opinion.

How good is it at writing short stories?

- I don't think GPT-J is dramatically better than the others, especially for text generation. I often see hallucinating, illogical, misconceived text generation. If you want a result like OpenAI's Davinci-003, you may be disappointed despite your fine tuning.

How resource-expensive is it to use locally?

- You need 40GB+ RAM if you're running on CPU. One of my friends has failed on her 32GB RAM and she had to increase her swap memory, then she succeeded with an extremely slow loading time. ( Almost 7~8 minutes ) If you want GPU power, VRAM with float16 need 32GB+ VRAM ( I saw someone using on 24GB ). CPU generates a text from a prompt in 30~45 seconds whereas a GPU generates a text from the same prompt in 3 to 5 seconds.

How do you deal with source code hosting?

u/jsonathan Shoot me a dm, I'd love to do a VSCode ext.

ML is not only just the backend. Technically you can code and run ml programs on OpenCL or OpenGL, but speed will be at least x2-x4 worse than specialized backend like cuda / rocm.

It's all about tuning programs (such as matmul) for each GPU model to achieve maximum performance. CUDA/Rocm already contains tuned programs.

Understand, thanks!

LOL. No kidding. Bit then sometimes I look at it and get a little bit nervous, since they also write code!

Don't give me hope like that.

An AI debugger would be very helpful. I actually see that as a use case...

I'm only peripherally involved with ML/AI in that I try and apply some helpful techniques to biological problems but recently I have enjoyed listening to discussions around AGI but most of the papers I've come across from a quick google scholar search seem to be *about* AGI and not attempts at implementing something closer to, or approaching, AGI.
Is that a fair assessment? Has my lack of depth in the field given me a false initial glance?
Are there any authors/labs working on AGI in particular whose papers you would recommend reading?
e.g. "Artificial General Intelligence vs. Industry 4.0: Do They Need Each Other?", "Deep Learning and Artificial General Intelligence: Still a Long Way to Go", "Why general artificial intelligence will not be realized", and, "Approaches to Artificial General Intelligence: An Analysis", all seem to be about AGI in contrast to, "Towards artificial general intelligence via a multimodal foundation model", which attempts to implement something.
Full disclosure: I haven't read these papers yet. I am trying to find good, reputable papers to read.

My goto model (after a linear model) is usually XGBoost. It's nice to see that the theoretical potential of a tree based model is as high as neural network. Though that's not necessarily true of XGBoost's boosted trees, they do perform well, and I really like how fast their training time is.

Just wait till you see what the rate for management will be after LLMs come for their jobs.

Managers are mostly people-interaction-managers, and LLMs are already 10x better at that than they are at creating novel code.

Can you turn this into an IOS app

Have you looked at other related papers? In particular, hypergradient descent and papers that cite that work (which you can find via google scholar)?

OK, how did you evaluate it? How do you tell it's working well or not?

Anecdotally, it is comparable.

Well for one, he's not using ChatGPT. GPT-3 is not the same.

Awesome thank you!

This is really good mate.