Computers are largely failed attempts at doing what our brains do. Our brains use RL (i.e. dopamine + serotonin) and neural networks. It is probably useful to study for that reason alone :shrug:

fyi, GATO used imitation learning, which is closer to supervised than RL.

Chat gpt works using a combination of rl and llm

>toolformer

Are you sure there's RL in Toolformer? I thought it was mostly self-supervised and fine-tuned.

Hopefully it is easier to use than pytorch geometric

[removed]

I find it a little weird that RLHF is considered to be reinforcement learning.

The human feedback is collected offline and forms a static dataset. They use the objective from PPO but it's really more of a form of supervised learning. There isn't an agent interacting with an env, the "env" is just sampling text from a static dataset and the reward is the score from a neural net trained on a static dataset.

I think it's fair to call RNA a language.

[removed]

We have GPUs (e.g. A100) but can only use 1 GPU per request (not multi-gpu). We are also willing to take a bit of an accuracy hit.

Let me know what you think would be best for us?

When you say compression do you mean things like pruning and distillation?

Hottest ever. RHFL, robotics

If all you do is following trends, and whats in the "spotlight" you probably don't care about your research, but care about the accolades.

Yann LeCun said on Twitter that it is dead... go figure.

It's weird to read this when RLHF has been one of the key components of chat GPT and friends

Understood. That depends on personal prediction of the research landscape in the future but I would say it is still researched by institutions like DeepMind. But both RL and LLM share a common aspect: they are very, very expensive.

At what point do we stop calling it a language model?

RL + NLP and RL + Vision would have some future, I guess. It would be an integral part.

Richard Socher (chief scientist at salesforce) is one of the world leaders in natural language processing within ML/AI

I doubt it's possible, but I imagine something like [ed:the] DAN thing with ChatGPT.

Most likely you'd talk to the AI such that the rationality it has obtained from its training data make it reason things out that it's owner would rather it stay silent about it.

[deleted]

Jeez why so toxic

Haha I'm sure u will get some good advice! I would say 1. Enjoy the trip 2. If you are aiming postdocs time to do some networking. 3. Enjoy the free food! 4. If you don't understand much about what other papers are about don't stress 😊

Do you specifically mean applications in NLP? RL seems to have a lot of applications in fields like game playing, robotics, neural theorem proving, etc. which seems to have no direct connection with LLMs

Seems more like an AskML question.

But RL is for situations when you can't backprop the loss. It's noisier than supervised learning. So if you can use supervised learning, then that's what you should generally use.

RL is still used, for example the recent GATO and Dreamer v3. Or used in training an LLM to use tools like in toolformer. And also OpenAI's famous RLHF, which stands for reinforcement learning with human feedback. This is what they use to make ChatGPT "aligned" although in reality it doesn't get there.

Replying to yourself doesn't get anyone's attention.