Recent comments in /f/MachineLearning

ilyakuzovkin t1_jazwvuh wrote

Reply to To RL or Not to RL? [D] by vidul7498

I think RL is a niche by definition, but that's not a bad thing. If the problem you want to solve is about agents operating in interactive environments and maximizing some kind of utility function along the way - surely RL is your workhorse here.

Over the course of the last years we have seen successful applications of RL outside that narrow field of problems, where a problem that is seemingly not about agents and environments can still be formulated as an MDP and then solved with an RL approach. Because of these examples there seems to be a looming sentiment that RL is somehow "instead of" supervised, and questions like "which is better RL or supervised" arise.

My take on this would he that both are applicable in their appropriate spaces of problem formulations. Some problems are made to be solved with SL, some other ones with RL. And while it is feasible to twist an SL problem into RL framework, or even vice versa, it does not imply that one or the other is the ultimate tool.

Same way as one wouldn't use RL to multiply two numbers (except for academic interest), one should not use RL if it is not the right framework for the problem at hand. But for some other problems RL will definitely be (and already is, like in Go, Chess, Startcraft) the future.

16

currentscurrents t1_jazwqft wrote

Reply to To RL or Not to RL? [D] by vidul7498

The reason you want to do RL is that there's problem scenarios where RL is the only way to learn the problem.

Unsupervised learning can teach a model to understand the world, and supervised learning can teach a model to complete a human-defined task. But reinforcement learning can teach a model to choose its own tasks to complete arbitrary goals.

Trouble is, the training signal in reinforcement learning is a lot smaller, so you need ridiculous amounts of training data. Current thinking is that you need to use unsupervised learning to learn a world model + RL to learn how to achieve goals inside that model. This combination has worked very well for things like DreamerV3.

18

rumovoice OP t1_jazl88i wrote

Reply to comment by ke7cfn in [P] LazyShell - GPT based autocomplete for zsh by rumovoice

> download and build sshfs for an m1 mac

its answer: git clone https://github.com/osxfuse/sshfs.git && cd sshfs && ./autogen.sh && ./configure && make && sudo make install

it doesn't do well in cases where it needs some recent knowledge like m1 issues

1

chungexcy t1_jaz8tm4 wrote

Reply to [R] High-resolution image reconstruction with latent diffusion models from human brain activity by SleekEagle

>Figure 3 shows the results of visual reconstruction for one subject (subj01). We generated five images for each test image and selected the generated images with highest PSMs.

Something is not quite right. When they select the generated image, they use PSM score to select the best in 5. To calculate the PSM, I believe you need the original image (target, ground truth). It's like the LDM gives you five choices and you use your target pick the most similar one and then claim that this one is similar your target?

1