Recent comments in /f/MachineLearning

Acceptable-Cress-374 t1_j67w859 wrote

Reply to comment by feloneouscat in [D] Do you know of any model capable of detecting generative model(GPT) generated text ? by CaptainDifferent3116

That was my first try. I went with the gut feeling that any training that they used for their model would assume bland prompts. I made mine different, and got 97% human generated the first try. Someone else mentioned other things that you could do, like mess around with temperature and such. Those work as well.

1

marcingrzegzhik t1_j67vr5l wrote

Reply to [D] Laptop recommendations for ML by PleasantBase6967

It depends on the scope of your projects. If you're only training small models (like GANs, CNNs, etc.), then a decent modern laptop with 8+ GB RAM and an intel i7 or Ryzen 7 processor should suffice. GPUs are nice to have, but with an Intel i7 or Ryzen 7 processor you can do most of the work without them. As for the OS, Windows and Linux are both fine, but I'd recommend Linux for ML projects for maximum compatibility. Hope this helps!

0

jimmymvp t1_j67uru2 wrote

Reply to [D] score based vs. Diffusion models by Individual-Cause-616

Diffusion models are effectively score-based, there's a connection with the reversal of the forward process being Gaussian and the noise estimate, effectively you're using scores of Gaussians in the reverse process. The time variable is irrelevant in sense of scale, the discrete time and continuous time essentially do roughly the same, the difference is that one is tied to a specific discretization of the SDE and the other can be solved to arbitrary precision, it's also a difference if you take steps wrt to variance or wrt time. Essentially the continuous formulation should be the limit of the discrete one. So effectively you can take a discrete sampling method and make it a continuous SDE/ODE

2

marcingrzegzhik t1_j67tza8 wrote

Reply to [P] Using algorithms or models from papers for commercial use by romantimm25

No, the license does not mean you cannot use the ideas from the paper in a commercial product. It just means that you cannot use the work itself or any derivative works for commercial purposes. However, you can use the ideas from the paper, as long as you don’t directly copy or use any of the code/materials from the paper. To be safe, you should also make sure that you don’t infringe on any patents associated with the paper.

2

marcingrzegzhik t1_j67s9fp wrote

Reply to [D] Could forward-forward learning enable training large models with distributed computing? by currentscurrents

Forward-forward learning is a very interesting concept, and I think that in some cases it could definitely yield better results than distributed learning with backprop. It really depends on the size of the model, the latency of the connection, and the bandwidth of the slowest machine. I'm sure that in some cases it could be much faster, but I'm curious to know if there are any other advantages to using forward-forward learning over backprop for distributed learning.

2

currentscurrents OP t1_j67lie8 wrote

Reply to comment by master3243 in [D] Could forward-forward learning enable training large models with distributed computing? by currentscurrents

I'm messing around with it to try to scale to a non-toy problem, maybe try to adapt it to one of the major architectures like CNNs or transformers. I'm not sitting on a ton of compute though, it's just me and my RTX 3060.

A variant paper, Predictive Forward-Forward, claims performance equal to backprop. They operate the model in a generative mode to create the negative data.

6

master3243 t1_j67jwad wrote

Reply to [D] Could forward-forward learning enable training large models with distributed computing? by currentscurrents

Hinton says that it does not generalize as well on the toy problems he investigates. An algorithm not doing well on toy problems is often not a good sign. I predict that unless someone discovers a breakthrough, it will be worse than backprop despite operating faster (due to not having the bottlenecks as you suggested).

13

deepestdescent t1_j67iajc wrote

Reply to [D] ImageNet2012 Advice by MyActualUserName99

I use PyTorch data loaders to load batches into memory in the background. I believe TensorFlow has similar functionality with tf.data. This should make your data loading speed basically negligible if you have a few CPU cores lying around.

1

yauangon t1_j67hram wrote

Reply to [D] Simple Questions Thread by AutoModerator

I'm trying to improve a CNN encoder, as a feature extractor for an AMT (automatic music transcription) model. As the model must be small and fast (for mobile deployment), we are limited to about 3-6 layers of 1D-CNN. I want to improve the encoder with residual block (of ResNet), but my question is: I don't known if Residual block would benefit on such a shallow CNN architecture? Thank everyone :D

1

knestleknox t1_j67ezg0 wrote

Reply to [D] MusicLM: Generating Music From Text by carlthome

As someone who works a lot with both music and ML, I'm really excited to see these multi-modal approaches. The image description -> music generation was really cool to see. But it would be incredible to see a (good/large) multi-modal model that can go from audio -> image. Free album artwork and visualizations for all my songs.

3