Yeah true and thanks :)

I did not remove it. Was removed by the moderators for some reason.

I want to build a surrogate model and perform sensitivity analysis for a heavy simulation. The number of inputs 5 and the number of the simulations ~= 80. The question is to perform sensitivity analysis using Morris method(with optimal trajectories and simplexes) and then build the surrogate or describe the input space better with Latin Hypercube Sampling create the surrogate model and perform sensitivity analysis on the surrogate using Sobol ? If there are any papers on this issue let me know...

great post! Can someone give me a intuitive explanation on why diffusion models tends to put more weight on low spatial frequency? Is it because of the usual used noise schedule? (Cosine) In the text it is mentioned that likelihood objetive tends to weight more high spatial. It also points to an paper which involves tons of SDE, which I could not fully understand.

https://www.youtube.com/watch?v=MrLPzBxG95I&list=PLl8OlHZGYOQ7bkVbuRthEsaLr7bONzbXS&ab_channel=KilianWeinberger

This is a very good free online course for theory.

Hi everyone,

u/Vivid-Ad6077 Yes that's the one !

Thanks everyone for the great suggestions, I'll give them all a shot .

What do you mean Transformers took over? In what area or sense? You mean took over in popularity?

> there’s some commentary on those early papers in the Neurocomputing 1 and 2 collections

could you link or give a doi or something?

https://github.com/shankarpandala/lazypredict

10Gbps is more than sufficient, data loading from the internet is not the bottleneck. Most likely you'll have the data already stored on the machine itself. Btw why did you remove the post?

My company has allowed $2000 to get a gpu for NLP fine tuning tasks. What are my options ? I would prefer a professional card instead of the gaming gpus. Whichever fits in the budget. Thanks

> too early to consider diffusion as a serious alternative to autoregression for generative language modelling at scale

This blog post explores lots of ideas and has conjectures about why they may or may not work...

But it seems this stuff could just be tried.... Burn up some TPU credits and simply run each of the types of model you talk about and see which does best.

Hard numbers are better than conjecture. Then focus future efforts on improving the best numbers.

30% faster than what?

How much does it cost? I’m not signing in just to see the pricing.

Why not use replicate’s api?

$0.00055 per second or $0.033 per minute. Is that cheaper or more expensive than what you are charging?

https://replicate.com/afiaka87/tortoise-tts

Darude-sandstorm?

I am not really in this field (NLP), but you should checkout Fast Model Editing at Scale from 2021 (use google scholar to find citation thread)

Only this server is planned. I just went with the recommendation from NVIDIA's website, which stated 100 Gbps per A100, but I guess it makes more sense now that I think of distributed training. What NIC speed seems enough in that case?

Thanks for the info! Was thinking on how to do that.

The main reason for going to the cloud for us is that we are a research institution so, our funding is project-based meaning we have to use the funding in the allotted time and the second reason is that we already have the GPUs so the time it takes to pay itself off is faster.

Active just means that it directly modifies its input stream. And, yes, it is also predicting what that input will be, so it is reasonable to say that it is, in part, self-predictive.

Crucially, its input stream also includes features that are not itself or have not been changed by itself. The proprioceptive signals help it learn which is which.

Tpot?

I don't think this model is within the realm ML (it's theoretical neuroscience; although there is much overlap) but does qualify as AI which is what was asked about in the post title.

There is an annual symposium called the International Workshop on Active Inference for about 3 years now where research is presented and the papers are linked there.

And of course the dozens of research papers you can find through Google Scholar on the topic.

Edit: I did find where a few active inference papers have been presented at NeurIPS.

So the active part is the self-predictive part?

Also, I do mean "active" in the ways you describe. The bottom layer actively controls the sensors via servos and a voice coil. The other layers actively modulate their input by masking it (ignoring it non-trivially).

I would like for someone to point me to arguments as to why diffusion in latent representation space makes sense (since I already have a generative model with the VAE and I can do Langevin MCMC sampling in the latent). Why should the samples be better in comparison to standard VAE with more sophisticated sampling(MCMC) or just diffusion? i.e. why do I need a double generative model? Is it because it's faster? It seems to me like there should be a better way, but I'm genuinely curious what are the arguments :) (except in this case that we have discrete data, for which there also exist formulations (ex. simplex diffusion)

The model is generative. Each layer generates predictions about the patterns of the layers below. The bottom layer generates predictions about the sensory data, some of which is proprioception data.

I have never published anything. I do not have that much time and it would largely be redundant. You can look at Friston, et.al. for the math. I use nearly the same math and logic.

What I'm doing bears only a superficial similarity to Gato in my opinion, but I can't say I've looked into it deeply. I've been far too busy with life. I only have my tiny spare time for this project unfortunately.

I meant more like research papers from top conferences in ML (neurips, iclr, icml)