[removed]

Are your friends?

edit: oh wait, you already told us they're not.

r u ok hun?

Why? I don't care about your friend's feelings.

This comment was a fine addition to the discussion until you thought you could tell me what to do.

[deleted]

Diffusion models can also generate exact likelihoods so maybe we'll see a shift to those in the future

You'll probably be fine learning transformers directly, but a better understanding of RNNs might make some of the NLP tutorials/papers containing transformers more easily comprehensible.

Attention is an very important component of transformers, but attention can be applied to RNNs, too.

Hey dude, this caught my eye before realising I spoke to you about this in person! I’ll have a play with it.

There are some papers showing diffusion working better for high-dimensional data in likelihood free inference, even just using an elbo bound. Can dig up later if wanted

Im not sure if they vary the sampling hyperparemeters. The point is that langauge modelling objectives are to some degree ill-posed because we calculate the loss on intermediate results rather than the final output that we care about.

I think it's probably the non-differentiable nature of the sampling techniques. If it's just about limited training data and using the reward model, in that case you can also use weakly supervised learning with that reward model.

Good point, so you mean they incorporate things like beam search + changing temperature, top-k sampling, and nucleus sampling in the RL PPO-based optimizaton?

Iterative as in continuing until there’s no more neighbours left as you continuously add neighbours to your index and query?

Explained in my forthcoming book:

https://udlbook.github.io/udlbook/

Should be a good place to start, and if it isn't then I'm really interested to know where you struggled so I can improve the explanation.

I would even say machine learning is not the be all and end all of solving problems with data.

OS. Kernel. Bus. Processor. Transistor. p-n junction

Thanks for the link!

Qdrant has a recommendation API that allows doing exactly what you want, I suppose: https://qdrant.tech/documentation/search/#recommendation-api

Yes ! You only need to change the last classifier layer (and initialize the added weights) to add more outputs and then further train the model on data containing all the classes (including the new ones)

Hey ChatGPT can you phrase this [sentence] to be politically correct?

That was an interesting read, but I don't think it solves my problem. Their examples don't show joint vector searches: https://github.com/google-research/google-research/blob/master/scann/docs/example.ipynb

I mean pass your queries through a self attention layer and then some fcns and have it output your final query vector

What about marking samples as "irrelevant"?

What does it mean for a vector to attend to another vector?

I mean the more you understand the better obviously. But it's not necessary, it's just context for what we don't do anymore.