Ah, I'd not seen the Block Recurrent Transformers paper before, interesting.

it just depends on the available data. someone has to tag those films with the appropriate info youre interested in. the modeling isnt necessarily any different.

Appears to be here: https://openreview.net/forum?id=ibNr25jJrf

edit: actually after reading it, I don't think this is the referenced publication, but it's still interesting

They're black boxes in a sense that it's hard to oversee all of the activations together. But it's very easy to understand what each neuron does, and you can even check outputs at each layer to see what's happening inside.

Look you sound like you went to an online course and have a basic understanding of basic buzzwords but have never studied the topic in depth.

Lol if you think rocks might be sentient, there's no way I can make you understand why LLMs are not.

You're even wrong on sentience and consciousness, for once you keep mixing these two concepts together which makes it harder to converse, as you keep changing what you're discussing. And then again, we do have a definition for sentience, and there have been studies that have proven for example in multiple animal species that they are in fact sentient, and zero studies that have shown the same on rocks. Even the notion is idiotic.

We don't know the supply demand curve, so we can't know for sure that the revenue increased.

Hah! Not even close, they're almost black boxes.

But even if we did, that wouldn't help us tell whether or not they're sentient, because we'd still need understand to sentience. For all we know everything down to dumb rocks could be sentient. Or maybe I'm the only conscious entity in the universe - there's just no data.

You got a point there! I haven't given it too much thought really -- I def need to check out ada.

But wouldn't the ChatGPT embeddings still be better? Given that they're cheap, why not use the better option?

This comment make me wonder if the same rules about using one-hot encoding instead of ordinal encoding for classifiers still apply to a neural net trained with a gradient-less search algorithm like a GA instead of backprop.

Who says genetic algorithms are dead? They're pretty much dead for training neural nets absolutely, but there are tons of other more general optimization problems that GAs (or more generally evolutionary algorithms) are well suited for.

Not to mention they still have plenty of utility as a search algorithm for hyperparameters so they aren't even dead for neural applications.

[deleted]

I use CMA-ES (a type of evolutionary algorithm) for training neural networks for finance stuff. The neural networks involved are not superhuge so it works out (IIRC the number of parameters is around ~500-1000).

The fitness function is pretty complicated and written in Rust and I put a lot of effort to making it fast because these algorithms need to evaluate it many many times. I feel using evolutionary algorithms makes coding simpler because you do not need to care that whatever you are writing is differentiable or that some backprop/gradient descent library needs to be able to "see" inside your function.

I do think my use case is a bit more niche. I live in hope that some breakthrough happened that made evolutionary algorithms practically usable for large neural networks.

It was. Dumb too. Because of the limitations of memory and computing power at the time you had to constantly tweak parameters to optimize learning speed, avoid overfitting, avoid local optimums, etc. Only to find that the best performing model was the one generated by your 2 AM code with the fundamental flaw and the random parameters you chose while high.

I’d say we need an /r/VXJunkies equivalent for statistical learning theory, but the real deal is close enough.

I think there are few problems were a couple extra assumptions that could make much more efficient methods work (not NNs necessarily of course) don't hold. I'm not sure there exist problems where genetic algos outperform other methods, disregarding problems where only genetic algos work.

Not necessarily. Bayesian methods work great when you have expensive objective function evaluations that can only be evaluated in serial (or limited parallel evaluations). Bayesian methods aren't ideal in massively parallelizable evaluations (evaluating >100 points at a time) or when evaluations are relatively cheap. It depends on the cost of optimizing the acquisition function. I've actually played around with combining BO with evolutionary algorithms to extend BO towards massively parallelizable evaluations and have seen some promising results.

TWIML as others mentioned, Eye On AI also

GCP, speechmatics, rev, otter.ai, assemblyai etc. etc. offer similar or better performance, as well as streaming and a much more rich output.

They might see a resurgence in dynamic multi-agent environments.

Yeah, I get that's that embeddings are used for semantic search but would you really want to use a model as big as ChatGPT to compute the embeddings? (Given how cheap and effective Ada is)

Nice... 😄

You can use the embeddings to search through documents. First, create embeddings of your documents. Then create an embedding of your search query. Do a similarity measurement between the document embeddings and the search embedding. Surface the top N documents.

[deleted]

It sounds very interesting. Is it available to read? Thanks

Hi! Sure, here it goes:

It's a course about making AI models that can create images. These models can that by learning from a dataset of example images. "Diffusion" is a new type of AI model that works very well for this task.

The course will work best for those familiar with training deep neural networks for generative tasks, so I would advise catching up on topics like VAEs or GANs. However, the video course material is quite short (about 1,5 hrs) so you can just play it and see if it works for you or not!

I think they're still useful for "no information at all" scenarios where attempting a solution is just too time consuming or not possible using other methods (e.g. traveling salesman problem).

As a practical matter, I think they're best integrated with other methods as "first cut" solutions that get you closer to something you can work out with a neural net or rule based system.

That said, I'm unaware of any NN or rule based solution better than a GA for solving the traveling salesman problem even now. So, maybe some P-NP problems will always be best attacked with GAs.