Awesome results! Would love to see a comparison with other accelerated evolution methods (eg https://github.com/google/evojax)

Scores: 5, 3, 2 The one with score 2 gave a very vague review. Hard to rebute anything or understand what the reviewer is really contesting. One with score 3 seems to be willing to change score if major points are convinced. What are my chances?

Pick the tools that work for the problems you have. If you are online training a model on an embedded device you need something optimized for that hardware.

I gave you a generic example of a problem domain where this applies. You can search for online training on embedded devices if you are interested but I can't talk about specific applications because they are not public.

All I'm saying is drawing a line in the sand and saying you'd never use X if it doesn't have Y is silly because what if you end up working on something in the future where the constraints are different?

Can you give me one example of this? And even if. My point is still valid because I did not say that you should delete the CPU support lol

I see, thanks! This seems like a great resource. Thank you for making it available. I’ll post any further questions here, unless GitHub is the preference.

Okay, you're picky :p

Try deploying a model for realtime online learning of streaming sensor data that needs to runs on battery power and then insist it needs to run on GPUs.

Plenty of legitimate use cases for non GPU ML.

Thats such a tenuous distinction and you're wrong anyway because you can pose any learning from data problem as a generic optimization problem.

They're very useful when your loss function is not differentiable but you still want to fit a model to input+output data pairs.

They're also useful when your model parameters have domain specific meaning and you can derive rules for how two parameter sets can be meaningfully combined with one another

Decision trees and random forests are ML too. What you probably mean is Deep Learning. But even that has a fuzzy boundary to surrounding methods.

Being a prescriptionist with these definitions is a waste of time because the research community as a whole cannot draw clear lines in the sand.

Looking at an ML Engineer role with the following qualifications:

"Strong experience in the area of developing machine learning training framework, or hardware acceleration of machine learning tasks"

"Familiar with hardware architecture, cache utilization, data streaming model"

Any recommendations for books/resources/courses in this area? How does one begin to develop these skills?

Thank you for your answer. If a paper on diffusion models pops into your mind that uses this method, feel free to post it.

How would you derive a quantitative evaluation from t-SNE? I thought it's mostly used for visualisation. I'm looking to compute some kind of score from the interpolation.

They not about learning from data, they are about optimization. They are from the broader AI field of study, but I wouldn't say they are ML. They serve a different purpose. Even though there are some research about using them to optimize models (instead of using gradient descent), but it's not their main use case.

NEPTUNE

They are not gradient-descent based (so they don't need GPU acceleration as much, but sometimes times still do depending on the problem) but they are definitely ML.

Have you checked https://dvc.org/ ?

another underperforming linear transformer again? ¯\_(ツ)_/¯

I'm not working on NLP but I have seen your idea in papers on diffusion models. You are basically linearly interpolating your latent space. There are other interpolation techniques you could try, but your idea will definitely give you some insight into your latent space.

Another possibiltiy would be some kind of grid search through the latent space, tho depending on your dimensions it could be too hard.

Lastly, you could visualize the latent space by projecting it into 2 or 3 dimensions via t-SNE or something similar.

Thanks, but I guess I should also mention I was looking for a free solution

CMA-ES! definitely playing around with this, thank you!

I've seen that before when the large range features were the most important for the clusters I wanted. It was essentially doing feature weighting but it was implicit in the scales

Evolutionary algorithms are not ML.

Call me picky, but I would not use a ML library that is not GPU accelerated. This should be default

You don't have to be Google to use special-purpose hardware for machine learning, either. I work for a company (Groq) that makes a machine learning acceleration chip available to anyone. Groq has competitors, like SambaNova and Cerebras, with different architectures.

Also, Google doesn't use GPUs, they designed their own cards which they call TPUs.

TPUs are ASICs designed specifically for machine learning, they don't have any graphics related components, they are cheaper to make, use less energy and can make as many as they want.

the guy above was kind of unclear, its an autoregressive langauge model so it does generate one at a time, puts it back into the input and generates the next one. It could be printed out in one go once they waitied for it to stop and then be sent to the client and pritned all at once but they went with the fancy GUI type, possibly yeah as a way to slow down spamming

This is really cool.

АХАХАХАХХАХ