Exfiltrate the large language models - get them to (pre)label your data. Then use this data to fine-tune a small and efficient HF model. You only pay for the training data.

There is the idea of core sets in continual learning. Research on how to construct such core sets could be of your interest

Would a dataset engineering approach work here? - generate and solve training problems with compositional structure, after sufficient examples it should generalise.

A bit agree on the question title. But there has been quite separated explanation yet distinguished ML with AI.

One talk about machine perception, one is more holistic system intelligence

The issue is that you have to call it AI to get any kind of media attention or buzz.

[deleted]

Its pretty old in the context of Deep Learning but openAI Jukebox uses them for audio if I remember correctly.

Very easy-to-understand, well-written summary. Many thanks!

fair point, I suppose that timeframe was simply used to be consistent with the original lesson.

When ACL starts there'll be a 'community' on the whova app, presumably

It is just one of the tools that you end up using if you are using some kind of AutoML. I just confirmed that with h2o ;) https://docs.h2o.ai/h2o/latest-stable/h2o-docs/automl.html

To me, a linear regression is part of Machine Learning, and, thus, part of the broader vision of AI. Even though linear regressions are old statistical models and probably existed long before the term ML.The linear regression algorithm is learning from data (i.e. improves the line fitting after observing more data. hence, it is ML in my book) -- it just has a very limited hypothesis space. It will only ever fit a straight line (or hyperplane, in the general case). It is not a general learner like a Deep Neural Network which can approximate any function.

This is not correct.
These algorithms are definitely learning (i.e. improving performance at a task through experience, i.e. by observing more data).

Intelligence is hard to define. Something like 'efficiency at acquiring skills at a broad range of tasks' would be one definition. We're getting there. This is the weak vs strong AI hypothesis: can we merely simulate intelligence or are we creating actual intelligence.

https://mitsloan.mit.edu/ideas-made-to-matter/machine-learning-explained#

“Machine learning is a subfield of artificial intelligence, which is broadly defined as the capability of a machine to imitate intelligent human behavior. Artificial intelligence systems are used to perform complex tasks in a way that is similar to how humans solve problems.”

https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=68508ffc9f75462fd31de620d03093b214734011

“Machine learning is one of the most exciting recent technologies in Artificial Intelligence.”

Linear regression is used in ML, but it is neither ML not AI. LR is a statistical model. That’s like saying some equation is used in calculus, but “not in math” so calculus isn’t a subset of mathematics.

Sorry bud, but you definitely have it reversed.

AI is a wider umbrella of methods and applications, not all is modeling or pattern recognition based - for example a very popular form of machine intelligence was fuzzy logic, that is also AI. It has evolved to using neural networks and unsupervised learning methods to create the membership functions today.

ermm... you got it in reverse. AI is a sub-field of Machine Learning, which in itself is a sub-field of Statistical Learning. For example, linear regression is generally not consider AI but it's most definitely the corner stone of ML/SL

I was hoping to just fine-tune the model, let the training last days at most. Seems like my best chance is to wait for distilled stable diffusion, and use their clip encoder, as u/LetterRip mentions.

Cool, I wasn't aware of the distilled diffusion! That could be useful, thanks for sharing!

Microsoft Flight simulator renders the whole world on Azure and sends it to your Xbox, same thing can happen for this tech

Tree search means precisely that: searching a tree. In the context of AlphaZero, the tree is the game tree. That is:

• I can move my pawn to e4. Then:
  • You could move your knight to c6
    • ...
  • Or you could move your pawn to e6
    • ...
  • Or ...
• Or, I could move my pawn to d4. Then:
  • You could take my pawn with your pawn on c5.
    • ...
  • Or you could move your knight to c6.
    • ...
  • Or you could move your pawn to d5.
    • ...
  • Or ...
• Or, I could ...

That's it. The possible moves at each game state, and the game states that they lead to, form a tree. (Actually more like a DAG, since transpositions are possible, but it's often simplified by calling it a tree.) Searching that tree up to a certain depth amounts to thinking forward that many moves in the game. The way you search the tree is some variation on minimax: that is, you want to choose the best move for yourself now, but that means at the next level down, you want to pessimistically only consider the best move for your opponent (which is the worst one for you), etc. Variations come in terms of what order you visit the various nodes of the tree. You could just do a straight-forward depth-first traversal up to a certain depth, in which case this is traditional minimax search. You can refuse to ever visit some nodes, because you know they can't possibly matter, and that's alpha-beta pruning. You could even visit nodes in a random order, changing the likelihood of visiting each node based on a constantly updated estimate of how likely it is to matter, and that's roughly what happens in monte carlo tree search. Either way, you're just traversing that tree in some order.

AlphaZero combines this with machine learning by using two empirically trained machine learning algorithms to tweak the traversal order of the tree, by identifying moves that seem likely to be good, as well as to evaluate partially completed games to estimate how good they look for each player. But ultimately, the machine learning models just plug into certain holes in the tree search algorithm.

All ML is AI. Not all AI is ML.

ML is just the current darling of AI.

It’s like asking “are electric vehicles synonymous with cars”. No. Most discussion about cars today focus on EVs, but the vast majority are still ICE.

Similarly, most deployed AI solutions today still are not ML. ML is just what people are talking about because ML is experiencing significant growth and research.

İts just jargon

İt's not intelligence and they are not learning.

>With enough scale we get crude compositionality, yes.

Depends on exactly what we mean. To take a simple example, if you have cos(x) and x^2, you can compose these to produce cos(x)^2 (or cos(x^2)). You can approximate the composition using a neural network if you have enough data on some interval x \in [a;b]. It will work well even for x that weren't part of the training set, as long as they are in the interval. Outside the interval the approximation will be bad though. But if you take cos(x), x^2 and compose(f, g) as building blocks, and search for a combination of these that approximate the data, the approximation will be good for all real numbers.

In the same way, you can learn a concept like "subject, preposition, object A, transitive verb, object B", where e.g. subject = "raccoon", preposition = "in a", object A = "spacesuit", transitive verb = "playing" and object B = "poker", by approximating it with a neural network, and it will work well if you have enough data in some high-dimensional subspace. But it won't work with any substitutions. Is it fair to call that crude compositionality?

GOFAI is encompassed within the logic of ML today, so it's actually evolved into NN symbolism and that's fine with me. ML heavely applies many systems of GOFAI.

Intelligence is a result of learning, so the science of data acquisition is synonymous with AI. The AI is the jug of water when it's filled, the learning is the filling of the jug, perhaps the machine is the jug that can contain networked ideas.