Definitions shift a bit, and people disagree, but this is what I stick to...

AI: Any system that responds 'intelligently' to its environment. A thermostat is, therefore, AI.

ML: A system that gets better at a task with more data.

Therefore ML is a subset of AI, one specific way of achieving the goal.

So they invented a new term for neuro symbolic computing?

Correct me if I am wrong AI: Niche part of ML ML: AI + Data Science

Edit: An “intelligent” computer uses AI to think like a human and perform tasks on its own. Machine learning is how a computer system develops its intelligence

https://azure.microsoft.com/en-us/solutions/ai/artificial-intelligence-vs-machine-learning/

lol no

If you have multiple GPUs with each around of 20G VRAM on your hand, you may try from the training. My question is this, does it worth it with an enterprise level of resources and a lot of time? Why would you choose this way instead of picking a pretrained model and finetuning specifically on the code generation from English. OpenAI's GPT-3 Davinci-003 does fairly good code generation from English but sometimes the result is a bit clunky. Therefore you still want to finetune it. They claim davinci-003 has hundreds of trillions of parameters.1.3 billion parameters of OPT or GPT-Neo need 8GB VRAM to just load the model. To finetune these 1.3 billion parameters, you need 16GB VRAM. You can probably do it as a single person.

its better to use a pre trained model and maybe further pretrain it to your needs imo. Try CodeT5. Its a decent model.

I view ML as a subset of AI.

AI has been around since the 50's, it just means appearing intelligent. ML is a subset of AI where algorithm outputs achieve the same initial goal of AI.

Not sure if you're tagging to warn me about him using my reddit name in his tweet, but I'm not really bothered (although I do appreciate your tag just in case, as I hadn't seen the edit). Did not expect this kind of childish reaction to what I thought was my, your, and several others' constructive (albeit sometimes a bit blunt) criticism, though - yikes.

it takes data, and evaluates that data against a performance metric, and then adapts the structure in response, creating new data.

From what I've seen, it's a promising field that should be possible. But so far but nobody's made it work for more than toy problems.

I don't think it's quite as arbitrary as you're making it out to be. I haven't perfectly defined the concept of a task here, but a core concept of ML is that it's focused on the the learning itself rather than producing a solution to some problem statement. The idea of learning implies an element of generalization, but that's different than general applicability or usefulness. The agent working on the task is our abstraction layer; our algorithm should work on the agent rather than producing the solution to the agent's task. Through some defined process, you're to create a generalized form of knowledge in that agent, without solutions for specific cases being explicitly programmed.

If you train a NN to generate a representative knowledge model that solves a "simple" problem that could have been solved with an explicit solution, you're still doing ML. It's not about how complicated the problem is to tackle, or how generally applicable or useful the result is, but whether what you have explicitly programmed is the direct solution to some problem, or is itself a modeled form of learning that can be applied to some agent that can then go on to solve the problem.

In the Strandbeest example, the program that is running is not modeling any learning. There is no agent. The output of the program is a direct solution to the problem rather than some embodied form of knowledge an agent might use to solve a generalized form of the problem. It's not ML and it's not a fuzzy question, at least in this case, imho. There could be perhaps be cases or situations where there is more fuzziness, but this isn't it.

Optimization, including heuristic optimization as in genetic algorithms, could find applied use in ML, but they are not themselves ML, and the use of a genetic algorithm to solve a problem explicitly is not ML.

It’s definitely infeasible to train and run inference on your own for a large language model. You would need many datacenter gpus. But you could maybe create an application that interfaces with a chatgpt api (or some other api accessible LLM)

How so?

Thank you, I will check those out.

Diffusion’s lack of causality constraint seems like a pretty tall hurdle for tasks with output formats requiring “fluency” (like summarization) though. Kind of like drawing hands early on in stable diffusion (or drawing most anything coherently for earlier models like disco diffusion). Multiple-choice question answering seems like a more natural domain, though certainly doesn’t show off the “expressive” generative abilities. Fluency probably improves significantly with scale and fine-tuning though.

[deleted]

Yeah no

That makes sense. After all it doesn't make sense to call something Artificial Intelligence if it doesn't act intelligently. I feel like it boils down to when the machine/neural net/function has learned to do a task which has reached a certain level of general applicability/usefulness then it can be considered AI / Machine Learning. And it makes sense that people draw this line at different points and disagree about where it should be drawn.

For example if you train a car to drive around a track but the track is a fixed width it's possible that you trained only a single parameter - the amount which the car should turn based on the difference between the distances to the left and right wall. Once that number is dialed in the car will be able to handle any track of that fixed width and will look pretty smart, but it could have been achieved with a simple function instead of a neural network.

I've heard similar concerns raised with AI related to Radiology for cancer screening since there is no way to actually know what factors the neural network is considering and how then it's possible that it was making the judgement based on something completely unrelated to the cancer. I tried to find a source for that but hopefully you get what I mean, basically just the black box problem.

The paper does talk about this and calls transformers "first generation compositional systems" - but limited ones.

>Transformers, on the other hand, use graphs, which in principle can encode general, abstract structure, including webs of inter-related concepts and facts.

> However, in Transformers, a layer’s graph is defined by its data flow, yet this data flow cannot be accessed by the rest of the network—once a given layer’s data-flow graph has been used by that layer, the graph disappears. For the graph to be a bona fide encoding, carrying information to the rest of the network, it would need to be represented with an activation vector that encodes the graph’s abstract, compositionally-structured internal information.

>The technique we introduce next—NECST computing—provides exactly this type of activation vector.

They then talk about a more advanced variant called NECSTransformers, which they consider a 2nd generation compositional system. But I haven't heard of this system before and I'm not clear if it actually performs better.

Ha ha

ChatGPT? Is that you there?

I think LLM and large transformer networks are basically finding structure and composition in raw data

And there’s some (as yet unknown) way to get rote symbolic manipulation by stacking some similar system in top of them— similar to how LLM guides diffusion models work

Compositionality is increasingly a significant area of concern across many subfields of deep learning. Winoground recently showed that all state-of-the-art vision-language models drastically fail to comprehend compositional structure, a feature which many linguists would argue is fundamental to the expressive power of language.

Smolensky is also a great guy and was affiliated with the PDP group that developed backprop in the 80's. The best path to neurosymbolic computing & compositional reasoning remains unclear, but Smolensky and his student Tom McCoy have done some great work over the last few years exploring how symbolic structures are implicitly represented in neural nets.

i guess hybrid AI have a chance though

You can use all kinds of algorithms in machine learning. This is a “uses a” relationship rather than an equivalence relationship, in this case. If I’m building a piece of furniture, I am a carpenter. I could employ the use of a hammer to help me build the furniture. The hammer is not a carpenter.

I think you can imagine that the machine learning approach in that video may also rely on various data structures including graphs, trees, etc, and perhaps many other things which are also not machine learning.

Your statement is incorrect. When people (in the field) say ML they mean the whole toolbox of learning algorithms, incl. Deep Learning, trees & forests, kernel methods, etc.