Recent comments in /f/MachineLearning

keepthepace t1_j7jgm75 wrote

Google has been the biggest team player when it comes to publish advances in AI. OpenAI has been the worst: AI research paper of big players.

Most of the techs that made ChatGPT possible were published by Google. Worse: OpenAI does not publish the 1% of things that makes ChatGPT unique (though we know enough to have a pretty good idea of what they did).

I'd be whiny in their place as well. The GPT family is not super innovative, they just ran away with an architecture mostly made by Google (Transformers/BERT), stripped it of everything that prevented huge parallelization (which many suspect included things that would allow it to stay "grounded" in reality) and slapped more compute on it.

30

mskogly t1_j7jevm7 wrote

I have a theory that human imagination/creativity is linked to our dreams, and that we learn and change faster because our different brain halves play off scenarious to each other to test them out. our internal dreamworld can suspend and jump over the limitations of the physical world (like time, place, senses), but still manage to improve how we understand and interact with the world when awake. I think a better understanding of the human brain and especially dreams is needed for the next big leap in machine learning, instead of the brute force techniques used now to train static models.

0

user4517proton t1_j7jda78 wrote

I'm not surprised. Honestly, Google is caught with their pants down on AI integration. They have focused on backend systems to make their ad revenue more profitable. What Microsoft is doing is adding value to the end user. That is a major shift in people's focus on what AI means to everyone, not just Google.

Microsoft is taking a very visible lead in AI for the masses by integrating ChatGPT with Bing, Microsoft 365, development tools, etc. If ChatGPT provides anything near the level of benefit that Co-Pilot does for developers Google has a very valid concern.

I think Microsoft's approach, focusing on the end user value, will make this event be pivotable for how AI is used. Also keep in mind Microsoft is also releasing the biochat GPT, and I suspect there will be a number of targeted releases in the next weeks or months.

A brave new world...

2

MisterBadger t1_j7jd47m wrote

Nothing means anything if you're unfamiliar with the commonly understood meaning of words.

The dictionary definition of "inspiration":

>the process of being mentally stimulated to do or feel something, especially to do something creative.

Diffusion models are not, and do not have minds.

3

currentscurrents t1_j7j78vc wrote

All the computation is happening on the GPU. Python is just making a bunch of calls to the GPU drivers.

Researchers spend a lot of time making neural networks as fast as possible. If switching to another language would have given a substantial speed boost, they would have done it already.

12