> I had an exhausting number of conversations with confused product managers, engineers and marketing managers on “No, we’re not using ChatGPT”.

They use your conversations for further training which means if you use it to help you with proprietary code or documentation, you're effectively disclosing that.

In addition to the already mentioned ones, I recommend The Thesis Review Podcast for deep conversations about pure research

This is a reasonable question but I believe you are misunderstanding. The randomization of parameters in a neural network (I assume you are talking about initialization?) is certainly not the same as a mutation in a GA. Mutation occurs randomly, sure, but is selected for and crossed over, whereas hill-climbing and gradient descent simply move on the gradient and do not use either random mutations or cross-over so are not genetic.

Don't Bayesian approaches generally work better in gradient-free optimization?

The two pair up very well though - now that there's a natural language API, you could leverage that for speech->text->ChatGPT. From what I've seen of the Whisper demos, it seems to be the best out there by quite a margin. Does anything else perform as well?

It was an interesting business decision to make a blog post announcing two rather different products (ChatGPT API and Whisper) at the same time...

ChatGPT is a best-in-class, or even only-in-class chatbot API... While Whisper is one of many hosted speech to text solutions.

I guess you haven’t visited any B2C websites in the last 5 years.

But also: there is a world model behind the chatbot which can translate between human languages, between computer languages, can compose marketing copy, summarise text...

> Don't let it demotivate competitors. They are making money somehow,

What makes you so confident?

Genetic algorithms are good, as you said, when you really understand the space and can come up with a really good candidate generation system. They’re okayish (or, the same as everything else) when you have no understanding of the space at all, and you’re just totally guessing. They can’t latch onto a curve in design space as well as things that look at a simpler gradient can. So maybe they’re best used for really complex spaces where gradient based methods don’t do well. The kind of places you’d use Gibbs sampling, or general optimization algorithms.

So, basically, they’re useful when you have good feature engineering already done, like many methods that have fallen out of vogue in the age of letting algorithms and data do your feature engineering for you. And they’re as good a shot in the dark as any when standard methods fail and you’ve got no clue how to proceed.

So, yeah, the number of times genetic algorithms are the “right” choice is pretty limited these days.

$20.00 / ($0.002/ 1k tokens) = 10m tokens. If you only used a few hundred k, you got scammed hard lol

Don't you mean the other way around?

MIT license has now been added to the project!

Aren't biases only a tiny tiny fraction of the total memory usage? Is it even worth trying to quantize them more than weights?

1 of 2 months??? How would that short time achieve the goal against well-funded competitors?

It would need to be multiple years of undercutting and even that might not be enough to lock google out.

They’ll need a stronger story around lock-in if that’s their strategy. One way would be to add structured and unstructured data storage to the APIs.

Thanks for pointing this out! I'll add some permissive license to the repository today to allow free use!

I think you are using the word revenue when you mean profit.

Looks great! Is the code under a specific licence?

It wasn't as bad as it sounds. The fun part was that you had to understand how every little piece of the algorithm works, and the nightmare was implementing all of this with 512mb of RAM. We didn't have the luxury of trying different solutions.

Excellent

You can edit what it replied of course (and then hope it builds off of that and keeps that specific vibe going, which always works in the playground) but damn, they locked it down tight. 😅

Even when you edit the primer/setup into something crazy (you are a grumpy or deranged or whatever assistant) and change some things it said into something crazy, it overrides the custom mood you set for it and goes right back to its ever serious ChatGPT mode. Even sometimes apologizing for saying something out of character (and by that it means the thing you 'made it say' by editing, so it believes it said that)

That's not diffusion though

This is actually a very cool approach, if more narrow in application. Other domains that have these linear representations like the Morgan fingerprint might also benefit.

Yes, there is a whole field https://arxiv.org/abs/2008.10937

I'll give it a try, thanks