Recent comments in /f/MachineLearning

viv1a t1_j3yfwu4 wrote

You can find it on amazon for cheap: https://www.amazon.com/Neurocomputing-Foundations-Research-James-Anderson/dp/0262510480

I second that it's a great book! it covers stuff until the late 80s and has very nice commentary on various foundational papers until then (McCullough and Pitts, Hebb, the Perceptron, Adaline, Neocognitron as well as Hopfield's works). The earliest paper it includes is actually from 1890 (!) and is by the psychologist William James who framed the mind as a kind of input-output machine.

There is a version out there with a cool cover depicting a neuron on a circuit board.

1

m98789 t1_j3xxyvm wrote

You are right that the trend is for costs to go down. It was originally reported that it took $12M in compute costs for a single training run of GPT-3 (source).

H100s will make a significant difference and all the optimization techniques. So I agree prices will drop a lot, but for the foreseeable future, still be out of reach for mere mortals.

2

Nmanga90 t1_j3xxwj6 wrote

Locally will not cut it unless you have a high performance computer with lab grade GPUs for inference. The reason the AI models are so expensive to use is because they are actually pretty expensive to run. They are running probably 2 parallel versions of the model on a single a100, and have likely duplicated this architecture 10,000 times. And an a100 is 10 grand used, 20 grand new. You can also rent them out for about $2 per minute.

1

Nmanga90 t1_j3xxi44 wrote

OpenAI is not going to close shop any time soon. Not sure if you know this, but Microsoft has been making huge investments into them, and has licensing rights to the GPT models. So Microsoft is pretty much the one who is serving the APIs, and they are right now looking into making another 10-billion-dollar investment into OpenAI.

1

farmingvillein t1_j3xui1m wrote

No, you can edit your original post and place it in there:

> OpenAI must be super confident about the generality of their AI and Microsoft product integration.

<-- add your link here.

> During weekdays, if you'd like to share a link, place it in a self-post and provide some context.

4