Depends on the service. What cloud service are you using to get access to the GPU?

You understimate how hard cross-platform is to achieve. Especially with GPUs. There is no GPGPU API standard, first and foremost, so ensuring cross-platform is a tedious task which essentially either means creating an API that has to accomodate every GPU, or writing "drivers" for every different GPU. GPUs can be vastly different between generations and models, unlike x86 and x86-64 CPU architectures which have mostly stayed the same for several decades now.

Vulkan Compute is nowhere near reaching feature parity with CUDA and cuDNN. ROCm's CUDA is way better and still too much of a pain to install and keep.

Furthermore, open standards mean nothing when a graphics vendor can just gimp the API, like, ironically, nvidia already does with Vulkan.

There is an open variant called OpenCL. But it will probably never be as mature as CUDA, even though 3.0 is apparently making great strides. There is absolutely no reason to push for Vulkan due to how cancerous developing anything in it is.

Of the cloud gpu

What instance

[deleted]

Ahh, ok so if I run the instance on my browser and I close my browser it will continue to run?

Depends how you access it.

Generally yes but if you access it via SSH you'll need to screen or tmux it since disconnecting from the session will also shut down your script.

So, is it worth exploring or not? I don't know if I should spend 2 months trying to understand the diffusion models only to find it can never be better. Vae based models had the same fate. I was studying them and suddenly transformers took over.

The work is probably not useful for most DL practitioners (yet), but has lots of applications for deep learning research, even for people outside of deep learning theory. As an example consider the work on the Neural Tangent Kernel, which considers one infinite-width limit of neural networks. While the work itself originally was just trying to understand wide fully connected networks, its impact now in 2023 is immense.

A lot of new algorithms for things like active learning, meta learning, etc. use NTK theory as motivation for their development. You could pretty much search "a neural tangent kernel perspective on ____" on google and get a ton of results, a mixture of applied algorithms and theoretical analyses.

So this is just one example of how understanding DL theory leads to better algorithms. One part Greg Yang's work could be considered generalizing NTK theory to different infinite width limits. At the moment, there doesn't seem to be too many applications of his work, but of course the same would have been said about the NTK in 2018. His "Tensor Programs V" paper shows that one application of his work is for choosing hyperparameters for large neural networks using smaller ones as a proxy.

So TL;DR - there might not be practical applications yet, but there are potentially a lot!

I am willing to be that 99% of the code is overprotected and no one in OpenAI would spend valuable time looking at it.

These protections mostly exist to justify some bullshit jobs within the company.

I was going to argue that employees will be able to bullshit their automated manager easily but well, it is not like humans are much better at handling it.

Yes, it's been like this for a while now.

Has it really come to this

Cool! Good times :)

What should you do when your model fails the test set? That is, the test set is supposed to give you an unbiased view of how your model should behave with real-world data, but the moment that test set prevents you from deploying a model and forces you to improve it, it is no longer unbiased. And if this keeps happening, what is the difference between a test set and a validation set?

> We're a long way from a robotic farmhand being able to perform those skills, certainly not for a price comparable to a farm laborer.

If we get AGI, we automatically get that as well, by definition. Those you listed are all currently hard problems, yes, but an AGI would be able to do them, no problem.

The issue is, will AGI ever be achieved, and if yes, when?

I think the answer to the first one is simple, the second one not as much.

The answer (in very short) is: Most likely yes, unless we go extinct first. Because we know that general intelligence is possible, so I see no reason why it shouldn't be possible to replicate artificially, and even improve it, and several, very wealthy companies are actively working on it, and the incentive to achieve it is huge.

As for the when, it's impossible to know until it happens, and even then, some people will argue about it for a while. I have my predictions, but there are lots of disagreeing opinions.

I don't know how someone even remotely interested in the field could think it will never happen for sure.

As for my prediction/opinion, I actually give it a decent chance of it happening in the next 10-20 years, with probability increasing every year until the 2040s. I would be very surprised if it doesn't happen by then, but of course, there is no way to tell.

[deleted]

Are you looking to do distributed training across machines? Otherwise the NIC seems complete overkill.

[deleted]

i just wanted to comment that your solution to the galaxy zoo contest forever ago was the first demonstration to really open my eyes to what was possible with clever data augmentation.

Pycaret?

As a recommendation I learned from a past job, use slurm or a similar program to make turns on the use of the gpu so you don’t end up dropping each other’s models

Being cross-platform and not tied to a single vendor's hardware would be a great plus. Vulkan Compute is for general purpose compute not graphics.

memeGPT

AutoML ?