Large scale is somewhere to the north of 1-2 TB of data. Even if you had that much data, in absolutely most cases tabular data has such a simplistic structure that you wouldn't need that much data to achieve the same performance - so I wouldn't call any kind of tabular data large scale to be frank

I don't think it implies that all datasets are equally likely. I think it only implies that given all possible datasets, there is no best approach to modelling them. All possible != All are equally likely

But I don't have my book with me, and I do t trust the internet since it seems to lead to random blogposts instead of the original paper (Wikipedia gave a 404 in the footnotes)

We use gradient boosting at quite a big scale. Not LLM big, but still big. It's just not NLP or CV at all. It's for fraud detection in large transactional tabular datasets. And it outperforms basically all neural network, shallow or deep, approaches.

Really interesting work. Might be worth doing even if courts decide this isn't going to work as a legal defense.

[removed]

We already handle that as our notion is not based on reproduction but is rather a information-theoretic notion. We also have a parameter that measures how much information we have "reproduced" vs adapted which can be set depending on the underlying models and the use case.

Not a lawyer, but yes.

Copyright infringement goes beyond reproduction of a work. An original adaptation can also violate copyright.

I wonder how flamingo would compare

There's no mathematical formulation of that statement because there's no mathematical reasoning behind that statement. It's just an opinion (which I believe it isn't true btw)

Interesting research! Let's try to convince artists with this kind of work

I agree with you. I was just pointing out that to say they are the only solution is foolish, as the quote implied

This quote could have just been used without much context, so grain of salt.

Are there any developments in the ASIC/analog computing space that people are really excited about? I think most people know about google's TPUs by now, but is there anything else with the potential to threaten the dominance of GPUs in the next few years?

hmm for some reason the arxiv links are giving 403 forbidden

Linear models are often preferred for the reasons you mentioned. Under fitting is almost always preferred to overfitting.

Because they are useful for some problems and not others, like every algorithm? Nowhere in my statement did I say they are monolithic in their use across all subdomains of ml

The statement was that deep learning is the only thing that works at scale. It’s not lol. Deep learning struggles in a lot of situations.

As you mentioned, this is highly dependent on the functional relationship of the data.

You wouldn’t not use domain knowledge to determine that.

Additionally, non linear models tend to have their own drawbacks. Lack of interpretability, high variability being some of them

This is a very good answer! I want to add that apart from generalization, the fact that we have efficient optimization algorithms that can find quite good minima also contributes a lot to the deep learning magic.

See this tutorial - converts to ONXX CPU, then to tensor-RT for a 3-6x speedup.

https://developer.nvidia.com/blog/optimizing-t5-and-gpt-2-for-real-time-inference-with-tensorrt/

[deleted]

[removed]

Even with angles you can still have exponentially many vectors that are nearly orthogonal to each other, if that's what you mean...

I agree the representations will be different. Indeed one issue may be that large negative entries will be penalized as much as large positive ones, which is not the case for logsumexp...

But on the other hand more "geometric" representations like this, based on angles, may make the vectors more suitable for stuff like LSH.

thank you.

"Ok, but you are going about it the wrong way. The possibility of god existing or not is irrelevant to what we can know about his existence. If something is unkowable, then any categorical statements about it are invalid. Yes, we can consider the possibility, but if you can't ever tell if it's true or not, this approach makes no sense.

we do not have enough evidence to suggest that god exists or does not exist in a black hole.

And what i'm saying is that since we will never have that evidence, no matter what, it is irrelevant to approach the problem from this angle."

I respect your position, but this is where we are going to have to agree to disagre, Ive got to shovel some ones drive way. It was nice talking to you.

alright have fun

>But the point i am trying to make is that it is a possibility that God exists.

Ok, but you are going about it the wrong way. The possibility of god existing or not is irrelevant to what we can know about his existence. If something is unkowable, then any categorical statements about it are invalid. Yes, we can consider the possibility, but if you can't ever tell if it's true or not, this approach makes no sense.

>we do not have enough evidence to suggest that god exists or does not exist in a black hole.

And what i'm saying is that since we will never have that evidence, no matter what, it is pointless to approach the problem from this angle.