> allow it to generalise to generate new ones!

FTFY, that's how you get skynet!

It's literally called importance sampling in the SGD literature. You normally have to downweigh the "important samples" to counter the fact that you're sampling them more often. Whether this practice actually accelerates convergence has been an important question in SGD until very recently. Check this paper.

Why? It's open access.

Can you check it again?

But like yeah if you zipped the NPY files storing the weight matrices if most likely would not run

Intelligence is compression

You're not making a lot of sense. It's not clear you understand what software is, what an AI is, what encryption is. Why do you think encrypting an AI would affect it running? Do you have some specific encryption in mind?

​

Are you just high?

I had the same thought... Im fairly sure any gpt based model can only handle 4k tokens.

Arxiv link?

If it really had human like intelligence the encryption key would be 123, so it would be easy to fix.

Homomorphic encryption

>Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first having to decrypt it. The resulting computations are left in an encrypted form which, when decrypted, result in an output that is identical to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and out-sourced to commercial cloud environments for processing, all while encrypted.

^([ )^(F.A.Q)^( | )^(Opt Out)^( | )^(Opt Out Of Subreddit)^( | )^(GitHub)^( ] Downvote to remove | v1.5)

It could work with a heavy efficiency penalty.

https://en.m.wikipedia.org/wiki/Homomorphic_encryption

Though I don't think gradient descent will select for something like this.

Far more likely is to obfuscate how it works so while it's not encrypted we learn little with the tools we have, and would have an extremely difficult time verifying something is absent.

[deleted]

Yes.

“SoTA” is also often ill-defined and while important, can sometimes be a bit overhyped IMO.

Most practitioners and engineers want something that is as good as it can be or is above some threshold in accuracy, given constraints that can often be severe. If a “SoTA” approach cannot meet these real-world constraints, I would argue it’s not “SoTA” for that particular problem of interest.

If you have something that performs very well under such real-world constraints and can demonstrate value to the practitioner, it should be considered for publication by the editors.

In Deep Reinforcement Learning is called prioritized experience replay: https://arxiv.org/abs/1511.05952

Here's one to get started https://proceedings.mlr.press/v98/cotter19a.html

Unless I missed it, the paper does mention the fact that the degradation mapping should be estimated but does not detail or cite papers that do that. (examples: KernelGAN, KernelNet, doubleDIP or MetaKernelGAN...)

Authors publish papers on research, experiments, findings, etc. They do not always release the code for the models they are studying.

The lucidrains' repos implement the models, creating an open-source implementation for the research

The next step would then be to train the model, which requires a lot more than just the code (most notably, money). I assume you're referring to these trained weights when you say "the needed AI model". Training would require a huge amount of time and money for a team, never mind a single person, to train even one of these models let alone a whole portfolio of them

For this reason, it's not very reasonable to expect lucidrains or any other person to train these models - the open-source implementations are a great contribution on their own!

Oversampling

This seems somewhat similar to hard example mining except that you already know which ones are hard here.

There is the claim that any system can be (approximately) reverse engineered if one has access to the results of the system. Are those too hidden from the public?

What is "best" is subjective. At least I was reading last week that any moderate fitness related interest brings quite unhealthy content very quickly. But it has to be better than Amazon's system, anyway.

Because you have a small batch size, my feeling is that you probably want a very small dropout rate on the important items, if only to decrease the chance the network overfits to them. Maybe 1 / 100 batches, excludes the important item and the rest include it. But perhaps it doesn't matter.

Well, I can't access it from India

[deleted]

https://en.wikipedia.org/wiki/Importance_sampling