Recent comments in /f/MachineLearning

notdelet t1_j7vv9pi wrote

Reply to comment by tdgros in [D] Constrained Optimization in Deep Learning by d0cmorris

You can get constrained optimization in general for unconstrained nonlinear problems (see the work N Sahinidis has done on BARON). The feasible sets are defined in the course of solving the problem and introducing branches. But that is both slow, doesn't scale to NN sizes, and doesn't really answer the question ML folks are asking (see the talk at the IAS on "Is Optimization the Right Language for ML").

2

zanzagaes2 OP t1_j7vpd89 wrote

Reply to comment by schludy in [P] Creating an embedding from a CNN by zanzagaes2

Yes, I think that's the case because I am getting far more reasonable values comparing the projection to 2d/3d of the embedding rather than the full 500 feature vector.

Is there a better way to do this than projecting into a smaller space (using reduction dimensionality techniques or encoder-decoder approach) and using L2 there?

1

DACUS1995 t1_j7vltrw wrote

Reply to [D] Constrained Optimization in Deep Learning by d0cmorris

As you said most deep-learning models use some sort of regularization at training so there is some implicit constraint on the actual values of the weights, even more so when the number of parameters goes in the range of billions where you will have an inherent statistical distribution of the feature importance. On the more explicit and fixed side, there are a couple of papers and efforts in the area of quantization where parameter outliers in various layers affect the precision of quantized representation, so you would want a reduced variance in the block or layers values. For example, you can check this: https://arxiv.org/abs/1901.09504.

3

vivehelpme t1_j7vfol7 wrote

Reply to comment by CeFurkan in [D] Are there any AI model that I can use to improve very bad quality sound recording? Removing noise and improving overall quality by CeFurkan

Instead of trying to salvage the original recording why not recreate it by putting the text transcript into a text-to-speech model?

As you have it transcribed you don't even need to do any advanced speech recognition that filters the noise, just paste the text into something a bit more advanced than Microsoft Sam

2

sonofmath t1_j7vehdk wrote

Reply to comment by mr_house7 in [D] List of RL Papers by C_l3b

Not really, I think the main strength of the library is that it is designed to be easy to understand how the algorithms are implemnted. At the time, the main alternative was OpenAI/Stable baselines, which was quite obscure to understand how the algorithms are implemented. On the other hand, the algorithms do not use some more advanced tricks that enhance performance

However, there are better libraries now. In the same spirit, there is CleanRL, that is clean (with algorithms in one file) , but also performent. If you are looking for a modular easy-to-use library, I would recommend Stable Baselines3

2

tdgros t1_j7vdocr wrote

Reply to [D] Constrained Optimization in Deep Learning by d0cmorris

With constrained optimization, you usually have a feasible set for the variables you optimize, but in a NN training you optimize millions of weights that aren't directly meaningful, so in general, it's not clear if you can define a feasible set for each of them.

3

mLalush t1_j7vcqph wrote

Reply to [P] Get 2x Faster Transcriptions with OpenAI Whisper Large on Kernl by pommedeterresautee

Love your write ups /u/pommedeterresautee . Especially the fact that they're written with human beings in mind. I mean that as a compliment, seeing as the vast majority of stuff concerning cuda and low level optimization is impenetrable.

I periodically check kernl.ai to see whether the documentation and tutorial sections have been expanded. My advice is put some real effort and focus in to examples and tutorials. It is key for an optimization/acceleration library. 10x-ing the users of a library like this is much more likely to come from spending 10 out of every 100 developer hours writing tutorials, as opposed to spending 8 or 9 of those tutorial-writing hours on developing new features that only a small minority understand how to use and apply.

9

TKMater OP t1_j7vayeo wrote

Reply to comment by BenoitParis in [D] Similarity b/w two vectors by TKMater

I have classified some data points as anomaly in timeseries data and calculated feature importance vectors for them. Now I want to calculate similarity between two anomalous data points based on their feature importance vector.

0