I think that your supervisor is using arxiv wrong, and giving you bad advice on how to use arxiv.

I can definitely imagine Newton posting this, "that fucker Leibiniz gets his name on the notation it's not fair!"

I agree. I just enjoyed diving into HTM at the time and learning cool stuff about the brain. It was a nice break from the math/engineering heavy DL concepts.

It can. You would need to flash the vbios to a wx9100. It would only have a single mini-displayport out though

Remember--you're unique just like everyone else!

I disagree with your supervisor and your post pretty much illustrates why!

I just bought one from Brainchip. They seem pretty good. I asked them some of their use cases, they have some videos on their YouTube on classification tasks of images of beer bottles, but they seem to be the same kind of tasks you can do on a regular GPU.

Brainchip PCI chip is interesting because you can code for them like regularly, and then send the built neural network to the chip and convert it from a CNN into a SNN, but there doesn't seem to be a great reason to use it this way. It seems like the main use case would be to run a native SNN on it. NPUs don't seem to scale the way GPUs do though either.

I think this is called 'syncronous serendipity'

DiffWave and WaveGrad are two nice TTS examples (see e.g. here https://andrew.gibiansky.com/diffwave-and-wavegrad-overview/), Riffusion (https://www.riffusion.com/) is also a fun example. Advances in audio generation always tend to lag behind the visual domain a bit, because it's just inherently more unwieldy to work with (listening to 100 samples one by one takes a lot more time and patience than glancing at a 10x10 grid of images), but I'm pretty sure the takeover is also happening there.

If you're talking about text-to-audio in the vein of current text-to-image models, I'm pretty sure that's in the pipeline :)

you might want to mention how excited you are that a fancy lab like theirs is working on similar stuff ... maybe you can give them a talk or something :)

Thanks man! I appreciate the words of encouragement.

Since it would technically be advertising as per cvpr rules. If you’re interested I can send you some after the review process finishes.

If both paper have similar results that's acually good IMO. That mean approach is actually works and not some hyperparameters fiddling.

Got it. Thanks a million for the advice!

I edit and would not mind if an author reached out as it’s my job. I would do it as soon as possible. But tread lightly and ask your advisor her opinion.

Thank you for this. It seems this paper could surely help answer my question, if only I could understand it!

A challenge I keep coming up against in my quest to quickly learn about ML/NN is that almost everything I read is either too high level to provide meaningful explanation or too technically dense for me to follow. I guess I will just take note of this paper for now and circle back to it when I'm a bit further along.

I see, and would this be done after the reviewed are released or should I reach out to the editor now?

Will do that!

>as it has in perceptual domains, like audio

citation needed

You could ask very politely to discuss this with your editor and just point out that the new work validates and demonstrates the importance of your idea. You can acknowledge the preprint in your revision making it clear it came after your original submission.

arxiv is really planting your flag on the topic. I dont trust it because its not peer reviewed, but its definitely a good way to call dibbs

If you were to graph the weighting that ensures the training loss corresponds to likelihood, you would find that it looks roughly like exp(-x). In other words, the importance of the noise levels decreases more or less exponentially (but not exactly!) as they increase. So if you want to train a diffusion model to maximise likelihood (which can be a valid thing to do, for example if you want to use it for lossless compression), your training set should have many more examples of low noise levels than of high noise levels (orders of magnitude more, in fact).

Usually when we train diffusion models, we sample noise levels uniformly, or from a simple distribution, but certainly not from a distribution which puts exponentially more weight on low noise levels. Therefore, relative to the likelihood loss, the loss we tend to use puts a lot less emphasis on low noise levels, which correspond to high spatial frequencies. Section 5 of my earlier blog post is an attempt at an intuitive explanation why this correspondence between noise levels and spatial frequencies exists: https://benanne.github.io/2022/01/31/diffusion.html#scale

"Variational diffusion models" is another paper that focuses on optimising likelihood, which you might find more accessible: https://arxiv.org/abs/2107.00630

these are concurrent works, the first to put on arxiv by a few days is not the one that gets bragging rights :) One thing you could do is send them your arxiv link, tell them it's concurrent work, and in the next arxiv iteration or whatever you should each cite eachother as concurrent work.

My supervisor advised me against it. She’s not a big fan of putting papers there before they’re published.

Will do so! Thank you :)