Of course we require no encoding model, so the notion of a latent space only holds up until closer inspection.

Hey, this is it actually! We are optimizing a discrete variational autoencoder with no gumbel-softmax trick.

Definitely! Come check out our discord server and introduce yourself.

Do you mean OpenWorm, where they try to code a nematode on a cellular level? Having the connectome mapped out doesn't mean they've managed to model its whole brain. A connectome is just the schematic and even that only with the individual cells removed. Kinda like an old school map, you can navigate based on it but it won't tell you where the red lights or shops are or what people do in the city.

I like how you criticize me for not providing scientific evidence for my reasoning, but then you go and make statements like "most people wouldn't consider it is sentient" and that's a general truth I'm supposed to accept.

I mentioned transformers only to point out both image generators and LLMs are similar in concept in a lot of ways, but yet people didn't start associating sentience with image generation. I didn't mean to imply a certain architecture allows or disallows sentience.

You're talking about the emergent qualities of consciousness. A common view about that seems to be that it emerges from the anatomical, cellular and network properties of the nervous system, and is necessarily associated with the vital, hedonic, emotional relevance of each experience and external cue, and intrinsically oriented to a behavioral interaction with the latter. In addition, many argue it doesn't even "eventually emerge" but is rather intrinsic and not added a posteriori. None of this is present in neural networks, as artificial neurons in neural networks don't have a continuously changing impulse pattern, but are basically just activation function giving a deterministic response. Yes, there's randomness introduced in these systems, but once trained, individual artificial neurons are pretty deterministic.

What I'm trying to say is that when scientists argue for the emergent nature of consciousness, they argue it emerges from the specific properties of our neural architecture, which is vastly different than that of neural networks'. So even if neural networks had some emergent features that emerge for that tiny bit of time (compared to our consciousness being on for most of the day) when they're generating an answer, I wouldn't call that sentience or consciousness, as it fundamentally differs from what we understand as sentience. In addition to that, a neural network doesn't continuously change and learn new things, it doesn't evaluate options and change its neurons' activation function. Once it's trained, it stays the same. The only things that temporarily change are in the memory module of the feedback systems, and that only serves the purpose of being able to hold conversation. Once your session ends, that gets deleted and it doesn't feed back into the system. Or at least in ChatGPT, there's no self-supervised learning present, and the whole system is basically immutable apart from those LSTM-like modules that allow it to have context. But even those get overloaded with time.

Congratulations on the initiative! Is there a way for people willing to help with the research as entry-level collaborators/volunteers? I am just starting my steps with ML, and I certainly don't have much to add but I'd love to get involved in an open-research initiative and help somehow!

I agree completely that pop-sci articles completely sensationalize this topic, but to be fair, they do that with every part of science. A funny one comes to mind of an article claiming something along the lines of "scientists create white hole in lab" but what actually happened is they ran a stream of water down on a flat surface and the spread acted mathematically similar to a white hole.

Nobody writes articles that nematodes are sentient despite fundamentally containing the same building blocks that human intelligence is built on. Side note- If mimicking real neurons is what you believe to be sentience, then the complete nematode connectome that you can emulate on your desktop already achieves that.

It is because most people would not consider their simple intelligence to be sentience, not because neurons as a building block are completely incapable of developing sentience.

As far as the architecture, wether it be Transformers or RNNs, even something simple like Markov chains, i dont think its relevant as I have seen no convincing pieces of evidence that any neural network type would never exhibit sentience as an emergent property.

[removed]

Thanks for open-sourcing it :-)

Well machines might eventually get an intelligence similar to our, but that would be AGI to which we really have no way to as of yet. These are all specialized systems that are narrow intelligences. The only reason this argument of sentient AI got picked up nowadays is because this model generates text, to which many more of us can relate than to generating art.

If you go down into the math/code level, both are basically built on the same building blocks and are largely similar (mostly transformer-based). Yet, no one started writing articles about how AI was sentient when it only generated pretty pictures. For LLMs to be conscious it would require for us to work in a very similar way, eg. to only take written language as proof for our consciousness. Written language doesn't solely define our consciousness.

Thanks

No it does not. In the past we felt that the best way to achieve our goals was to focus almost exclusively on training large models though, and we no longer feel that’s the case.

Link: https://youtube.com/@TheRobotBrainsPodcast

Link: https://youtube.com/@SuperDataScienceWithJonKrohn

Link: https://youtube.com/@MachineLearningStreetTalk

There could be a few simple solutions to extending this to 64x64x64 and each would have certain pros and cons. The two key decisions to make are in regards to the data format (perhaps there is a way to compress/reformat data so it's more digestible than direct 64x64x64) and in regards to the type of the underlying architecture (most importantly, do we use a 2D or 3D CNN, or a differnt type of topology altogether).

A trivial approach would be to use a 2D architecture with 64 channels instead of the usual 3, which could be very easily implemented with the existing framework. I predict that would be quite hard to train, however, though you might still try.

This is an area of active research (beyond DreamFusion and other popular papers I'm not very familiar with it), so exploring different solutions to this is still required, and if you discover something that works reasonably well then that will be really exciting!

I love it! Great to see these kind of applications

Thanks for the input. I have seen some papers claiming SOTA in image segmentation using diffusion so I am also curious to see how they perform.

I have another question, if you don't mind. How difficult would it be to extend the code for image-to-image translation so that it works on 3D data (64x64x64 for example)?

currently it is fairly unlikely as far as I can tell, but most arguments given are not restricted to "at its current size and complexity it doesn't appear to have the traits of a truly sentient being" and are essentially declarations that machines can never have any degree of sentience or that it would require some uobtainium mcguffin type math that is currently impossible.

[removed]

Thank you! Yes, in principle, you can generate segmentation maps using the code from the course by treating the segmentation map as the output. I'm not sure how that would compare to a non-diffusion segmentation with the same backbone network but definitely it would be interesting to explore that!

Please remember that the diffusion process generally expects data bound in [-1,+1] range, so in the framework, the images are shifted from the assumed [0,1] limits to that range automatically (via input_T and output_T). So if you go beyond the binary and use more classes within a single channel, make sure the output ground truth values are still between [0,1] (alternatively, you can split each class confidence into a separate channel but it should still be bound).

But yeah, for binary, it should work with no special adjustment!

Our lab made a python package that combines simulated annealing and genetic algorithms for helping to solve really gnarly inverse problems in chemistry modeling. The package is on GH, and here is the link to the paper where it was primarily used.

I think GA is still fairly popular in operations management to e.g. produce solutions for delivery scheduling problems.

Interesting, thanks for sharing your thoughts! I'm a bit curious about why genetic algorithms might be better for these strange objective functions, as compared to something like simulated annealing. I can understand that a pure gradient method could easily be insufficient, but do the underlying components of genetic algorithms (like cross-over, etc.) really provide a distinct advantage here? Especially when the fitness is probably directly related to the gradient anyways

Congrats! Can't wait until you get your first $10-billion investment from a major tech company.

Hmm. Hadn't thought of that, but that's probably true. I'm less familiar with the mechanics of these but I bet they'll be similarly good for low or no-information scenarios or P-NP problems.