In an alternate universe, Numenta's Hierarchical Temporal Memory model has won the race against Deep Learning. Sometimes I wish I lived in that universe.

It's an ML paradigm deeply rooted in neuroscience. Things like HTM School on Youtube are just awesome. I believe that if the ML community had jumped on the HTM wagon at the time (and Transfomers hadn't shown up) and had invested the same amount of effort into it as it has into DL, we would be at a similar point in development but with a better ML framework.

Also if not 100% native, compiling libraries for architectures is a big complication.

Awesome. I dont, but there is a VSCode extension, so that would be integrated already. Or do you have any special integration of copilot?

>This seems to be a very ambitious project, as there are several ML projects that have very obscure dependencies that dont't work out of the box. This is especially true for older repos. I would personally be very interested at a reasonable price level (compatible to vast ai or runpod) to check out some repos without hassling with setup.

Yeah that makes sense, appreciate the feedback! We're hoping it works out as well haha

You mentioned users between expert and non-technical folks, and we think that this is intended audience for our app. Most of this is due to the fact that we're building this for ourselves(as recent ML grad students), and it made sense to us to solve a problem we're familiar with.

I also DM'd you a link to the app, if you have time to check it out would appreciate your feedback.

>Any alternative which would be able
to solve the same problems would probably require a similar
architecture: lot of parameters, deep connections.

If handwritten character recognition (and generation) counts as one such problem, then here is a model that solves it with a handful of parameters: https://www.cs.cmu.edu/~rsalakhu/papers/LakeEtAl2015Science.pdf

[removed]

OK got it that makes sense. We're actually trying to figure out how to do a tiered access, possibly a free tier with CPUs only might make sense.

​

I'll also DM you with a link to try the app!

Yeah definitely true. I've use all of these except sagemaker, and I felt the solutions weren't perfect for my workflows. I guess we're figuring out if other people feel the same way!

Gotcha. We have a VSCode editor built in, but haven't implemented a .vscode integration yet so we'll add that in.

Do you also use Github Copilot?

Yeah. This is a very nice summary of it.

That’s a pretty big hope that all data can be modeled linearly

I work in the financial world- and to elaborate on your comment:

Speaking strictly for myself, there are a few reasons I rarely use neural networks at my day job:

-Model explainability: Often stakeholders care about being able to explicitly label the "rules" which lead to a specific outcome. Neural Networks can perform well, but it is harder to communicate why the results happened than with simpler models.

-Negligible performance gains: While I am working on multi-million row datasets, the number of features I am working with are often small. The performance improvements I get for running a tensorflow/pytorch model are nearly on par with running an sklearn model. As a result, deep learning is overkill for most of my tasks.

-Developer Time & Speed: It is much quicker and easier to make an effective model in sklearn than it is in tensorflow/pytorch. This is another reason Neural Networks are not my default solution.

There are some out-of-the-box solutions available in tools like Sagemaker or Transformers. But even still, finding and implementing one of these is still just going to take slightly longer than whipping up a random forest.

-Legacy processes: There's a mentality of "if it ain't broke, don't fix it". Even though I am considered a subject matter expert in data science, the finance people don't like me tweaking the way things work without lengthy consultations.

As a result, I am often asked to 'recreate' instead of 'innovate'. That means replacing a linear regression with another linear regression that uses slightly different hyperparameters.

-Maintainability: There are significantly more vanilla software engineers than data scientists/ML Engineers at my company. In the event I bugger off, it's going to be easier for the next person to maintain my code if the models are simple/not Neural Networks.

Oh wow I never considered if I should see them as different. Transformers are a pretty big conceptual change, but they’re used in a deep way. 🤔 food for thought

The key word you to use for better answers are “control problem” and “AI safety”. For my personal opinion ChatGPT/GPT-3.5 is an inflection point. GPT-3.5 can understand programming code well and do a passable job generating it. This includes its own code. One of the beginner tutorials is using GPT to program its own API.

That said, GPT-3.5 has many limitations. It isnt a threat. Future versions of GPT have the potential to be very disruptive.

Do you define transformers as Deep Learning? Cause if not then transformers.

There’s living neurons in an artificial network that’s more of neuro/cognitive science. Learned pong with less iterations but there are ethical issues.

https://www.npr.org/sections/health-shots/2022/10/14/1128875298/brain-cells-neurons-learn-video-game-pong

Would this run on windows?

PhD candidate on AI here. Gradient boosting is the real deal when tabular data is concerned (for both regression and classification on ML). However, thx to UAT neural nets are awesome approximators to really complex functions and therefore are the way to go for complex tasks, like the ones presented by scientific machine learning, for example. LeCun (not so) recently said that deep learning is dead and differentiable programing (another way to describe SciML) is the new kid in the block.

Yeah thats true. My comment relates to unconditional diffusion Models a la Song and not stable Diffusion. The Argument might be adapted for conditional Generation.

If I train the model on a 1x1 pixel set of images that only have 2 states, black or white, and two labels, black or white, then shouldn't prompting 'black' generate a 1x1 black image 100% of the time?

Well of course. They just usually aren't as good on unstructured data (pictures, video, text, etc.). If they start performing better, then people will use them. Simple as that.

Linear regression is the future, maybe? 😭

All I really need is a storage unit that doesn't make me move large GB files up and down like colab (takes forever, has to be done everytime), and that lets me use a A100 on the fly when needed (instead of confusing compute units bs) without having to switch runtimes (which makes you move data again).

I think what we can expect in the future is combinations of deep learning and new, as of yet unknown methods. Just as reinforcement learning made older AI concepts new again, I bet we will see a merge. One that I'm watching right now is IBMs neurosymbolic approaches, which are attempting to merge formal methods learned through deep learning techniques.

Haven't watched yet, but does he address criticism by e.g. The Principles of Deep Learning Theory, regarding the infinite width limit?