Unless the task is not present in the human language distribution it learned to mimic and in your prompt, it will not be able to do it.

While counting is one task that shows that it doesn't actually understand anything, there are many more, among those it doesn't outright refuse to answer to. Some examples are math in general (especially derivatives and integration), logic to some extent or pretty much anything too big for its memory (my assumption is it is able to fit a hundred or two hundred sentences before it forgets things).

For things not present in your prompt, it is also heavily biased. For example, even though it claims it doesn't give out opinions, it prefers Go as a programming language, AWD for cars, hydrogen and EVs for fuel technology (possibly because of its eco-terrorist stances), the color red... These biases might be preventing it from doing some tasks it usually should be able to do.

For example, if you ask it to objectively tell you what the best car is, it might say Toyota Mirai, even though it's actually a terrible car to have even in California, the best place to have one. You might be thinking that its thinking is broken, but in reality, the biases screwed it over.

I'm curious how you feel about the following:

There are humans that can't do the task you outlined. Why use it as a metric for AGI? Put in other words, what about a "less intelligent" AGI, that crawls before it walks? An AGI equivalent to a human with lower IQ, or some similar measurement that correlates with not being capable of the same things as those in your example?

Second, if an A.I can do 80% of what a human can, and a human can do 10% of what an A.I can, would you still claim the system isn't an AGI? As in, if humans can do X, A.I can do X * 100 things, but there's a venn diagram with some things unique to humans and many things unique to A.I, does it not count because you can point to human examples of tasks it cannot complete?

Finally, considering a human system has to account for things irrelevant to an AGI (body homeostasis with heart rate and such, immune system, etc) and an AGI can build on code before it, what do you see as the barrier to AGI? Is it not a matter of time?

checked

I've seen the exact same thing when interviewing people that have a great CV and can talk at length about their experience but then can't do a very simple task if you ask them to.

Hello! I have a question. Will it be possible to create a NN that replicates the behaviour of prophet? I dont want to do it, I just wanted to understand from a theoretical point of view what will be the most similar way to do it (optimize a function that take into account seasonality and provides an infinite 'regression' way to predict new values based just on dates. Thanks in advance!

Can I implement NECSTransformer architecture from this paper practically? What do you think?

I have tried this one just now and this is the response "There are 8 ones in the binary number 01010101110101010." But able to solve the following below: Numbers 1 to 10 in binary values 1 = 0001 2 = 0010 3 = 0011 4 = 0100 5 = 0101 6 = 0110 7 = 0111 8 = 1000 9 = 1001 10 = 1010

Yeah. You‘re right. The essence is to construct a right model for “counting”

Wrong tool for this kind of task, it should generate a Python function which will give you the answer when evaluated on the input. And this approach would generalize better. The Turing machine approach is useful when you're dealing with concepts that don't fit well into Python code.

Thank you, I am glad you like it! At the moment, only the web server is public. You can find it here: https://github.com/niclaswue/arxiv-smry It is a Hugo server with a blog theme. Every blog is a markdown file. When a new file is pushed to git it automatically gets published on the blog.

The rest is basically a bunch of (messy) Python scripts for extracting the text, then asking GPT-3 for a summary and compiling the answers to a markdown file. Finally, I use GitPython to automatically push new summaries to the repo.

<3

don't need to check the comment history that style of formatting shines like a red light now, woe betide anyone that naturally types like that. They'll need to change things up to avoid being accused of being a bot!

(or on the other extreme, people get so used to reading things formatted with that cadence, it starts being unconsciously replicated in their own work.)

Thank you. I'll try.

It seems that we should teach it to build a Turing machine that solves the question instead of letting it comes out one (which is likely to be wrong) itself?

Don't just tell it to do it step by step, show it how to do it step by step and keep track as it goes. It will handle it then.

Prompt: I would like you to process the the following strings like this example: "1 0 1 1 0 1 0 0 0" Step 1. Start with a count of 0. Step 2: Look at the first digit in the string and remove it, it's a 1, so add that value to the count. The new count is 1. The new string is "0 1 1 0 1 0 0 0". Continue until the string is empty. Can you try on "1 0 1 1 1 0 1 0 1 0 1"?

> Start with a count of 0

> Look at the first digit in the string "1 0 1 1 1 0 1 0 1 0 1" which is 1, so add that value to the count. The new count is 1. The new string is "0 1 1 1 0 1 0 1 0 1"

> Look at the first digit in the new string, which is 0, so add that value to the count. The new count is 1. The new string is "1 1 1 0 1 0 1 0 1"

...

> Look at the first digit in the new string, which is 0, so add that value to the count. The new count is 6. The new string is "1"

> Look at the first digit in the new string, which is 1, so add that value to the count. The new count is 7. The new string is ""

> The string is now empty, the final count is 7.

I know clearly about "linear algebra, calculus, and probability." And yes, I'm treating ChatGPT like a black box: not the training algorithm as a black box but the parameters it learned from corpus as a black box. There're billions of parameters and as far as I know most AI researchers treat them as a black box too. If you know some of AI research, DL models' interpretability is a long-standing difficult problem. Shortly, they are hard to understand. Moreover, we can have some tuition about DL models: CNN's filters represent image objects' features in different levels and transformer's Q-K-V matrices are about attention. What I'm asking is why such design can outperform traditional NLP methods so much.

BTW, I'm a bit infuriated when you say I "have to read some papers." My Zotero library contains a hundred read AI papers and more importantly, I've just posted two papers I have read in this post. They give a direct explaination about why ChatGPT fails in some regex and CFG tasks. My question is just one step further after reading these two papers.

The tone in the images is just for fun because I orininally posted this as a joke to my personal circle on the social media. I do have at least CS-grad-level knowledge about how DL models work.

This comment just doensn't make any sense. AI does not think. AI does not talk. So what? You're still talking with it about the weather and it responds you with a word string that seems very meaningful to you. Actually I'm talking about why ChatGPT's ability is week when compared with a finite state automaton, a push-down automaton, and not to say, a Turing machine, but can still achieve such a performance in NLP tasks.

The first image transformer is pretty clear that it works better at scale. You might not need a transformer for interesting work though.

You can do so much with that GPU. I think transformers are heavier models, but my background is on CNNs and those work fine on your GPU.

It's a little discouraging when every interesting paper has a cluster of 64 A100s in their methods section.

[deleted]

[removed]

I’m not sure why you think that that’s such a crummy graphics card. I’ve trained a lot of interesting things for grad school and even in the work place on 4GB less. If you’re fine tuning then it’s not really going to take that long to get decent results, and 16 GB is not bad.

It's failing because you're asking the ai to think. It does not think.

A true AGI has way too many edge cases to be possible in the timeframe you describe. It's also not necessary to create AGI in order to make a lot of money from AI. You can find the specific jobs that you want to replace and create a task-specific AI to do it.

Can you bench training with https://github.com/karpathy/nanoGPT and 100M+ GPT model?

Looks really cool. Great job. Is the code open-sourced? I'd like to learn how to do that sort of things