Recent comments in /f/MachineLearning

coconautico OP t1_ja3nvs7 wrote

I have manually copy-pasted a few interesting questions (i.e, my input) that I have asked chatGPT previously, that encouraged lateral thinking or required specialized knowledge.

However, I'm not so sure it would a good idea to load thousands of questions indiscriminately, because just as we wouldn't express a question on Reddit in the same way we would in person, when we ask a question to chatGPT (or Google), we slightly modify the way we talk by taking into account the weaknesses of the system. And given that we are looking for a high-quality dataset of natural conversations, I don't think this would be a very good strategy in the short term.

Moreover, we also have to consider that the project prioritizes quality above all else, and unless the number of volunteers ranking questions/replies increases considerably, the "ratio of trees to ready exported" wouldn't increase much either.

3

MysteryInc152 OP t1_ja3hozj wrote

>Deep-learning language models have shown promise in various biotechnological applications, including protein design and engineering. Here we describe ProGen, a language model that can generate protein sequences with a predictable function across large protein families, akin to generating grammatically and semantically correct natural language sentences on diverse topics. The model was trained on 280 million protein sequences from >19,000 families and is augmented with control tags specifying protein properties. ProGen can be further fine-tuned to curated sequences and tags to improve controllable generation performance of proteins from families with sufficient homologous samples. Artificial proteins fine-tuned to five distinct lysozyme families showed similar catalytic efficiencies as natural lysozymes, with sequence identity to natural proteins as low as 31.4%. ProGen is readily adapted to diverse protein families, as we demonstrate with chorismate mutase and malate dehydrogenase.

28

tradegreek t1_ja336tf wrote

I have just been testing out some machine learning as I am new to it and have a simple dataset currently 500k rows the target value is literally the sum of each row. I was using model = Sequential()model.add(Dense(4, input_dim=4, activation='ELU'))model.add(Dense(1, activation='linear'))# Compile the modelmodel.compile(loss='mean_squared_error', optimizer='adam') model.fit(X, y, epochs=4000, verbose=1, batch_size=120). I then fed the model some unseen data to see if it could make the new calculations again literally just sum up the values.

22000 - 11000 - 6000 - 1500 should equal 3500 but instead, i got 3499.9915. The results for the other new unseen data was all similar I was wondering how i can fix this I know ai models need a lot of data but surely for something so trivial I would have expected it to get the values perfectly correct. My long term goal is to build data validation through calculations which is why i am practicing such a basic model.

1

visarga t1_ja2r2fe wrote

Wouldn't it be better if people could donate their interactions with chatGPT, BingChat and other models? Make a scraping extension, it should collect chat logs and anonymise them. Then you got a diverse distribution of real life tasks.

I suspect this is the reason OpenAI and Bing offered their models for free to the public - to find the real distribution of tasks people want to solve with AI bots.

9

alterframe t1_ja2f7xu wrote

Part of the answer is probably that DL is not a single algorithm or a class of algorithms, but rather a framework or a paradigm for building such algorithms.

Sure, you can take a SOTA model for ImageNet and apply it to similar image classification problems, by tuning some hyperparameters and maybe replacing certain layers. However, if you want to apply it to a completely different task, you need to build a different neural network.

1

D4rkthorn t1_ja2d441 wrote

Expect to get rejected. Don't take the conference too seriously, they are usually filled with people who fell very important, so it can be necessary to drink your brains out to get through it.

If you get accepted remember to make a poster.

4

radi-cho OP t1_ja27qnn wrote

Paper: https://arxiv.org/pdf/2302.12251.pdf GitHub: https://github.com/nvlabs/voxformer

Abstract: Humans can easily imagine the complete 3D geometry of occluded objects and scenes. This appealing ability is vital for recognition and understanding. To enable such capability in AI systems, we propose VoxFormer, a Transformer-based semantic scene completion framework that can output complete 3D volumetric semantics from only 2D images. Our framework adopts a two-stage design where we start from a sparse set of visible and occupied voxel queries from depth estimation, followed by a densification stage that generates dense 3D voxels from the sparse ones. A key idea of this design is that the visual features on 2D images correspond only to the visible scene structures rather than the occluded or empty spaces. Therefore, starting with the featurization and prediction of the visible structures is more reliable. Once we obtain the set of sparse queries, we apply a masked autoencoder design to propagate the information to all the voxels by self-attention. Experiments on SemanticKITTI show that VoxFormer outperforms the state of the art with a relative improvement of 20.0% in geometry and 18.1% in semantics and reduces GPU memory during training by ~45% to less than 16GB.

31