Recent comments in /f/MachineLearning

Oceanboi t1_j5n6p7b wrote

Reply to comment by [deleted] in [D] Simple Questions Thread by AutoModerator

my advice is to proceed. its cool to know the math underneath, but just go implement stuff dude, if it doesn't work you can always remote/rent GPU. what i did for my thesis is google tutorials and re-implement them using my dataset. through all the bugs and the elbow grease, you will know enough to at least speak the language. just do it and don't procrastinate with these types of posts (i do this too sometimes)

EDIT: a lot can be done on colab these days regarding neural networks and huggingface. google huggingface documentation! i implemented a huggingface transformer model to do audio classification (and im a total noob i just copied a tutorial). it was total misuse of the model and accuracy was bad, but at least i learned and given a real problem i could at least find my way forward.

1

FastestLearner t1_j5mwz47 wrote

Reply to comment by ArnoF7 in [D] Multiple Different GPUs? by Maxerature

It is possible, but it would require you to write custom code for every memcopy operation that you want to perform i.e. tensor.to(device), which you can get away with on a smaller project but could become prohibitively cumbersome on a large project. Also you'd still need to do two forward passes (one with the data on the 3080 itself, and then another with the data on the 1080 after having it transferred to the 3080). Whether or not this is beneficial boils down to differences in transfer rates between the RAM-3080 route and the RAM-1080-3080 route. I won't be able to tell which one is faster without benchmarking.

DeepSpeed handles the RAM-3080 to-and-fro transfers for large batch sizes automatically.

1

suflaj t1_j5mnzq3 wrote

Reply to comment by WigglyHypersurface in [D] Embedding bags for LLMs by WigglyHypersurface

Why would this matter?

If such examples are present in the training set and adequately expressed, then the model will learn whatever it needs to learn from those words.

If they are not in the training set, you should not expect the model to understand them the same way you do.

I realize this defeats the point of generalization, but LLMs learn to mimic generalization through exposure, not by actually learning to understand the underlying principles. These models do not analyze text like we humans do, but they have been shown to outperform the average human despite that.

Ultimately to do what you are doing you would need to have a tokenizer that has all the syntactical knowledge embedded within itself for a given subset of the language that will be the input. Wasn't AlexNet, a decade ago, enough to convince you to always relegate these kinds of tasks to the DL model, which will always beat a human provided it has the capacity and the data?

0

Pavarottiy t1_j5mhsjq wrote

Reply to comment by EducationalLayer1051 in [D] Automated Extraction of Building Geometry by EducationalLayer1051

Yes, it is a general line detection method, so any point cloud input with lines in it can be inputted. For 3d scans, point clouds have some sensor noise and a robust implementation/version of this method would be better suited. But in case of a perfect cad model like the example, even the one that I provided in my prev comment should work fine. Do you only want to detect lines, or planes and lines? For planes, there are plane fitting approaches and then, one can find intersection of planes (as an alternative to Hough).

1