Recent comments in /f/MachineLearning

the_scign t1_j43jx5z wrote

Reply to [D] Would you consider the computer program Theo Jansen used to design the Strandbeest (beach walking mechanisms) to be Machine Learning? by lavaboosted

I tend to think of "machine learning" as the use of some automated algorithm to learn a ruleset as opposed to manually programming that ruleset. More often than not this algorithm requires some external dataset from which to learn the rules but in this case the algorithm is using another ruleset configured by Jansen to learn the rules. In that sense, since there was an automated algorithm that generated a "model" that abided by a set of externally provided rules, I would class this as machine learning.

That said, some people consider only scenarios where external data points were provided, rather than a set of rules, as machine learning. They may be right and I may be wrong - I'm open to debate on that.

2

suflaj t1_j43eanz wrote

Reply to [D] Can someone point to research on determining usefulness of samples/datasets for training ML models? by HFSeven

Well depends on what usefulness is.

If you can prove that all of your samples belong to the same distribution, then simply looking up which have the greatest gradient norm will be a measure of how useful they are for the model. Another approach is looking at how much their contribution would be in improving the performance of other samples, but then your dataset becomes a dependent variable.

But obviously this is dependent on the current weights, the loss function and various other biases. This is because gradient norm is proportional to the error, and so the samples for which the model predicts the most erroneous result will end up being most useful, given the perfect LR for it.

1

BrisklyBrusque t1_j43dsux wrote

Reply to [D] Are there any papers on optimization-based approaches which combine learned parameter initializations with learned optimisers? by Decadz

You might enjoy “Well-Tuned Simple Nets Excel on Tabular Data”

https://arxiv.org/abs/2106.11189

Authors wrote a computer routine that leverages BOHB (Bayesian optimization and Hyberband) to search an enormous search space of possible neural network architectures. The authors allowed the routine to select different regularization techniques, including many ensemble techniques like dropout, snapshot ensembles, and others that render the choice of parameter initializations less critical. However, authors used the same optimizer (AdamW) in all experiments.

Not exactly what you are looking for but hopefully interesting.

3

TheGreatHomer t1_j433r8m wrote

Reply to comment by lavaboosted in [D] Would you consider the computer program Theo Jansen used to design the Strandbeest (beach walking mechanisms) to be Machine Learning? by lavaboosted

>What do you think of the car example

I haven't read the paper, but only watched the brief video. I wouldn't say that's Machine Learning either.

Maybe a bad analogy but one I can come up with on a spot: A hinge isn't carpentry but metalwork and pretty much everyone agrees on that. Now if you build a wooden cabinet, you are probably using hinges; Nevertheless, you'd still call the cabinet as such carpentry, not metalwork.

Anyway, the definitions aren't clear and consistent enough to make super good and objectively true distinctions. In the end it often boils down to personal subjective interpretations.

Edit: Especially the classification of evolutionary algorithms has been an ongoing discussion for, like, decades. Which goes to show that there probably isn't an objectively right clear classification - if only because people don't agree on a single definition of Machine Learning as is. However, by the most common definitions that I know, evolutionary computation is its own subfield next to ML.

0

lavaboosted OP t1_j431okr wrote

Reply to comment by CurrentMaleficent714 in [D] Would you consider the computer program Theo Jansen used to design the Strandbeest (beach walking mechanisms) to be Machine Learning? by lavaboosted

Interesting, thanks. It seems a lot of people do lump it in with machine learning such as this video using a neural network and evolutionary algorithm to teach a car to drive around a track. Does the use of a feed forward neural network make it qualify as machine learning or still no? Or is it just a gray area?

1

Insighteous t1_j430uuc wrote

Reply to comment by chatterbox272 in [D] Is making a dataset publicly accessible necessary for acceptance at top-tier conferences in ML? by newperson77777777

Publishing everything is a good thing. At the moment I am trying to reproduce some results of a paper and have to work with „we created X datasets by three methods“. And NO WHERE in the paper it is stated what these three methods are. Also no code.

It is so annoying. Cannot put it in words.

1

suflaj t1_j4308sf wrote

Reply to comment by manOnPavementWaving in [D] Is there a distilled/smaller version of CLIP, or something similar? by alkibijad

Ah, wasn't aware they published the weights. But if that's too big I am not aware of anything significantly smaller that would retain most of the performance.

It should be relatively easy to pretrain a significantly smaller network yourself given the pretrained resnet weights with good enough sampling and a month or so training...

2