Recent comments in /f/MachineLearning

Affectionate_Leg_686 t1_j8ebfju wrote

Reply to comment by GFrings in [D] Is a non-SOTA paper still good to publish if it has an interesting method that does have strong improvements over baselines (read text for more context)? Are there good examples of this kind of work being published? by orangelord234

I second this adding that "reviewer roulette" is now the norm in other research communities too. Some conferences are making an effort to impriove the reviewing process, e.g., ICML has metareviewers and an open back-and-forth discussion between the authors and the reviewers. Still, it has not solved the problem.

​

Regarding your work, If possible, define a metric that encapsulates accuracy vs. cost (memory and compute), show how this varies across different established models, and then use that as part of your case: why is your model much more "efficient" than the alternative of running X models in parallel.

In my experience, using a proxy metric for cost is preferable for the ML crowd. I mean something like operation counts and bits transferred. Of course, if you can measure time on existing hardware, say a GPU or CPU that would be best.

Good luck!

8

AdamAlexanderRies t1_j8eb94a wrote

Reply to comment by daking999 in [D] Quality of posts in this sub going down by MurlocXYZ

I'm unaffiliated but pretty passionate about good design in general. Discord's really the spiritual successor to IRC, which predates the world wide web. The server-channel-role skeleton comes from IRC, but it's so feature rich and easy to use that I can see it supplanting a large portion of the social internet over the next decade. For the last month I've been developing my first discord bot (with chatgpt assistance) and the dev interface is excellent, too.

No experience with slack, so I can't comment on it.

5

ReginaldIII t1_j8e9sc3 wrote

Reply to comment by dojoteef in [D] Quality of posts in this sub going down by MurlocXYZ

It's been going downhill for a lot longer than that, and it's not something that can be solved with better moderation.

The people who are engaging with the sub in higher and higher frequencies than before simply do not know anything substantive about this field.

How many times will we have people try to asininely argue about stuff like a models "rights" or that "they" (the model) have "learned just like a person does", when the discussion should have just been about data licensing laws, intellectual property, and research ethics.

People just don't understand what it is that we actually do anymore.

25

pyepyepie t1_j8e7gjp wrote

Reply to comment by bballerkt7 in [R] [N] Toolformer: Language Models Can Teach Themselves to Use Tools - paper by Meta AI Research by radi-cho

Thanks :) I agree it's useful but I don't see how it's related to AGI. Additionally, it was already done a long time ago, many "AI" agents used the internet before. I feel that the real challenge is to control language models using structured data, perform planning, etc., not to use language models to interact with the world (which seems trivial to me, sorry), but of course, it's just my opinion - which is probably not even that smart.

6

cantfindaname2take t1_j8e4x9q wrote

Reply to comment by Cherubin0 in [D] Can Google sue OpenAI for using the Transformer in their products? by t0t0t4t4

No, it's not like that at all. IMO that analogy does not make any sense. First, r&d is not just thinking up stuff and then making them. In drug discovery it involves expensive trials. In other fields it may involve a lot of building and scraping things, sometimes from expensive material. Patent should be an incentive to do all that knowing that once that it's done it can monetized in a way that does not allow other companies just to copy and paste without effort. Should they be able to do it for everything and forever? Probably not and that is what I was referring to.

3

dojoteef t1_j8e2m8g wrote

Reply to [D] Quality of posts in this sub going down by MurlocXYZ

Tbh, it's because I took a step back and haven't been moderating the sub the past week and a half. I've been the one mod doing the majority of the filtering of these posts over the past couple of years and the noise has just been going up exponentially over that time. It's very time consuming and I'm pretty burned out doing it, so I've taken some time away. I brought this up with the other mods before stepping back a bit.

It's probably good to try to get more mods, but I think the majority of the current mods are afraid to hire on new mods that might have a different philosophy of moderating, thus changing the feel of the sub.

1

Glum-Mortgage-5860 t1_j8e23xa wrote

Reply to [D] What are resources to start with GNN and GraphML? by chhaya_35

IMO the order of papers should be, although i realise this may be a bit too much looking back

  • start off with spectral graph papers and the label propogation papers such as zhu 2003 zhou 2004.
  • then the spectral convolution papers such as defferrard 2016.
  • then the gcn paper and maybe the gat paper and how powerful are graph nns.

From there you are well set up to pick your poison on which type of graph ml to focus on. Dynamic vs static, hetro vs homo etc.

Some cool people to follow (i dont know much about the social media stuff)

Bronstein at twitter, petar velickovic at deep mind, xavier bresson, william hamilton. Sure there are loads more

Pytorch geometric and dgl have loads of good docs for practical examples.

2

womenrespecter-69 t1_j8dwqtl wrote

Reply to comment by cajmorgans in [D] Can Google sue OpenAI for using the Transformer in their products? by t0t0t4t4

Are you talking about swipe typing? It was a lot faster than peck typing back when phones were small enough to fit in one hand.

AFAIK the company that patented it (Swype) screwed up by making their patent more specific than they needed to and apple/google were able to work around it without licensing it. They eventually got acquired and killed a few years ago.

1

uhules t1_j8dsggs wrote

Reply to comment by dustintran in [D] Quality of posts in this sub going down by MurlocXYZ

Aside from "We've just published X" threads (which are usually comprised of healthy praises, questions and critiques), I loathe most ML twitter discussions. They tend to have all the usual "hot take" issues from the platform, even from prominent names in the field. Not really a great place to discuss ML as a whole.

4

qalis t1_j8driqb wrote

Reply to [D] What are resources to start with GNN and GraphML? by chhaya_35

I am working in this field for my PhD, so I think I can help.

A bit of self promotion, but my Master's thesis was about GNNs: https://arxiv.org/abs/2211.03666. It should be very beginner-friendly, since I had to write it while also learning about this step by step.

"Introduction to. Graph Neural Networks". Zhiyuan Liu and Jie Zhou. Tsinghua University is slightly outdated due to how fast this field is going on, but good intro.

"Graph Neural Networks Foundations, Frontiers, and Applications" (https://graph-neural-networks.github.io/) is cutting-edge, good reviews. I haven't read it though, but looks very promising.

Overviews and articles are also great, e.g. https://distill.pub/2021/gnn-intro/ or a well known (in this field) https://arxiv.org/abs/1901.00596. You should also definitely read papers about GCN (very intuitively written), GAT, GraphSAGE and GIN, the most classic 4 graph convolution architectures.

Fair comparison is, unfortunately, not common in this field. Many well-known works, e.g. GIN, do not even use a test set, and are quite unclear about this, so approach every paper with a lot of suspicion. This paper about fair comparison is becoming more and more used: https://arxiv.org/abs/1912.09893. This baseline, not GNN but similar, gives very strong results: https://arxiv.org/abs/1811.03508. I will be releasing a paper about a related method, LTP (Local Topological Profile), you can look out for it in the later part of the year.

Other interesting architectures to read about: graph transformers, Simple Graph Convolution (SGC), DiffPool, gPool, PinSAGE, DimeNet.

This very exciting area is just starting to develop, despite a lot of work done. There is no well working way to do transfer learning, for example. It is very hard to predict what will happen in 4-5 years, but e.g. Google Maps travel time prediction is currently based on GAT, and Pinterest recommendations on PinSAGE, so graph-based ML is already used in large-scale production systems. Those methods are also more and more commonly used in biological sciences, where molecular data is ubiquitous.

3

csreid t1_j8dqcrs wrote

Reply to [D] Quality of posts in this sub going down by MurlocXYZ

I like that /r/science (I think?) has verification and flair to show levels of expertise in certain areas, and strict moderation. I wouldn't hate some verification and a crackdown on low-effort bloom-/doom-posting around AI ("How close are we to star trek/skynet?").

1