Yup. And then they also will detect human texts that start in the same way as MS COCO dataset as GTP-generated.

If you don't engage the second GPU, it will remain dormant, and should not automatically interfere with anything. For example if you are training a network in PyTorch without using DP or DDP, then it will use the first GPU by default. You can always change which GPU it uses using the environment variable CUDA_VISIBLE_DEVICES. Also, make sure the primary GPU occupies the first PCIe slot. You could verify this with nvidia_smi. When you have the display hooked up to it, the primary GPU will have a slightly higher memory usage (~100 MB) because of display server processes like Xorg, than all other GPUs.

Looking at the model configs in the repo there’s definitely dynamic batching going on.

I think what’s really interesting is the fact that even with default parameters for dynamic batching the response times are superior and very consistent.

Is it possible to disable the secondary GPU when not performing ML tasks so that it doesn't interfere with other tasks?

Depending on how much time I have there just might be a PR coming your way 😀…

Triton is really a somewhat hidden gem - the implementation and toolkit surrounding it is pretty impressive!

Just create an algorithm that adds some unusual space characters which acts like a public key.

Tim Dettmers wrote about this in one of his articles. AFAIK, SLI is not required for DL (it’s a gaming thing where sync between GPUs becomes important for smooth gameplay). In DL tasks, any GPU can just wait for others to finish. So you can use any combination of any number of Nvidia GPUs as long as you can interface with them (PCIe or Ethernet). The catch is that the speed of training/inference will be limited by the weakest link in the chain, i.e. the weakest GPU will bottleneck all other GPUs. But on the flip side, you should be able to fit more data owing to the increased VRAM.

The other thing that you can do is run two different experiments on each GPU simultaneously. In that way, you can maximize the usage of your GPUs.

If you do want to fit more data on the 3080, look for pytorch plug-ins, such as deepspeed or FP16 or simply do two forward passes per backward pass, which will double your batch size.

>trivially obvious that AI should never be open-source

Wow. Trivially obvious? I'd very much like to know how that statement is trivially obvious, because it goes against what pretty much every single expert in this field advocates.

Obviously open-source AI brings problems, but what is the alternative? A single entity controlling one of the most disrupting technologies ever? And ignoring for a second the obvious problems with that, how would you enforce it? Criminalize open-sourcing of software? Can't say I'm a fan of this line of thinking.

I think the worst possible idea is allowing a single person or handful of people to have near-total control over the future of AI, which will be the future of humanity. The process should be democratized as much as can be. Open source is one way to accomplish that, though it brings its own dangers as well

Implicit in OP’s question is the apparent assumption that COVID is not a factor in projected demand. If that assumption is not true, then they may need to use the last 3 years as their baseline rather than trying to work around them.

Thank you for the feedback. I am looking forward to using the Triton's model analyzer possibly with different batch sizes and also FP16! Lets see how that goes :)

Check out gptZero as one example. It uses perplexity and other characteristic differences between human and AI generated text. Not perfect, but works on longer text passages. Unfortunately, on can train AI to have more variation, this defeating the detector.

I used a detector called GPTZero and it did pretty good, but completely missed something written as a tweet or in the style of…

Signed authenticity for news and other high quality human content needs to scale. I know some news ors have been working on this for years. It is time to roll it out at scale.

Great book. Required reading back in the mid 90s when I worked at WIRED.

Reminds me of the movie multiplicity, in which each copy gets dumber.

For top-k product search you could also follow this: Index all product embeddings through faiss. To get a top match for a given product, take it's embedding & query it with built faiss index, you'll get top-k matches from it. This is way faster than brute force comparision between each pair.

Faiss: https://github.com/facebookresearch/faiss

Could you? Probably, but with a nontrivial error rate. Should you? No, that would make YTA.

This.

Or in human speech. Very faint but there, so that no matter how anonymously you post, you write a sentence of enough words and your identity leaks.

Kinda one of the premises of the book Dodge from Neil Stephenson.

Yeah but its trivial to remove that when you run the source yourself.

> Besides that, OP stated that he wants to use a llm for this, not me.

Actually, you introduced that concept first when you said:

> If u want some AI to alter the text for you, you again need a LLM.

OP had not mentioned applying an LLM to the case prior to that. It was explicit in their original comment, and implicit in all comments thereafter, that a watermark-free LLM was only one of the ways in which this problem could be tackled.

Meanwhile:

> Synonym engines wouldnt change an n-gram watermarks significantly enough as a synonym is the same type of word so there are token patterns persisting.

Right. Hence why I said they "get halfway there". Halfway is clearly not "all the way", and thus not "significantly enough".

And finally:

> Rules for r/MachineLearning > 1. Be nice: no offensive behavior, insults or attacks

In light of your recent description of an interlocutor's "limited capacity brain", you seem to be catastrophically failing at (1) understanding the problem space being discussed, (2) understanding the deficiencies in your own arguments, and (3) understanding basic norms and rules of interpersonal decency....

Just my two cents, but this forum probably isn't the right space for you until you level up a bit.

Try models that exclude it, include it, and try to compare potential variance 2022 onwards. You’re right that pandemic prediction is out of scope, but assessing variation and noise is a great way to show maturity to your company. :)

Maybe say if we can download our answer then in the file we get the answer some watermark is encoded.

> I wonder to what extent we’ll manage to get science fiction out about these things before we start seeing them in practice.

Its not an exact match, but reminds me quite a lot of Snow Crash