Feds start making their honeypot model trained with irl cp 
@MoeBritannica So how was it confirmed? Screenshot of a post that says another guy got a letter?
@applejack the torrent hashes is right there 
@MoeBritannica Of the models, ye? How's it confirmed it's trained on CP?
His ISP may have just detected torrents going on and sent him legal threats against piracy for all that post actually says, if the guy that says he got a letter isn't lying, if the guy in the screenshot isn't lying, if the screenshot is real to begin with
@applejack @MoeBritannica >How's it confirmed it's trained on CP?
It's possible to reconstruct the training data from the model, and there are multiple papers on how one might achieve this. This is a problem inherent to neural networks, and cannot be avoided no matter what the architecture or training method is.
It is why the US government *cannot* provide neural networks that detect CP to big tech companies - this is functionally equivalent to giving them a giant pile of CP.
It's possible to reconstruct the training data from the model, and there are multiple papers on how one might achieve this. This is a problem inherent to neural networks, and cannot be avoided no matter what the architecture or training method is.
It is why the US government *cannot* provide neural networks that detect CP to big tech companies - this is functionally equivalent to giving them a giant pile of CP.
Follow
@ceo_of_monoeye_dating @MoeBritannica Really? I would have thought it's just a one-way-function that ends up as a bunch of weights
I know the trained data becomes huge though so I guess that sounds plausible
The upshot is that a neural network can be treated as a database which poorly fetches information from the training set. (This is NOT how experts think of it typically, but it is true). Because of this, it's possible that someone with the right information might come along and grab information you don't really want them grabbing [1].
For literally the first example of someone doing this I came across on google, see [2]. If you want to start diving into how long people have been doing this, check the references. (I am surprised they don't mention Dwork's paper at all, as it's sort of fundamental.)
1) http://audentia-gestion.fr/MICROSOFT/dwork.pdf
2) https://arxiv.org/abs/2206.07758