Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How may predictions to steal one ANN model with 6 hidden layers? #1

Open
wanghs09 opened this issue Jan 5, 2017 · 2 comments
Open

Comments

@wanghs09
Copy link

wanghs09 commented Jan 5, 2017

Dear Florian,

May I ask how may predictions the code needs to get one black box ANN(artificial neural network) or MLP in your paper? Suppose there are 200 neurons per layer, the total number of parameters is a lot.

Looking forward to your reply!
Thanks a lot!

@ftramer
Copy link
Owner

ftramer commented Jan 8, 2017

This really depends on the total number of parameters in the network, and whether you assume access to the exact class probabilities or just the predicted class labels.. In some of our experiments, we achieved high extraction accuracy (>98%) with 10 times less predictions than there were parameters in the network.
My guess is that if you use a much smaller number of predictions, you can still get a non-trivial extraction accuracy (maybe around 80%) but we haven't tried that out on deep networks.

@wanghs09
Copy link
Author

wanghs09 commented Jan 9, 2017

Thanks, Florian! Actually 90%+ accuracy is surprisingly good enough without the training data.
Just want to make sure that I understand you correctly :D,

  • What's the max hidden layer for ANN you tested? 3, 4?
  • As far as I know, to make the training job easy, ANNs usually use local and the same weight for adjacent inputs(convolutional network), so the unknown parameters are actually not so many. But if the extracted model(by reverse engineering) is not exactly the same original one, would there be a problem that the model will behave differently in some circumstances that were not tested?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants