on to the next image. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. In. While this class draws upon ideas from optimization, it's not an optimization class. In. This code replicates the experiments from the following paper: Understanding Black-box Predictions via Influence Functions. He, M. Narayanan, S. Gershman, B. Kim, and F. Doshi-Velez. We'll see first how Bayesian inference can be implemented explicitly with parameter noise. The ACM Digital Library is published by the Association for Computing Machinery. We have a reproducible, executable, and Dockerized version of these scripts on Codalab. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. Gradient-based Hyperparameter Optimization through Reversible Learning. Infinite Limits and Overparameterization [Slides]. Rethinking the Inception architecture for computer vision. We would like to show you a description here but the site won't allow us. Liu, D. C. and Nocedal, J. Model selection in kernel based regression using the influence function. Understanding Black-box Predictions via Influence Functions - YouTube AboutPressCopyrightContact usCreatorsAdvertiseDevelopersTermsPrivacyPolicy & SafetyHow YouTube worksTest new features 2022. This will naturally lead into next week's topic, which applies similar ideas to a different but related dynamical system. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. %PDF-1.5 Riemannian metrics for neural networks I: Feed-forward networks. RelEx: A Model-Agnostic Relational Model Explainer influence function. Stochastic gradient descent as approximate Bayesian inference. thereby identifying training points most responsible for a given prediction. If you have questions, please contact Pang Wei Koh (pangwei@cs.stanford.edu). In this paper, we use influence functions --- a classic technique from robust statistics --- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. numbers above the images show the actual influence value which was calculated. << x\Y#7r~_}2;4,>Fvv,ZduwYTUQP }#&uD,spdv9#?Kft&e&LS 5[^od7Z5qg(]}{__+3"Bej,wofUl)u*l$m}FX6S/7?wfYwoF4{Hmf83%TF#}{c}w( kMf*bLQ?C}?J2l1jy)>$"^4Rtg+$4Ld{}Q8k|iaL_@8v Not just a black box: Learning important features through propagating activation differences. S. L. Smith, B. Dherin, D. Barrett, and S. De. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through . 2172: 2017: . If there are n samples, it can be interpreted as 1/n. Online delivery. calculated. In, Mei, S. and Zhu, X. >> We'll mostly focus on minimax optimization, or zero-sum games. Optimizing neural networks with Kronecker-factored approximate curvature. Influence functions efficiently estimate the effect of removing a single training data point on a model's learned parameters. Understanding Black-box Predictions via Influence Functions ": Explaining the predictions of any classifier. S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. . To scale up influence functions to modern machine learning settings, Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Muller. Gradient descent on neural networks typically occurs on the edge of stability. To manage your alert preferences, click on the button below. Understanding Black-box Predictions via Influence Functions ICML2017 3 (influence function) 4 We'll consider two models of stochastic optimization which make vastly different predictions about convergence behavior: the noisy quadratic model, and the interpolation regime. Chatterjee, S. and Hadi, A. S. Influential observations, high leverage points, and outliers in linear regression. How can we explain the predictions of a black-box model? Dependencies: Numpy/Scipy/Scikit-learn/Pandas Differentiable Games (Lecture by Guodong Zhang) [Slides]. calculate which training images had the largest result on the classification While one grad_z is used to estimate the This site last compiled Wed, 08 Feb 2023 10:43:27 +0000. Neither is it the sort of theory class where we prove theorems for the sake of proving theorems. Cook, R. D. Detection of influential observation in linear regression. The security of latent Dirichlet allocation. Helpful is a list of numbers, which are the IDs of the training data samples vector to calculate the influence. Either way, if the network architecture is itself optimizing something, then the outer training procedure is wrestling with the issues discussed in this course, whether we like it or not. Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. On the importance of initialization and momentum in deep learning. Most weeks we will be targeting 2 hours of class time, but we have extra time allocated in case presentations run over. You signed in with another tab or window. Therefore, this course will finish with bilevel optimziation, drawing upon everything covered up to that point in the course. We are given training points z 1;:::;z n, where z i= (x i;y i) 2 XY . Imagenet classification with deep convolutional neural networks. Why Use Influence Functions? Understanding short-horizon bias in stochastic meta-optimization. On the accuracy of influence functions for measuring group effects. Apparently this worked. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. When can we take advantage of parallelism to train neural nets? Here, we used CIFAR-10 as dataset. Deep learning via Hessian-free optimization. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Understanding Black-box Predictions via Influence Functions - ResearchGate A. Lectures will be delivered synchronously via Zoom, and recorded for asynchronous viewing by enrolled students. NIPS, p.1097-1105. Cook, R. D. and Weisberg, S. Characterizations of an empirical influence function for detecting influential cases in regression. Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. On the origin of implicit regularization in stochastic gradient descent. Check if you have access through your login credentials or your institution to get full access on this article. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. (b) 7 , 7 . Validations 4. ( , ) Inception, . Rather, the aim is to give you the conceptual tools you need to reason through the factors affecting training in any particular instance. We'll start off the class by analyzing a simple model for which the gradient descent dynamics can be determined exactly: linear regression. Understanding Black-box Predictions via Influence Functions Simonyan, K., Vedaldi, A., and Zisserman, A. Google Scholar Digital Library; Josua Krause, Adam Perer, and Kenney Ng. Understanding Black-box Predictions via Influence Functions Unofficial implementation of the paper "Understanding Black-box Preditions via Influence Functions", which got ICML best paper award, in Chainer. In. Haoping Xu, Zhihuan Yu, and Jingcheng Niu. where the theory breaks down, Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., and Suh, J. Modeltracker: Redesigning performance analysis tools for machine learning. Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks. PVANet: Lightweight Deep Neural Networks for Real-time Object Detection. On linear models and convolutional neural networks, The details of the assignment are here. The marking scheme is as follows: The problem set will give you a chance to practice the content of the first three lectures, and will be due on Feb 10. I am grateful to my supervisor Tasnim Azad Abir sir, for his . After all, the optimization landscape is nonconvex, highly nonlinear, and high-dimensional, so why are we able to train these networks? This will also be done in groups of 2-3 (not necessarily the same groups as for the Colab notebook). A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach. For this class, we'll use Python and the JAX deep learning framework. Christmann, A. and Steinwart, I. (a) train loss, Hessian, train_loss + Hessian . Understanding Black-box Predictions via Influence Functions - Github In, Mei, S. and Zhu, X. Understanding Black-box Predictions via Influence Functions Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. A theory of learning from different domains. Highly overparameterized models can behave very differently from more traditional underparameterized ones. , . GitHub - kohpangwei/influence-release With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. A. S. Benjamin, D. Rolnick, and K. P. Kording. prediction outcome of the processed test samples. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Despite its simplicity, linear regression provides a surprising amount of insight into neural net training. use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Fast exact multiplication by the hessian. ( , , ). As a result, the practical success of neural nets has outpaced our ability to understand how they work. Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., and Darrell, T. Decaf: A deep convolutional activation feature for generic visual recognition. , . So far, we've assumed gradient descent optimization, but we can get faster convergence by considering more general dynamics, in particular momentum. Time permitting, we'll also consider the limit of infinite depth. In. Yuwen Xiong, Andrew Liao, and Jingkang Wang. . Debruyne, M., Hubert, M., and Suykens, J. We try to understand the effects they have on the dynamics and identify some gotchas in building deep learning systems. All information about attending virtual lectures, tutorials, and office hours will be sent to enrolled students through Quercus. 10.5 Influential Instances | Interpretable Machine Learning - GitHub Pages most harmful. A. The datasets for the experiments can also be found at the Codalab link. The answers boil down to an observation that neural net training seems to have two distinct phases: a small-batch, noise-dominated phase, and a large-batch, curvature-dominated one. Metrics give a local notion of distance on a manifold. For details and examples, look here. We look at what additional failures can arise in the multi-agent setting, such as rotation dynamics, and ways to deal with them. Influence functions are a classic technique from robust statistics to identify the training points most responsible for a given prediction. ordered by helpfulness. Abstract. Please download or close your previous search result export first before starting a new bulk export. The algorithm moves then The deep bootstrap framework: Good online learners are good offline generalizers. In order to have any hope of understanding the solutions it comes up with, we need to understand the problems. Datta, A., Sen, S., and Zick, Y. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. ICML 2017 best paperStanfordPang Wei KohPercy liang, x_{test} y_{test} label x_{test} , n z_1z_n z_i=(x_i,y_i) L(z,\theta) z \theta , \hat{\theta}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta), z z \epsilon ERM, \hat{\theta}_{\epsilon,z}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta)+\epsilon L(z,\theta), influence function, \mathcal{I}_{up,params}(z)={\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0}=-H_{\hat{\theta}}^{-1}\nabla_{\theta}L(z,\hat{\theta}), H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta) Hessien, \begin{equation} \begin{aligned} \mathcal{I}_{up,loss}(z,z_{test})&=\frac{dL(z_{test},\hat\theta_{\epsilon,z})}{d\epsilon}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T {\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T\mathcal{I}_{up,params}(z)\\&=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta) \end{aligned} \end{equation}, lossNLPer, influence function, logistic regression p(y|x)=\sigma (y \theta^Tx) \sigma sigmoid z_{test} loss z \mathcal{I}_{up,loss}(z,z_{test}) , -y_{test}y \cdot \sigma(-y_{test}\theta^Tx_{test}) \cdot \sigma(-y\theta^Tx) \cdot x^{T}_{test} H^{-1}_{\hat\theta}x, \sigma(-y\theta^Tx) outlieroutlier, x^{T}_{test} x H^{-1}_{\hat\theta} Hessian \mathcal{I}_{up,loss}(z,z_{test}) resistencevariation, \mathcal{I}_{up,loss}(z,z_{test})=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta), Hessian H_{\hat\theta} O(np^2+p^3) n p z_i , conjugate gradientstochastic estimationHessian-vector productsHVP H_{\hat\theta} s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta) \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta) , H_{\hat\theta}^{-1}v=argmin_{t}\frac{1}{2}t^TH_{\hat\theta}t-v^Tt, HVPCG O(np) , H^{-1} , (I-H)^i,i=1,2,\dots,n H 1 j , S_j=\frac{I-(I-H)^j}{I-(I-H)}=\frac{I-(I-H)^j}{H}, \lim_{j \to \infty}S_j z_i \nabla_\theta^{2} L(z_i,\hat\theta) H , HVP S_i S_i \cdot \nabla_\theta L(z_{test},\hat\theta) , NMIST H loss , ImageNetInceptionRBF SVM, RBF SVMRBF SVM, InceptionInception, Inception, , Inception591/60059133557%, check \mathcal{I}_{up,loss}(z_i,z_i) z_i , 10% \mathcal{I}_{up,loss}(z_i,z_i) , H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta), s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta), \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta), S_i \cdot \nabla_\theta L(z_{test},\hat\theta).
