Wengert, R. E. (1964). A simple automatic derivative evaluation program. Communications of the ACM, 7(8), 463–464.
A. Radul, Introduction to Automatic Differentiation , 2013. [Online]. A vailable:https://alexey.radul.name/ideas/2013/introduction- to-automatic-differentiation/.
Hogan, R. J. (2014). Fast reverse-mode automatic differentiation using expres- sion templates in c++. ACM Transactions on Mathematical Software (TOMS), 40(4), 1–16.
C. H. Bischof, M. R. Haghighat, et al., “Hierarchical approaches to auto- matic differentiation,” Computational Differentiation: Techniques, Ap- plications, and Tools , pp. 83–94, 1996.
S. P. Jones, Automatic differentiation for dummies . [Online]. A vailable: https : / / 2019 . ecoop . org / details / ecoop - 2019 - papers / 11 / Automatic-Differentiation-for-Dummies .
D. Oktay, N. McGreivy, J. Aduol, A. Beatson, and R. P. Adams, Ran- domized automatic differentiation , 2021. arXiv: 2007.10412 [cs.LG] .
J. M. Siskind, “Automatic differentiation: Inverse accumulation mode,” in Program Transformations for ML Workshop at NeurIPS 2019 .
Backpropagation
T. Vieira, Backprop is not just the chain rule . [Online]. A vailable:https: //timvieira.github.io/blog/post/2017/08/18/backprop-is-not- just-the-chain-rule/ .
W. Wang, Z. Dang, Y. Hu, P. Fua, and M. Salzmann, Backpropagation- friendly eigendecomposition, 2019. arXiv: 1906.09023 [cs.LG] .
Differentiable Optimization
A. Agrawal, B. Amos, S. Barratt, S. Boyd, S. Diamond, and Z. Kolter, Differentiable convex optimization layers, 2019. arXiv: 1910.12430 [cs.LG].
B. Amos and J. Z. Kolter, Optnet: Differentiable optimization as a layer in neural networks , 2021. arXiv: 1703.00443 [cs.LG] .
D. Maclaurin, D. Duvenaud, and R. Adams, “Gradient-based hyper- parameter optimization through reversible learning,” in Proceedings of the 32nd International Conference on Machine Learning , F. Bach and D. Blei, Eds., ser. Proceedings of Machine Learning Research, vol. 37,
Lille, France: PMLR, 2015, pp. 2113–2122. [Online]. A vailable: https: //proceedings.mlr.press/v37/maclaurin15.html.
Y. Li, A. Božič, T. Zhang, Y. Ji, T. Harada, and M. Nießner, “Learning to optimize non-rigid tracking,” in Proc. Computer Vision and Pattern Recognition (CVPR), IEEE , 2020.
A. Mensch and M. Blondel, Differentiable dynamic programming for structured prediction and attention, 2018. arXiv: 1802.03676 [stat.ML].
Differentiable Programming
K. Farrahi and J. Hare, COMP6248 Differentiable Programming (and Deep Learning) , 2022. [Online]. A vailable: http : / / comp6248 . ecs . soton.ac.uk.
R. O’Connor, Differentiable programming - a simple introduction , 2022. [Online]. A vailable:https://www.assemblyai.com/blog/differentiable- programming-a-simple-introduction/ .
M. C. Lin, CMSC 838B / 498Z: Differentiable Programming , 2021. [Online]. A vailable: https : / / www . cs . umd . edu / class / fall2021 / cmsc838b/.
Y. Hu, L. Anderson, T.-M. Li, et al., “Difftaichi: Differentiable program- ming for physical simulation,” ICLR, 2020.
Z. Huang, Y. Hu, T. Du, et al., “Plasticinelab: A soft-body manipulation benchmark with differentiable physics,” arXiv preprint arXiv:2104.03311, 2021.
A. McNamara, A. Treuille, Z. Popović, and J. Stam, “Fluid control using the adjoint method,” ACM Transactions On Graphics (TOG) , vol. 23, no. 3, pp. 449–456, 2004.
F. de A vila Belbute-Peres, K. Smith, K. Allen, J. Tenenbaum, and J. Z.
Kolter, “End-to-end differentiable physics for learning and control,” Ad-
vances in neural information processing systems , vol. 31, 2018.
J. Liang, M. Lin, and V. Koltun, “Differentiable cloth simulation for inverse problems,” in Advances in Neural Information Processing Sys- tems, vol. 32, Curran Associates, Inc., 2019. [Online]. A vailable: https: / / proceedings . neurips . cc / paper _ files / paper / 2019 / file / 28f0b864598a1291557bed248a998d4e-Paper.pdf.
Y.-L. Qiao, J. Liang, V. Koltun, and M. C. Lin, Scalable differentiable physics for learning and control , 2020. arXiv: 2007.02168 [cs.LG] .
X. Li, T.-K. L. Wong, R. T. Q. Chen, and D. Duvenaud, Scalable gra- dients for stochastic differential equations , 2020. arXiv: 2001 . 01328 [cs.LG].
K. Um, R. Brand, Y. R. Fei, P. Holl, and N. Thuerey, “Solver-in-the- loop: Learning from differentiable physics to interact with iterative pde- solvers,” Advances in Neural Information Processing Systems , vol. 33, pp. 6111–6122, 2020.
P. Holl, V. Koltun, and N. Thuerey, “Learning to control pdes with differentiable physics,” arXiv preprint arXiv:2001.07457 , 2020.
B. Ummenhofer, L. Prantl, N. Thuerey, and V. Koltun, “Lagrangian fluid simulation with continuous convolutions,” in International Con- ference on Learning Representations , 2020.
M. Geilinger, D. Hahn, J. Zehnder, M. Bächer, B. Thomaszewski, and S.
Coros, Add: Analytically differentiable dynamics for multi-body systems with frictional contact , 2020. arXiv: 2007.00987 [cs.GR] .
S. L. Brunton, B. R. Noack, and P. Koumoutsakos, “Machine learning for fluid mechanics,” Annual review of fluid mechanics , vol. 52, pp. 477– 508, 2020.
T. Dorigo, A. Giammanco, P. Vischia, et al. , Toward the end-to-end op- timization of particle physics instruments with differentiable program- ming: A white paper , 2022. [Online]. A vailable: https://arxiv.org/ abs/2203.13818.
Differentiable Rendering
A. Tewari, J. Thies, B. Mildenhall, et al. , Advances in neural rendering , 2022. [Online]. A vailable: https://www.neuralrender.com/.
S. Zhao, W. Jakob, and T.-M. Li, “Physics-based differentiable ren- dering: From theory to implementation,” in ACM SIGGRAPH 2020
Courses, ser. SIGGRAPH ’20, Virtual Event, USA: Association for Com-
Papers with Code - Neural Rendering . [Online]. A vailable: https : / / paperswithcode.com/task/neural-rendering.
Awesome Neural Rendering . [Online]. A vailable: https://github.com/ weihaox/awesome-neural-rendering.
T.-M. Li, M. Aittala, F. Durand, and J. Lehtinen, “Differentiable monte carlo ray tracing through edge sampling,” ACM Transactions on Graph- ics (TOG) , vol. 37, no. 6, pp. 1–11, 2018.
S. Bangaru, T.-M. Li, and F. Durand, “Unbiased warped-area sampling for differentiable rendering,” ACM Trans. Graph., vol. 39, no. 6, 245:1– 245:18, 2020.
T. Zeltner, S. Speierer, I. Georgiev, and W. Jakob, “Monte carlo estima- tors for differential light transport,” Transactions on Graphics (Proceed- ings of SIGGRAPH) , vol. 40, no. 4, Aug. 2021. doi: 10.1145/3450626. 3459807.
D. Vicini, S. Speierer, and W. Jakob, “Path replay backpropagation: Dif- ferentiating light paths using constant memory and linear time,” Trans- actions on Graphics (Proceedings of SIGGRAPH) , vol. 40, no. 4, 108:1– 108:14, Aug. 2021. doi: 10.1145/3450626.3459804.
C. Zhang, Z. Dong, M. Doggett, and S. Zhao, “Antithetic sampling for monte carlo differentiable rendering,” ACM Trans. Graph., vol. 40, no. 4, 77:1–77:12, 2021.
S. Laine, J. Hellsten, T. Karras, Y. Seol, J. Lehtinen, and T. Aila, Modu- lar primitives for high-performance differentiable rendering , 2020. arXiv: 2011.03277 [cs.GR] .
Differential Geometry processing
D. Smirnov and J. Solomon, “HodgeNet: Learning spectral geometry on triangle meshes,” ACM Transactions on Graphics (TOG) , vol. 40, no. 4, 166:1–166:11, 2021.
Y. Wang, V. Kim, M. Bronstein, and J. Solomon, “Learning geomet- ric operators on meshes,” in Representation Learning on Graphs and Manifolds 2019 (ICLR workshop) , 2019.
R. Hanocka, A. Hertz, N. Fish, R. Giryes, S. Fleishman, and D. Cohen-
Or, “Meshcnn: A network with an edge,” ACM Transactions on Graphics
(TOG), vol. 38, no. 4, pp. 1–12, 2019.
J. Huang, H. Zhang, L. Yi, T. Funkhouser, M. Nießner, and L. Guibas,
Texturenet: Consistent local parametrizations for learning from high-
resolution signals on meshes , 2019. arXiv: 1812.00020 [cs.CV] .
J. Schult, F. Engelmann, T. Kontogianni, and B. Leibe, Dualconvmesh- net: Joint geodesic and euclidean convolutions on 3d meshes , 2020. arXiv: 2004.01002 [cs.CV] .
A. Deva Prasad, A. Balu, H. Shah, S. Sarkar, C. Hegde, and A. Krishna- murthy, “Nurbs-diff: A differentiable programming module for nurbs,”