What is Differentiable Programming?

Differentiable programming refers to programs that rewrite themselves within at least one component, like neural networks do, using optimization algorithms like gradient descent.

Examples of Differentiable Programming

Some applications of differentiable programming are combining deep learning with physics engines in robotics, analyzing electronic structures with differentiable density functional theory, and differentiable ray tracing.
 

Why is Differentiable Programming important?

Differentiable programming allows companies to build a new program using weighted parameters, trained from examples using gradient-based optimization. These programs can rewrite parts of themselves in a more optimized manner using gradients.

Differentiable Programming vs. Other Technologies & Methodologies

Differential programming vs. probabilistic programming

Differentiable programming uses automatic differentiation and gradient-based optimization to approximate a loss function, such as neural networks. In contrast, the probabilistic programming approach uses various forms of Markov Chain Monte Carlo (MCMC) and differentiable inference to approximate a probability density function.

Differential programming vs. deep learning

Differentiable programming consists of applying the techniques of deep learning to complex to reuse the knowledge embedded in them.