A Physics-Informed Neural Network (PINN) is a Deep Learning (DL) framework for approximating solutions to Partial Differential Equations (PDE). Though PINNs benefit from the power and flexibility of DL, they lack the robust approximation theory of traditional numerical methods and, in the absence of real-world data, traditional methods outperform PINNs when solving PDE except in very high dimensions. However, the network architecture of PINNs is amenable to PDE for which traditional methods cannot be employed. This study highlights ways that the traditional and DL methods can be hybridized to exploit advantages inherent to both frameworks. We focus on the class of neural nets trained to satisfy Initial Boundary Value Problems (IBVP) (starting with the PINN framework) and investigate an approximation theory that is emerging in response to this class of physics-informed networks. We also present three areas of current research utilizing DL methods: First, using implicitly-computed antiderivatives, a mesh-free integration technique is presented on a 2D square domain as a starting point for approximating integrals in higher dimensions. Then, by investigating both continuous and discrete time formulations we may develop an efficient network for accelerating iterative methods. Lastly, the DeepONet solution for parametric PDE could allow us to create efficient and persistent physical systems which can generate data in a coupled system via a network forward pass.