Scientific machine learning (SciML) combines scientific computing and machine learning to solve complex physical problems. In this talk, we discuss neural network and neural operator approaches for differential equations, trained using data and/or physics-based loss functions. The focus of this talk is on two aspects. First, localization techniques based on domain decomposition are used to improve scalability, reduce computational costs, and help the models capture challenging spatio-temporal scales. Second, we discuss the training of these models, which constitutes the major computational cost. We investigate the training dynamics and how training may be improved. Numerical experiments on representative academic model problems (including multiscale problems) illustrate the performance of the discussed approaches.