Back in 2015 interest in Torch was peaking. Around that time, I took a look under the hood to try and map out how the framework was put together.
Torch is built on top of a stand-alone tensor library written in C called TH with a corresponding port to CUDA.
TH capabilities include serialization.
From what I can tell, PyTorch is derived from the same C library. The Lua front end is getting replaced with Python.
A corresponding setup for Mac is not hard to imagine. A front end tensor library built in Swift, attached to a Metal back end, integrated into an Xcode playground, storing native .mlmodel files. A lot of technical foundation is already in place for an interesting training environment.