Perhaps there is a problem where people are splitting a perfectly good monolith into microservices, but I do wonder how do you deal with large scale machine learning without microservices? I am a rather small operation and I still have models taking several gigabytes worth of memory and ANN indexes of about the same size, which clearly couldn't operate in a monolith unless it was a massive machine, and even if it could not every request would necessitate such power. How does a dogmatic monolith approach solve these problems?
Is there any problem with separating the ML into a different service/machine, and everything else together? Then you can treat the ML in the same way you treat an external service, or your DB or Redis (if external). While no longer a pure monolith that certainly doesn't qualify as a microservices architecture.
Note: no idea what I'm talking about, I'm genuinely curious if that's a valid solution.
You could do that, but then I wonder if that isn't going down the road of microservices? The ANN services for example would need to interface to the database if you want some kind of real time ANN service.
