After examining my model and code closer, albeit as rudimentary as it is, I'm now pretty certain I didnt fuck up the encoder, which means the system is in fact learning the training sequence pairs, which means it SHOULD be able to actually factor semiprimes.

And looking at this, it means I fucked up the decoder and I have a good idea how. So the next step is to fix that.

I'm also fairly certain, using a random forest approach, namely multiple models trained on other variables derived from the semiprimes, I can extend the method, and lower loss to the point where I can factor 400 digit numbers instead of 20 digit numbers.