Before giving my two cents on BaseballMan's question, I wanted to make sure to thank dfswans for their great contributions! The pre-NL parks are extra hard, so I'm extra impressed.
As far as the AI/photorealistic backgrounds--those (great!) images BaseballMan posted for the South End Grounds and St. George Grounds could already be input as-is for 2D play, as LandsdowneSt suggests.
(Long exposition follows)
The thing with doing it for 3D is that (at least in my models) some elements are 3D objects while the background is a 2D texture. I'm sure basically everybody already knows this, but for any newbies reading and to make sure we're on the same page, I put in a screenshot of Detroit's Recreation Park below. The park, buildings, and trees were done in Sketchup (I made the park and made or downloaded the trees/buildings from 3D Warehouse). The background pictures on the rectangles that line the circle's circumference generally come from Google Earth in my models.
Different folks handle the background differently, and while I haven't done any experimenting, I suspect there's a tradeoff between the camera angles you use and the size of the ballpark vs. how much of the background you even care about. In these smaller parks, where I'm using the default minor league camera angles, there's a lot that's visible outside the ballpark itself, which has driven me to including some of the nearby streets (which then usually means putting things like trees and houses on those streets) and a 360-degree background, with a scene that usually stretches 500-600 feet from home plate. Games and camera angles set in physically bigger stadiums just show less of the outside world, so people do great jobs with models that show less (or nothing) outside the stadium itself.
The 2D samples above look great, and I agree there's some real potential. I think some factors for making 3D backgrounds this way are going to be:
--The camera angles you want to show will dictate how much of the background you want
--I expect anything should be made in color, since folks who want to play 3D in black and white can do so via the in-game filters and some folks will want them in color.
--Obviously, the parks themselves need to not be in the prospective 3D background.
--The existing backgrounds live in the texture folders, so in principle folks could work from those and replace them with AI-ed versions of the same size using the same tricks that some folks do with replacing logos, ads, etc. if they felt like experimenting
--I think good-looking standing fans in the outfield is a whole other topic, though one I feel could be solvable by someone clever. I've wondered if the same kind of thing that's done to make the animated crowd could be used to create an animation for a line of standing fans and/or a line of fans sitting on the ground, which we could then use the way we use the crowd texture?
That's it on this for now. I'm happy to update or experiment with models as need be or desired with backgrounds that folks give that they think will work, but I don't have the bandwidth to do any of the hard work on it.
I do have a new park to post a little later today, plus I was going to summarize where I think we are with this.