Edit: turn text size up, they might not be visible with longer names
they are as often wrong as right.. so definitely not something you should read as terribly accurate. possibly with incredibly inaccurate scouting they may be a better indicator? leaving that stuff open-ended
but, used along with other feedback, it can make a tough decision lean one way or another... the proverbial "straw" ... not important on it's own, but may contribute to a particular decision.
probably more useful with ratings on... and i don't agree that they are only ratings based... they are the view of the mil managers according to the manual, if i recall...
i've seen both: low ratings with an up arrow and good stats with a down arrow. i'd wager it's the mil manager's player evaluation sliders in the editor which affect the arrows.
typically my scout is more often correct in both situations - mileage may vary. i usually wait for my scout to correct himself on prospects (18->21ish or recently drafted is less accurate info), and the way i handle 'filler' mil players causes alot of simmering if they stay under the radar ratings-wise... but the arrows will help at some point even if my scout cannot.
e.g. maybe slower than normal, but continue to be promoted and succeed, despite ratings.
i cut players after they prove they cannot play at the next level, typically... i use stats alot more for decisions on these players... if low rated and terrible stats at previous level, i will cut those players without testing most of the time. if they are performing well, i let it ride in spite of ratings. (relative to service time limits at each level -- filler are there to provide wins, and not caring whatsoever about their development -- works very well, btw)