I will address only your conductor size...your arithmetic doesn't seem right for the sizing of your conductors.
I use Mike Holt's formulas for sizing conductors (to adjust for voltage drop over long distances [www.mikeholt.com]).
The standard formula for sizing a conductor to compensate for distance and stay within the generally accepted 3% voltage drop limitation is:
CM = 2 x K x I x D/volts dropped
CM = circular mils (area of a conductor in circular mils taken from NEC Chap 9, Table 8).
2 (2 conductors)
K = the constant K value is 12.9 ohms for copper
I = amperes (50 in your case)
D = distance
volts dropped = 3% x 240 = 7.2 volts
2x12.9x50x500/7.2 = 89,583 circular mils...the smallest conductor that size is 1/0!
I've done this using 3 different formulas and all the answers indicate 1/0 conductor size to keep the voltage drop within limits.
IF my arithmetic is correct, then your #4 ground (equipment ground) is the minimum size also (250-122(b) requires the equipment ground to be proportionately oversized ..."where conductors are adjusted in size to compensate for voltage drop, equipment grounding conductors, where installed, shall be adjusted proportionately according to circular mil area)....I won't get into the arithmetic for that but my calculations indicate #4 is needed.
...I suspect your neutral is way too small...won't get into that formula.