Large eddy simulations of wind farms are performed to study the effects of wind turbine row alignment with respect to the incoming flow direction. Various wind farms with fixed stream-wise spacing (7.85 rotor diameters) and varying lateral displacements and span-wise turbine spacings are considered, for a fixed inflow direction. Simulations show that, contrary to common belief, a perfectly staggered (checker-board) configuration does not necessarily give the highest average power output. Instead, the highest mean wind farm power output is found to depend on several factors, the most important one being the alignment that leads to minimization of wake effects from turbines in several upstream rows. This alignment typically occurs at significantly smaller angles than those corresponding to perfect staggering. The observed trends have implications for wind farm designs, especially in sites with a well-defined prevailing wind direction.