Understanding my Reloads....

Yesterday I went to the range with my Bula XM21 and shot my second batch of reloads. Since these actually grouped much better than my first batch, I now have to understand what is going on with my loads because it does not make much sense to me. 40.5gr and 40.7gr grouped fairly well except the fliers and starts to open at 41 gr and 41.3 is like a shotgun, but then 41.5 grain grouped fairly well too. Its a bit confusing because I thought since 41.3gr did not group well than 41.5 gr would group like crap as well.. Which gr load should I stick with? I`m just trying to find what load my m1a likes the best. Thanks !

Midwayusa Blem 168gr Bullet
2015 Lake City Brass
CCI #34
Headspaced to 1.628
Brass Trim = 2.005
OAL = 2.82in
IMR 4895 from left to right in the picture (40.5gr,40.7gr,41gr,41.3gr,41.5)