Fansub encoders usually use a constant rate factor (constant quality) and not a constant bitrate. This causes file sizes to vary for the reasons explained by Leggy.
Streaming services (CrunchyRoll, Hulu, Daisuki, etc. — files ripped by HorribleSubs) on the other hand use a constant bitrate, leading to very uniform file sizes.
You could always use Hiryuu's version until FFF's is released. I did have some weird subtitle issues in it though (the subs were inconsistent until after the opening).