Enable flash attn by default？

Created by: roywei

Hi MetaSeq team,

Is there any plan to enable flash attn by default, I found by enabling it the WPS from benchmarking almost doubled. I came across this PR https://github.com/facebookresearch/metaseq/pull/387 and tried it out, but I did not see any documentation about it. Before 1031763.71429 WPS After enabling XFormer + Triton 1943702.28571 WPS

happy to raise a PR if we want it enabled by default. Let me know if there is any concerns.

Thanks!