Flame tests are too brittle

[charleskawczynski]

It seems that very minor changes can result in our flame allocation tests to break. I suspect that this is due to inference failures, and we should probably fix this. A quick patch for now, to at least alleviate some of the headache is to use a buffer, like we do for the threaded tests (which are inherently flakey).

Thoughts @Sbozzolo?

[charleskawczynski]

I think they're even flakey, which is worse. I updated the allocation limits in #2673, and re-running still resulted in failure. This is another indicator that the issue is related to inference.

[charleskawczynski]

#2673 is going to change buffer = occursin("threaded", job_id) ? 1.4 : 1 to buffer = occursin("threaded", job_id) ? 1.4 : 1.1 so that we have a bit of a buffer.

[charleskawczynski]

But we should revert this and fix the issue.

[Sbozzolo]

I had precisely the same thought and I would have proposed the same solution.