Foresight Summary

    
Major Impacts

build-and-test-windows duration(4 seconds) has decreased 40 minutes 23 seconds compared to main branch avg(40 minutes 27 seconds).

🔎 See details on Foresight

^{*You can configure Foresight comments in your organization settings page.}

Before we make a change to the component, I think we need to articulate more clearly why this change is necessary. I don't think it's enough to say that test assumptions were broken. What specifically is the effect of this change and what criteria would someone use to determine if they should use the setting?

Before we make a change to the component, I think we need to articulate more clearly why this change is necessary. I don't think it's enough to say that test assumptions were broken. What specifically is the effect of this change and what criteria would someone use to determine if they should use the setting?

@djaglowski In our case, we're using the file exporter as part of a test workflow. The tests expect that data will be exported quasi-synchronously by the file exporter, as was the previous behavior without the buffering. I would expect that other people using the file exporter for testing processes would also want to turn the buffering off. I can certainly add more documentation to the README to more clearly articulate when to use this parameter, if that helps.

Another solution would be to put a time limit on how long data is buffered. That would be a cleaner solution for this case, but I couldn't find a way to do that. Suggestions welcome for that. (A parameter would still be needed to allow users to adjust it.)

I can also accept that perhaps the testing use case isn't in scope for the file exporter. If it isn't in scope, I'm OK with that. We'll look for alternative testing methods in that case.

@loomis-relativity, thanks for explaining in more detail.

Generally speaking, I don't think we can guarantee a particular delivery latency. It is a meaningful metric but any given exporter must have the flexibility to balance this with other priorities such as overall throughput or reliability over a longer period of time. That said, it seems we have overlooked this consideration when adding the buffer. I just want to make sure we're not rushing into a configuration option that is only needed for tests and may need to be maintained indefinitely.

I'd appreciate hearing other perspectives on this before we make a decision. Maybe @MovieStoreGuy or @atoulme have a better idea. Personally, given the options mentioned, I am inclined to favor the flush interval over an entirely separate writer because this seems more broadly useful.

Workaround found with rotation. Change not needed.