Tags: Rohan-flutterint/fluvio
Tags
feat: update changelog for `0.10.4` (infinyon#2958) Update CHANGELOG to cover all important changes (fixes and features) from all pre-release versions on `0.10.4`.
Update segment.rs (infinyon#2950) Currently when using `Isolation::ReadCommitted`, we read records until current HW. https://github.com/infinyon/fluvio/blob/934544dec2403cd65113eda5c2a6fb6b509793d4/crates/fluvio-storage/src/replica.rs#L106 Here, there are two scenarios. If Start offset is greater or equal than base offset of current active segment. Then read_records is call in that segment. If HW is greater than last offset in that segment it will fail. (I think that this scenario should not happen since current active segment should be ok) https://github.com/infinyon/fluvio/blob/934544dec2403cd65113eda5c2a6fb6b509793d4/crates/fluvio-storage/src/replica.rs#L348 On the other hand, if the start offset is in a old segment, it calls find_slice, that first checks in which segment is the start offset and then call `read_records` there. This read_records will fail if that HW offset is not in the current segment. https://github.com/infinyon/fluvio/blob/934544dec2403cd65113eda5c2a6fb6b509793d4/crates/fluvio-storage/src/replica.rs#L361 This PR updates read_records to truncate max_offset to end_offset of current segment if it is larger than that so we can read records of that segment without failures Co-authored-by: Luis Moreno <[email protected]>
refactor: migrate error to anyhow (infinyon#2916) Migrate to use `anyhow` for Result. This removes the error wrapping business, leading to more straightforward error handling by making error flat. Move the previous `eyer` error message to a specific error display conversion to make it compatible with existing code and tests. We will keep the existing error structure even though some fields are unnecessary. They will be removed in future versions once the rest of the ecosystem is migrated.
fix: recover from invalid segments (infinyon#2909) resolves infinyon#2509. Recover from invalid logs due to incorrect batch encoding. It does this by setting the log size to the last correct batches. Tested with data from corrupted segments. - More clean and simplification of the Log validator. - Migrated more API to use anyhow
feat: added sink connector support to SDK (infinyon#2906) Added initial support for `#[connector(sink)]` to Connector SDK and an example of usage. Fixes infinyon#2884
Release 0.10.3 (infinyon#2880) Co-authored-by: Luis Moreno <[email protected]>
chore: emphasize flush in docs (infinyon#2850) Co-authored-by: Dave Beesley <[email protected]>
PreviousNext