Skip to content

bug: flusher & reclaimer needs graceful shutdown #44

@MrCroxx

Description

@MrCroxx

When foyer-storage-bench finished, the following panic log is printed because flusher is dropped but inner read slice is not properly destroyed.

thread 'thread 'tokio-runtime-workertokio-runtime-worker' panicked at '' panicked at 'future is not consumed: Slice { slice: Slice { ptr: 0x7fdcbbff5000, len: 67108864 }, allocator: Some(AlignedAllocator { align: 4096 }) }future is not consumed: Slice { slice: Slice { ptr: 0x7fdcb7ff3000, len: 67108864 }, allocator: Some(AlignedAllocator { align: 4096 }) }', ', /home/mrcroxx/github/mrcroxx/foyer/foyer-storage/src/region.rs/home/mrcroxx/github/mrcroxx/foyer/foyer-storage/src/region.rs::511511::1313

stack backtrace:
thread 'thread 'tokio-runtime-workertokio-runtime-worker' panicked at '' panicked at 'future is not consumed: Slice { slice: Slice { ptr: 0x7fdcc3ff9000, len: 67108864 }, allocator: Some(AlignedAllocator { align: 4096 }) }future is not consumed: Slice { slice: Slice { ptr: 0x7fdcbfff7000, len: 67108864 }, allocator: Some(AlignedAllocator { align: 4096 }) }', ', /home/mrcroxx/github/mrcroxx/foyer/foyer-storage/src/region.rs/home/mrcroxx/github/mrcroxx/foyer/foyer-storage/src/region.rs::511511::1313

   0: rust_begin_unwind
             at /rustc/28a29282f6dde2e4aba6e1e4cfea5c9430a00217/library/std/src/panicking.rs:577:5
   1: core::panicking::panic_fmt
             at /rustc/28a29282f6dde2e4aba6e1e4cfea5c9430a00217/library/core/src/panicking.rs:67:14
   2: core::ptr::drop_in_place<foyer_storage::region::ReadSlice<foyer_storage::device::io_buffer::AlignedAllocator>>
   3: core::ptr::drop_in_place<foyer_storage::flusher::Runner<foyer_storage::device::io_buffer::AlignedAllocator,foyer_storage::device::fs::FsDevice,foyer_intrusive::eviction::lfu::Lfu<foyer_storage::region_manager::RegionEpItemAdapter<foyer_intrusive::eviction::lfu::LfuLink>>,foyer_intrusive::eviction::lfu::LfuLink>::run::{{closure}}>
   4: std::panicking::try
   5: tokio::runtime::task::harness::Harness<T,S>::shutdown
   6: tokio::runtime::task::list::OwnedTasks<S>::close_and_shutdown_all
   7: tokio::runtime::scheduler::multi_thread::worker::Context::run
   8: std::thread::local::LocalKey<T>::with
   9: tokio::runtime::context::runtime::enter_runtime
  10: tokio::runtime::scheduler::multi_thread::worker::run
  11: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
  12: tokio::runtime::task::core::Core<T,S>::poll
  13: tokio::runtime::task::raw::poll
  14: tokio::runtime::blocking::pool::Inner::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: rust_begin_unwind
             at /rustc/28a29282f6dde2e4aba6e1e4cfea5c9430a00217/library/std/src/panicking.rs:577:5
   1: core::panicking::panic_fmt
             at /rustc/28a29282f6dde2e4aba6e1e4cfea5c9430a00217/library/core/src/panicking.rs:67:14
   2: core::ptr::drop_in_place<foyer_storage::region::ReadSlice<foyer_storage::device::io_buffer::AlignedAllocator>>
   3: core::ptr::drop_in_place<foyer_storage::flusher::Runner<foyer_storage::device::io_buffer::AlignedAllocator,foyer_storage::device::fs::FsDevice,foyer_intrusive::eviction::lfu::Lfu<foyer_storage::region_manager::RegionEpItemAdapter<foyer_intrusive::eviction::lfu::LfuLink>>,foyer_intrusive::eviction::lfu::LfuLink>::run::{{closure}}>
   4: std::panicking::try
   5: tokio::runtime::task::harness::Harness<T,S>::shutdown
   6: tokio::runtime::task::list::OwnedTasks<S>::close_and_shutdown_all
   7: tokio::runtime::scheduler::multi_thread::worker::Context::run
   8: std::thread::local::LocalKey<T>::with
   9: tokio::runtime::context::runtime::enter_runtime
  10: tokio::runtime::scheduler::multi_thread::worker::run
  11: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
  12: tokio::runtime::task::core::Core<T,S>::poll
  13: tokio::runtime::task::raw::poll
  14: tokio::runtime::blocking::pool::Inner::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: rust_begin_unwind
             at /rustc/28a29282f6dde2e4aba6e1e4cfea5c9430a00217/library/std/src/panicking.rs:577:5
   1: core::panicking::panic_fmt
             at /rustc/28a29282f6dde2e4aba6e1e4cfea5c9430a00217/library/core/src/panicking.rs:67:14
   2: core::ptr::drop_in_place<foyer_storage::region::ReadSlice<foyer_storage::device::io_buffer::AlignedAllocator>>
   3: core::ptr::drop_in_place<foyer_storage::flusher::Runner<foyer_storage::device::io_buffer::AlignedAllocator,foyer_storage::device::fs::FsDevice,foyer_intrusive::eviction::lfu::Lfu<foyer_storage::region_manager::RegionEpItemAdapter<foyer_intrusive::eviction::lfu::LfuLink>>,foyer_intrusive::eviction::lfu::LfuLink>::run::{{closure}}>
   4: std::panicking::try
   5: tokio::runtime::task::harness::Harness<T,S>::shutdown
   6: tokio::runtime::task::list::OwnedTasks<S>::close_and_shutdown_all
   7: tokio::runtime::scheduler::multi_thread::worker::Context::run
   8: std::thread::local::LocalKey<T>::with
   9: tokio::runtime::context::runtime::enter_runtime
  10: tokio::runtime::scheduler::multi_thread::worker::run
  11: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
  12: tokio::runtime::task::core::Core<T,S>::poll
  13: tokio::runtime::task::raw::poll
  14: tokio::runtime::blocking::pool::Inner::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: rust_begin_unwind
             at /rustc/28a29282f6dde2e4aba6e1e4cfea5c9430a00217/library/std/src/panicking.rs:577:5
   1: core::panicking::panic_fmt
             at /rustc/28a29282f6dde2e4aba6e1e4cfea5c9430a00217/library/core/src/panicking.rs:67:14
   2: core::ptr::drop_in_place<foyer_storage::region::ReadSlice<foyer_storage::device::io_buffer::AlignedAllocator>>
   3: core::ptr::drop_in_place<foyer_storage::flusher::Runner<foyer_storage::device::io_buffer::AlignedAllocator,foyer_storage::device::fs::FsDevice,foyer_intrusive::eviction::lfu::Lfu<foyer_storage::region_manager::RegionEpItemAdapter<foyer_intrusive::eviction::lfu::LfuLink>>,foyer_intrusive::eviction::lfu::LfuLink>::run::{{closure}}>
   4: std::panicking::try
   5: tokio::runtime::task::harness::Harness<T,S>::shutdown
   6: tokio::runtime::task::list::OwnedTasks<S>::close_and_shutdown_all
   7: tokio::runtime::scheduler::multi_thread::worker::Context::run
   8: std::thread::local::LocalKey<T>::with
   9: tokio::runtime::context::runtime::enter_runtime
  10: tokio::runtime::scheduler::multi_thread::worker::run
  11: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
  12: tokio::runtime::task::core::Core<T,S>::poll
  13: tokio::runtime::task::raw::poll
  14: tokio::runtime::blocking::pool::Inner::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions