[codec] support no_std codec builds #1477

roberto-bayardo · 2025-08-26T17:51:37Z

Makes codec crate buildable under no_std. A "std" feature flag is introduced for conditionally compiling codec trait implementations for std:: types such as HashMap and HashSet.

Confirmed the package now builds in a no_std architecture:

]: cargo check -p commonware-codec --no-default-features --target thumbv7em-none-eabihf

    Blocking waiting for file lock on build directory
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.45s

Copilot

Pull Request Overview

This PR adds support for building the codec crate in no_std environments by introducing conditional compilation features and separating std-dependent functionality from core functionality.

Introduces std feature flag (enabled by default) to control std vs no_std compilation
Replaces std:: imports with core:: equivalents where possible
Separates std-dependent collections (HashMap, HashSet) from no_std-compatible ones (BTreeMap, BTreeSet)

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
codec/Cargo.toml	Adds feature flags for std/no_std support
codec/src/lib.rs	Adds no_std attribute and alloc extern declaration
codec/src/varint.rs	Replaces std:: with core:: imports for basic functionality
codec/src/types/primitives.rs	Updates memory size calculations to use core::mem
codec/src/types/vec.rs	Adds conditional import for alloc::vec::Vec
codec/src/types/mod.rs	Conditionally includes std-dependent modules
codec/src/types/set.rs	Removes BTreeSet implementation (moved to set_nostd)
codec/src/types/set_nostd.rs	New file with BTreeSet implementation using alloc
codec/src/types/map.rs	Removes BTreeMap implementation (moved to map_nostd)
codec/src/types/map_nostd.rs	New file with BTreeMap implementation using alloc
codec/src/error.rs	Updates error trait bounds to use core::error
codec/src/config.rs	Replaces std::ops with core::ops imports

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

patrick-ogrady · 2025-08-26T20:20:30Z

It would be great to add a no_std CI compilation check for crates/modes that support it:

monorepo/.github/workflows/fast.yml

Lines 118 to 139 in ce5341d

    
             WASM: 
        
               runs-on: ubuntu-latest 
        
               timeout-minutes: 10 
        
               steps: 
        
               - name: Checkout repository 
        
                 uses: actions/checkout@v4 
        
               - name: Run setup 
        
                 uses: ./.github/actions/setup 
        
               - name: Add WASM target 
        
                 run: rustup target add wasm32-unknown-unknown 
        
               - name: Build cryptography 
        
                 run: cargo build --target wasm32-unknown-unknown --release --manifest-path cryptography/Cargo.toml && du -h target/wasm32-unknown-unknown/release/commonware_cryptography.wasm 
        
               - name: Build macros 
        
                 run: cargo build --target wasm32-unknown-unknown --release --manifest-path macros/Cargo.toml # can't check size because it is a proc-macro 
        
               - name: Build utils 
        
                 run: cargo build --target wasm32-unknown-unknown --release --manifest-path utils/Cargo.toml && du -h target/wasm32-unknown-unknown/release/commonware_utils.wasm 
        
               - name: Build runtime 
        
                 run: cargo build --target wasm32-unknown-unknown --release --manifest-path runtime/Cargo.toml && du -h target/wasm32-unknown-unknown/release/commonware_runtime.wasm 
        
               - name: Build consensus 
        
                 run: cargo build --target wasm32-unknown-unknown --release --manifest-path consensus/Cargo.toml && du -h target/wasm32-unknown-unknown/release/commonware_consensus.wasm 
        
               - name: Build storage 
        
                 run: cargo build --target wasm32-unknown-unknown --release --manifest-path storage/Cargo.toml && du -h target/wasm32-unknown-unknown/release/commonware_storage.wasm

patrick-ogrady · 2025-08-26T20:21:25Z

codec/src/lib.rs

 //!   [f32], [f64], [u8; N],
 //!   and [usize] (must fit within a [u32] for cross-platform compatibility).
-//! - Collections: [`Vec<T>`], [`Option<T>`]
+//! - Collections: [`Vec`], [`Option`], `BTreeMap`, `BTreeSet`


nit: wrap in []

Unfortunately this doesn't work, since it's not a default visible type. If you fully specify alloc::collections::BTreeMap instead, then this fails because cargo doc runs in std mode and doesn't have alloc visibility.

patrick-ogrady · 2025-08-26T20:21:40Z

codec/src/types/btreemap.rs

+    error::Error,
+    RangeCfg,
+};
+use alloc::collections::BTreeMap;


til: BTreeMap works for this 👀

Yeah the only reason Hash* collections don't work is because of the use of a non-deterministic hash function to avoid DoS.

roberto-bayardo · 2025-08-26T20:34:12Z

It would be great to add a no_std CI compilation check for crates/modes that support it:

Added!

BrendanChou · 2025-08-26T20:48:45Z

codec/src/varint.rs

-        const _COMMIT_OP_ASSERT: () =
-            assert!(std::mem::size_of::<Self>() == std::mem::size_of::<Self::UnsignedEquivalent>());
+        const _COMMIT_OP_ASSERT: () = assert!(
+            core::mem::size_of::<Self>() == core::mem::size_of::<Self::UnsignedEquivalent>()


nit: my fault, but might be nice to just use core::mem::size_of instead of qualifying it everywhere

Fine to not do this if just want to merge when things pass

patrick-ogrady · 2025-08-26T20:58:33Z

.github/workflows/fast.yml

+    - name: Run setup
+      uses: ./.github/actions/setup
+    - name: Add minimal no_std target
+      run: rustup target add thumbv7em-none-eabihf


confirmed that this target is commonly used for this 👍

codec/src/types/hashmap.rs

patrick-ogrady · 2025-08-26T21:27:50Z

Put up a small PR to add testing for no_std builds as well: #1478

codecov · 2025-08-26T22:50:06Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.69%. Comparing base (ce5341d) to head (31cb85b).
⚠️ Report is 1 commits behind head on main.

@@            Coverage Diff             @@
##             main    #1477      +/-   ##
==========================================
- Coverage   91.71%   91.69%   -0.02%     
==========================================
  Files         278      280       +2     
  Lines       70501    70361     -140     
==========================================
- Hits        64659    64519     -140     
  Misses       5842     5842

Files with missing lines	Coverage Δ
codec/src/config.rs	`100.00% <ø> (ø)`
codec/src/types/btree_map.rs	`100.00% <100.00%> (ø)`
codec/src/types/btree_set.rs	`100.00% <100.00%> (ø)`
codec/src/types/bytes.rs	`100.00% <ø> (ø)`
codec/src/types/hash_map.rs	`100.00% <100.00%> (ø)`
codec/src/types/hash_set.rs	`100.00% <ø> (ø)`
codec/src/types/primitives.rs	`99.52% <100.00%> (ø)`
codec/src/types/vec.rs	`100.00% <100.00%> (ø)`
codec/src/varint.rs	`96.59% <100.00%> (ø)`

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce5341d...31cb85b. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

make codec work in no_std builds

6161bf7

roberto-bayardo force-pushed the nostd-digest branch from 537353e to 12b2db8 Compare August 26, 2025 18:13

roberto-bayardo marked this pull request as ready for review August 26, 2025 18:14

roberto-bayardo requested a review from Copilot August 26, 2025 18:14

Copilot AI reviewed Aug 26, 2025

View reviewed changes

roberto-bayardo force-pushed the nostd-digest branch from 12b2db8 to 020929c Compare August 26, 2025 18:19

roberto-bayardo requested a review from BrendanChou August 26, 2025 18:20

allow btree classes to work with no_std

8874071

roberto-bayardo force-pushed the nostd-digest branch from 020929c to 8874071 Compare August 26, 2025 18:26

better naming

acd64c5

roberto-bayardo force-pushed the nostd-digest branch from c66cd67 to acd64c5 Compare August 26, 2025 18:32

disable default-features for thiserror and bytes

2b0778c

roberto-bayardo force-pushed the nostd-digest branch from 7da86c9 to 2b0778c Compare August 26, 2025 20:15

patrick-ogrady reviewed Aug 26, 2025

View reviewed changes

patrick-ogrady added this to the v0.0.60 milestone Aug 26, 2025

add CI rule to verify no_std builds

4c3709a

roberto-bayardo force-pushed the nostd-digest branch from eed226a to 4c3709a Compare August 26, 2025 20:37

BrendanChou previously approved these changes Aug 26, 2025

View reviewed changes

review nits

4c51b87

roberto-bayardo dismissed BrendanChou’s stale review via 4c51b87 August 26, 2025 20:53

BrendanChou previously approved these changes Aug 26, 2025

View reviewed changes

roberto-bayardo requested a review from patrick-ogrady August 26, 2025 20:56

merge use core directives

8913849

roberto-bayardo dismissed BrendanChou’s stale review via 8913849 August 26, 2025 20:56

patrick-ogrady reviewed Aug 26, 2025

View reviewed changes

codec/src/types/hashmap.rs Show resolved Hide resolved

better filenames

eb55f27

roberto-bayardo requested a review from patrick-ogrady August 26, 2025 21:12

[codec] Improve no_std Testing (#1478)

31cb85b

patrick-ogrady approved these changes Aug 26, 2025

View reviewed changes

patrick-ogrady merged commit e3fffd3 into main Aug 26, 2025
37 checks passed

patrick-ogrady deleted the nostd-digest branch August 26, 2025 21:40

[codec] support no_std codec builds #1477

[codec] support no_std codec builds #1477

Uh oh!

Conversation

roberto-bayardo commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

patrick-ogrady commented Aug 26, 2025

Uh oh!

patrick-ogrady Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

roberto-bayardo Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

patrick-ogrady Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

roberto-bayardo Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

roberto-bayardo commented Aug 26, 2025

Uh oh!

BrendanChou Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

BrendanChou Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

roberto-bayardo Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

patrick-ogrady Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

patrick-ogrady commented Aug 26, 2025

Uh oh!

Uh oh!

codecov bot commented Aug 26, 2025

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

roberto-bayardo commented Aug 26, 2025 •

edited

Loading

roberto-bayardo Aug 26, 2025 •

edited

Loading