Skip to content

SJTU-IPADS/copier

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

copier-artifact

This is the artifact of Copier --- a system service that provides asynchronous copying to optimize memory copying across the entire system.

We have open-sourced the source code required to reproduce all the data in the paper. We are currently working in progress to organize the tool chain and integrate the kernel modifications into an easy-to-install kernel module.

Getting started

Artifact structure:

artifact
    |____ hello-world (a demo of async copy)
    |____ libs ___ openssl (benchmarks for openSSL, Figure. 13(a))
    |        |____ protobuf (benchmarks for protobuf, Figure. 13(b))
    |____ micro __ syscall (micro benchmarks for send and recv syscalls, Figure. 10)
    |        |____ tp (micro benchmarks for copying throughput, Figure. 9)
    |____ redis (benchmarks for redis SET and GET, Figure. 11 and Figure. 14)
    |____ tinyproxy (benchmarks for TinyProxy, Figure. 12)
    |____ zIO (zIO source code, download from zIO artifact (https://github.com/tstamler/zIO))

Hardware platform:

CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
DRAM: 2400MHz, 128GB
Network: loopback

Software settings:

Ubuntu: Ubuntu 22.04.4 LTS
Linux: 5.15.131
GCC: 11.4.0
Redis: 6.2
Tinyproxy: c4df45b7
OpenSSL: 81f438d4
Protobuf: 23.4

We provide benchmark scripts (xx.sh) for all benchmarks. The data files (xx.txt) generated by these scripts are located in the results directory in the scripts' directories. When the scripts run, they detect the progress of the last run and continue from there. Therefore, if you need to rerun a script, you need to delete or clear the corresponding data file (xx.txt). You can also delete the last few lines of the data file to rerun the benchmarks corresponding to those lines.

When running the benchmarks, we will specify the Linux kernel you need to use (by specifying a branch). You need to compile and install the corresponding kernel. You can use the following commands. On the AE machines we provide, you don't need to select the kernel through GRUB because the machine will default use the newly installed kernel. Please clone the kernel source code from the kernel branches.

# in the directory of kernel
cp /path/to/articact/kernel.config .config # only needed for the first compile
make menuconfig # just save and exit, only needed for the first compile

git switch <breanch>
make -j24
sudo make modules_install -j24
sudo make install
sudo reboot

We provide machines for the AE committee. Please contact the authors.

To save the AE committee's time, we simplified the benchmarks and reduced their duration. Running all the benchmarks takes about 2--3 hours.

Hello world

We prepare a demo to help you verify your environment. Please use the kernel-recv02 kernel branch.

cd hello-world
make
./normal 65536 1024 # copy an array and find the largest number
./copier 65536 1024 # copy an array and find the largest number with copier's async copy

Redis

For SET benchmarks (Figure. 12), you need to use the kernel-recv01 kernel branch and run set-baseline.sh, set-copier.sh, and set-zio.sh.

For GET benchmarks (Figure. 12), you need to use the kernel-send kernel branch and run get-baseline.sh, get-copier.sh, get-zero-zopy.sh and get-zio.sh.

For multi-client benchmarks (Figure. 15), you need to use the kernel-public kernel branch and run multi-client-baseline.sh, multi-client-copier.sh.

TinyProxy

You need to use the kernel-absorb kernel branch, run tinyproxy.size.baseline.sh, tinyproxy.size.copier.sh, and tinyproxy.size.zIO.sh for Fig.-a, and run tinyproxy.thread.baseline.sh, tinyproxy.thread.copier.sh for Fig.-b.

Libs

For Protobuf (in protobuf directory), you need to use kernel-recv01 kernel branch and run *.sh.

For OpenSSL (in openssl directory), you need to use kernel-recv02 kernel branch and run *.sh (please run init.sh first to prepare OpenSSL).

Micro-benchmarks

For send syscall (in syscall directory), you need to use kernel-send kernel branch and run send.*.sh.

For recv syscall (in syscall directory), you need to use kernel-recv01 kernel branch and run recv.*.sh.

For copying throughput (in tp directory), you need to use kernel-tp kernel branch and run *.sh.

About

Copy as an OS Service

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published