QEMU soft-TLB improvements #150
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Benchmarks for booting MFS_ROOT kernels: RISC-V purecap kernel: RISC-V hybrid kernel: |
arichardson
commented
Jun 17, 2021
arichardson
commented
Jun 17, 2021
nwf
requested changes
Jun 17, 2021
target/cheri-common/cheri_tagmem.c
Outdated
| * we let it "see around" the TLB capability load inhibit. That should | ||
| * perhaps change? | ||
| * The Mips CLoadTags, can "see around" the TLB capability load inhibit. | ||
| * That should perhaps change? |
There was a problem hiding this comment.
Is that true only of MIPS? I don't think there's harm in making CLoadTags respect the tag-clearing or tag-load-trapping behaviors. It's used very rarely, and at present exclusively by the kernel through the dmap region which never has those attributes set (that is, the dmap always permits cap loads and stores).
|
Not that it's indicative of perfect functionality by any means, but I rebased #117 atop this and it passes the basic |
|
Benchmarks for booting to multi-user and shutting down: hybrid kernel + purecap userspace |
arichardson
force-pushed
the
soft-tlb-tags-dev
branch
from
db85ac5 to
2c853b2
Compare
arichardson
commented
Jun 18, 2021
There was a problem hiding this comment.
I've pushed some scripts/checkpatch.pl fixes on top of the original commit.
I think this looks good. We need a reasonable commit message for "Always use soft-TLB for tag reads/writes" and the v2r comment needs to be updated. Once that has been done I think we can merge it.
jrtc27
reviewed
Jun 18, 2021
nwf
reviewed
Jun 18, 2021
arichardson
force-pushed
the
soft-tlb-tags-dev
branch
from
2c853b2 to
75a9723
Compare
nwf
approved these changes
Jun 21, 2021
jrtc27
reviewed
Jun 21, 2021
jrtc27
reviewed
Jun 21, 2021
jrtc27
reviewed
Jun 21, 2021
|
Addressed most of the comments. If somebody wants to clean up clang-format to not do silly things, I guess feel free. The only real issue is what whether we are happy for loadtags to do what I just changed it to do. Just to re-itterate: not only have I removed the ifdef for MIPS, I have also made it only be exceptional in the case the result of the load contains some tags, for the sake of consistency. |
|
|
This will be used by Morello. Also slightly refactor getting / setting multiple tags with atomicity in mind.
This significantly speeds up QEMU since a large fraction of the total
runtime is spent looking up the physical address for tag memory.
A purecap kernel+purecap userspace boot is almost twice as fast with
this change (310 instead of 610 seconds) and purecap userspace with
a hybrid kernel now takes 140 instead of 207 seconds.
purecap kernel + purecap userspace:
```
alr48@waltham:/local/scratch/alr48/cheri/cheribsd(dev u=)> hyperfine -L qemu /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.baseline,/local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.softtlb '/home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel.CHERI-PURECAP-QEMU/kernel --qemu-cmd {qemu} --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest' -m 2
Benchmark #1: /home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel.CHERI-PURECAP-QEMU/kernel --qemu-cmd /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.baseline --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest
Time (mean ± σ): 611.770 s ± 3.921 s [User: 597.733 s, System: 12.143 s]
Range (min … max): 608.997 s … 614.543 s 2 runs
Benchmark #2: /home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel.CHERI-PURECAP-QEMU/kernel --qemu-cmd /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.softtlb --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest
Time (mean ± σ): 311.638 s ± 0.213 s [User: 300.455 s, System: 2.044 s]
Range (min … max): 311.488 s … 311.789 s 2 runs
Summary
'/home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel.CHERI-PURECAP-QEMU/kernel --qemu-cmd /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.softtlb --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest' ran
1.96 ± 0.01 times faster than '/home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel.CHERI-PURECAP-QEMU/kernel --qemu-cmd /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.baseline --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest'
```
hybrid kernel + purecap userspace:
```
alr48@waltham:/local/scratch/alr48/cheri/cheribsd(dev u=)> hyperfine -L qemu /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.baseline,/local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.softtlb '/home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel/kernel --qemu-cmd {qemu} --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest' -m 2
Benchmark #1: /home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel/kernel --qemu-cmd /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.baseline --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest
Time (mean ± σ): 208.867 s ± 2.144 s [User: 194.562 s, System: 2.332 s]
Range (min … max): 207.351 s … 210.382 s 2 runs
Benchmark #2: /home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel/kernel --qemu-cmd /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.softtlb --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest
Time (mean ± σ): 141.067 s ± 0.650 s [User: 121.470 s, System: 7.372 s]
Range (min … max): 140.607 s … 141.526 s 2 runs
Summary
'/home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel/kernel --qemu-cmd /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.softtlb --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest' ran
1.48 ± 0.02 times faster than '/home/alr48/devel/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/alr48/.ssh/insecure_id_ed25519.pub --architecture riscv64-purecap --kernel /local/scratch/alr48/cheri/output/rootfs-riscv64-purecap/boot/kernel/kernel --qemu-cmd /local/scratch/alr48/cheri/output/sdk/bin/qemu-system-riscv64cheri.baseline --disk-image /local/scratch/alr48/cheri/output/cheribsd-riscv64-purecap.img --no-run-cheribsdtest'
```
TODO: write a more detailed commit message including performance numbers
arichardson
force-pushed
the
soft-tlb-tags-dev
branch
from
d213410 to
1e3ee06
Compare
jrtc27
reviewed
Jun 28, 2021
jrtc27
approved these changes
Jun 28, 2021
Drop the cpu_transaction_failed() call and only keep the error logging.
arichardson
force-pushed
the
soft-tlb-tags-dev
branch
from
1e3ee06 to
8484fab
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is just a rebase onto dev, I haven't reviewed the code yet.