Add a new check-parallel target that utilizes all available CPUs to run shellcheck in parallel on test files, significantly reducing validation time from ~40s to ~11s on a 48-core system. The original check target remains unchanged for compatibility. blktests (master) # vim 0001-Makefile-add-check-parallel-target-for-faster-shellc.patch blktests (master) # time make check shellcheck -x -e SC2119 -f gcc check common/* \ tests/*/rc tests/*/[0-9]*[0-9] src/*.sh check:634:20: note: Double quote to prevent globbing and word splitting. [SC2086] check:667:20: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:447:5: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:453:6: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:553:42: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:554:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:556:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:557:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:558:23: note: Double quote to prevent globbing and word splitting. [SC2086] tests/pr.bk/001:9:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/001:11:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/001:12:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/002:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/002:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/002:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/003:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/003:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/003:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/004:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/004:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/004:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/005:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/005:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/005:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/006:9:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/006:11:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/006:12:1: warning: TIMED appears unused. Verify use (or export if used externally). [SC2034] make: *** [Makefile:22: check] Error 1 real 3m43.712s user 3m38.640s sys 0m3.321s Running shellcheck with 48 parallel jobs... tests/pr.bk/005:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/005:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/005:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/006:9:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/006:11:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/006:12:1: warning: TIMED appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/001:9:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/001:11:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/001:12:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/002:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/002:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/002:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/004:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/004:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/004:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/003:8:3: note: Not following: tests/pr/rc: openBinaryFile: does not exist (No such file or directory) [SC1091] tests/pr.bk/003:10:1: warning: DESCRIPTION appears unused. Verify use (or export if used externally). [SC2034] tests/pr.bk/003:11:1: warning: QUICK appears unused. Verify use (or export if used externally). [SC2034] check:634:20: note: Double quote to prevent globbing and word splitting. [SC2086] check:667:20: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:447:5: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:453:6: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:553:42: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:554:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:556:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:557:28: note: Double quote to prevent globbing and word splitting. [SC2086] common/nvme:558:23: note: Double quote to prevent globbing and word splitting. [SC2086] make: *** [Makefile:30: check-parallel] Error 1 real 0m22.464s user 7m24.518s sys 0m11.703s 10.0x faster than sequential Concrete numbers: - make check: 223.7s (3m43.7s) - make check-parallel: 22.5s Signed-off-by: Chaitanya Kulkarni --- Makefile | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 36dbcaf..7825c79 100644 --- a/Makefile +++ b/Makefile @@ -16,6 +16,7 @@ install: # SC2119: "Use foo "$@" if function's $1 should mean script's $1". False # positives on helpers like _init_scsi_debug. SHELLCHECK_EXCLUDE := SC2119 +NPROCS := $(shell nproc) check: shellcheck -x -e $(SHELLCHECK_EXCLUDE) -f gcc check common/* \ @@ -24,4 +25,15 @@ check: ! grep TODO tests/*/rc tests/*/[0-9]*[0-9] ! find -L -name '*.out' -perm /u=x+g=x+o=x -printf '%p is executable\n' | grep . -.PHONY: all check install +check-parallel: + @echo "Running shellcheck with $(NPROCS) parallel jobs..." + @ret=0; \ + find tests -type f -name '[0-9]*[0-9]' | \ + xargs -P $(NPROCS) -n 1 shellcheck -x -e $(SHELLCHECK_EXCLUDE) -f gcc || ret=1; \ + shellcheck -x -e $(SHELLCHECK_EXCLUDE) -f gcc check common/* tests/*/rc src/*.sh || ret=1; \ + shellcheck --exclude=$(SHELLCHECK_EXCLUDE),SC2154 --format=gcc new || ret=1; \ + grep TODO tests/*/rc tests/*/[0-9]*[0-9] && ret=1; \ + find -L -name '*.out' -perm /u=x+g=x+o=x -printf '%p is executable\n' | grep . && ret=1; \ + exit $$ret + +.PHONY: all check check-parallel install -- 2.40.0