Adjust normal runtimes for nofib along with related changes
authorklebinger.andreas@gmx.at <klebinger.andreas@gmx.at>
Wed, 14 Nov 2018 17:07:43 +0000 (18:07 +0100)
committerAlp Mestanogullari <alpmestan@gmail.com>
Wed, 14 Nov 2018 17:11:13 +0000 (18:11 +0100)
commit08cc9b6b2c7f7fdaaaf80361ab84a501f0a573c5
tree32ac5979c9c0c100d547c380f78ef9157429be40
parenta80baacfc29cc2e7ed50e94f3cd2648d11b1d7d5
Adjust normal runtimes for nofib along with related changes

Runtime for nofib benchmarks was all over the place.
This patch adjusts runtime for most benchmarks such
that it falls into the 0.2-2s range.

This means that:
* A default run will take longer
* Time spent will be better distributed among benchmarks.
* More benchmarks have runtimes long enough to be used
  for runtime analysis.

Some more changes were done which go hand in hand
with changing runtimes.
* Some benchmarks now create their input files during boot.
* Moved input files for anna in their own directory.
* Remove printing of output for some of the floating
  point heavy benchmarks.
* Added a comment about desired runtimes to README.
* Made grep actually benchmark something.
* Throw cachgrind out of the default benchmarks.
  The nondeterministic behaviour has been an issue for a
  while and it doesn't seem like an essential benchmark.

Test Plan: run nofib in modes slow/normal/fast

Reviewers: O26 nofib, alpmestan

Reviewed By: alpmestan

Subscribers: sgraf, alpmestan

Differential Revision: https://phabricator.haskell.org/D4989
280 files changed:
.gitignore
README.md
gc/circsim/Makefile
gc/circsim/circsim.slowstdout [new file with mode: 0644]
gc/circsim/circsim.stdout
gc/constraints/Makefile
gc/constraints/constraints.slowstdout [new file with mode: 0644]
gc/constraints/constraints.stdout
gc/gc_bench/Makefile
gc/happy/TestInput.hs
gc/hash/Makefile
gc/lcss/Makefile
gc/lcss/lcss.faststdout
gc/lcss/lcss.slowstdout
gc/lcss/lcss.stdout
gc/mutstore1/Makefile
gc/power/Makefile
gc/power/power.faststdout [moved from spectral/power/power.slowstdout with 87% similarity]
gc/power/power.slowstdout
gc/power/power.stdout
imaginary/bernouilli/Makefile
imaginary/bernouilli/bernouilli.faststdout [new file with mode: 0644]
imaginary/bernouilli/bernouilli.stdout
imaginary/digits-of-e1/Makefile
imaginary/digits-of-e1/digits-of-e1.slowstdout
imaginary/digits-of-e1/digits-of-e1.stdout
imaginary/digits-of-e2/Makefile
imaginary/digits-of-e2/digits-of-e2.slowstdout
imaginary/digits-of-e2/digits-of-e2.stdout
imaginary/exp3_8/Makefile
imaginary/exp3_8/exp3_8.faststdout [new file with mode: 0644]
imaginary/exp3_8/exp3_8.slowstdout [deleted file]
imaginary/exp3_8/exp3_8.stdout
imaginary/gen_regexps/gen_regexps.faststdin [new file with mode: 0644]
imaginary/gen_regexps/gen_regexps.faststdout [new file with mode: 0644]
imaginary/gen_regexps/gen_regexps.slowstdin [deleted file]
imaginary/gen_regexps/gen_regexps.slowstdout [deleted file]
imaginary/gen_regexps/gen_regexps.stdin
imaginary/gen_regexps/gen_regexps.stdout
imaginary/integrate/Makefile
imaginary/integrate/integrate.faststdout [new file with mode: 0644]
imaginary/integrate/integrate.slowstdout
imaginary/integrate/integrate.stdout
imaginary/paraffins/Makefile
imaginary/paraffins/paraffins.faststdout [new file with mode: 0644]
imaginary/paraffins/paraffins.slowstdout [deleted file]
imaginary/paraffins/paraffins.stdout
imaginary/primes/Makefile
imaginary/primes/primes.slowstdout
imaginary/primes/primes.stdout
imaginary/queens/Makefile
imaginary/queens/queens.faststdout [new file with mode: 0644]
imaginary/queens/queens.slowstdout [deleted file]
imaginary/queens/queens.stdout
imaginary/rfib/Makefile
imaginary/rfib/rfib.faststdout [moved from imaginary/rfib/rfib.slowstdout with 100% similarity]
imaginary/rfib/rfib.stdout
imaginary/tak/Makefile
imaginary/tak/tak.slowstdout [deleted file]
imaginary/wheel-sieve1/Makefile
imaginary/wheel-sieve1/wheel-sieve1.slowstdout
imaginary/wheel-sieve1/wheel-sieve1.stdout
imaginary/wheel-sieve2/Makefile
imaginary/wheel-sieve2/wheel-sieve2.slowstdout [deleted file]
imaginary/wheel-sieve2/wheel-sieve2.stdout
imaginary/x2n1/Makefile
imaginary/x2n1/x2n1.faststdout [new file with mode: 0644]
imaginary/x2n1/x2n1.slowstdout [deleted file]
imaginary/x2n1/x2n1.stdout
real/Makefile
real/anna/Makefile
real/anna/anna.faststdin [moved from real/anna/big.cor with 100% similarity]
real/anna/anna.faststdout [new file with mode: 0644]
real/anna/anna.stdin [new file with mode: 0644]
real/anna/anna.stdout
real/anna/cor_files/ap_CaseAlts.cor [moved from real/anna/ap_CaseAlts.cor with 100% similarity]
real/anna/cor_files/ap_CaseArgs.cor [moved from real/anna/ap_CaseArgs.cor with 100% similarity]
real/anna/cor_files/ap_CaseOfCase.cor [moved from real/anna/ap_CaseOfCase.cor with 100% similarity]
real/anna/cor_files/ap_CaseOfCase2.cor [moved from real/anna/ap_CaseOfCase2.cor with 100% similarity]
real/anna/cor_files/ap_CaseOfCase3.cor [moved from real/anna/ap_CaseOfCase3.cor with 100% similarity]
real/anna/cor_files/ap_FuncCall.cor [moved from real/anna/ap_FuncCall.cor with 100% similarity]
real/anna/cor_files/ap_ListOfList.cor [moved from real/anna/ap_ListOfList.cor with 100% similarity]
real/anna/cor_files/ap_SemiLazyAdd.cor [moved from real/anna/ap_SemiLazyAdd.cor with 100% similarity]
real/anna/cor_files/ap_SemiLazyCase.cor [moved from real/anna/ap_SemiLazyCase.cor with 100% similarity]
real/anna/cor_files/ap_SimpleLazy.cor [moved from real/anna/ap_SimpleLazy.cor with 100% similarity]
real/anna/cor_files/ap_SimpleStrict.cor [moved from real/anna/ap_SimpleStrict.cor with 100% similarity]
real/anna/cor_files/ap_Unzip.cor [moved from real/anna/ap_Unzip.cor with 100% similarity]
real/anna/cor_files/ap_Zip.cor [moved from real/anna/ap_Zip.cor with 100% similarity]
real/anna/cor_files/append.cor [moved from real/anna/append.cor with 100% similarity]
real/anna/cor_files/avlTree.cor [moved from real/anna/avlTree.cor with 100% similarity]
real/anna/cor_files/big.cor [new file with mode: 0644]
real/anna/cor_files/bmark.cor [moved from real/anna/bmark.cor with 100% similarity]
real/anna/cor_files/bomb.cor [moved from real/anna/bomb.cor with 100% similarity]
real/anna/cor_files/bomb2.cor [moved from real/anna/bomb2.cor with 100% similarity]
real/anna/cor_files/bug.cor [moved from real/anna/bug.cor with 100% similarity]
real/anna/cor_files/bug_Anna1.cor [moved from real/anna/bug_Anna1.cor with 100% similarity]
real/anna/cor_files/bug_Anna2.cor [moved from real/anna/bug_Anna2.cor with 100% similarity]
real/anna/cor_files/bug_Anna3.cor [moved from real/anna/bug_Anna3.cor with 100% similarity]
real/anna/cor_files/bug_Anna4.cor [moved from real/anna/bug_Anna4.cor with 100% similarity]
real/anna/cor_files/bug_Anna5.cor [moved from real/anna/bug_Anna5.cor with 100% similarity]
real/anna/cor_files/bug_Anna6.cor [moved from real/anna/bug_Anna6.cor with 100% similarity]
real/anna/cor_files/bug_types1.cor [moved from real/anna/bug_types1.cor with 100% similarity]
real/anna/cor_files/bug_types2.cor [moved from real/anna/bug_types2.cor with 100% similarity]
real/anna/cor_files/bury.cor [moved from real/anna/bury.cor with 100% similarity]
real/anna/cor_files/cfoldr.cor [moved from real/anna/cfoldr.cor with 100% similarity]
real/anna/cor_files/concat.cor [moved from real/anna/concat.cor with 100% similarity]
real/anna/cor_files/concat22.cor [moved from real/anna/concat22.cor with 100% similarity]
real/anna/cor_files/concat24.cor [moved from real/anna/concat24.cor with 100% similarity]
real/anna/cor_files/concat44.cor [moved from real/anna/concat44.cor with 100% similarity]
real/anna/cor_files/coreExpr.cor [moved from real/anna/coreExpr.cor with 100% similarity]
real/anna/cor_files/coreExpr0.cor [moved from real/anna/coreExpr0.cor with 100% similarity]
real/anna/cor_files/divide.cor [moved from real/anna/divide.cor with 100% similarity]
real/anna/cor_files/dividetree.cor [moved from real/anna/dividetree.cor with 100% similarity]
real/anna/cor_files/dot_3.cor [moved from real/anna/dot_3.cor with 100% similarity]
real/anna/cor_files/dot_4.cor [moved from real/anna/dot_4.cor with 100% similarity]
real/anna/cor_files/eta.cor [moved from real/anna/eta.cor with 100% similarity]
real/anna/cor_files/filter.cor [moved from real/anna/filter.cor with 100% similarity]
real/anna/cor_files/foldrFoldl.cor [moved from real/anna/foldrFoldl.cor with 100% similarity]
real/anna/cor_files/fourier.cor [moved from real/anna/fourier.cor with 100% similarity]
real/anna/cor_files/funcDomain.cor [moved from real/anna/funcDomain.cor with 100% similarity]
real/anna/cor_files/higherOrder.cor [moved from real/anna/higherOrder.cor with 100% similarity]
real/anna/cor_files/john.cor [moved from real/anna/john.cor with 100% similarity]
real/anna/cor_files/llfiasco.cor [moved from real/anna/llfiasco.cor with 100% similarity]
real/anna/cor_files/manyIterate.cor [moved from real/anna/manyIterate.cor with 100% similarity]
real/anna/cor_files/mutualRec.cor [moved from real/anna/mutualRec.cor with 100% similarity]
real/anna/cor_files/ol_num.cor [moved from real/anna/ol_num.cor with 100% similarity]
real/anna/cor_files/pairid.cor [moved from real/anna/pairid.cor with 100% similarity]
real/anna/cor_files/parallelOr.cor [moved from real/anna/parallelOr.cor with 100% similarity]
real/anna/cor_files/poly_loseGain.cor [moved from real/anna/poly_loseGain.cor with 100% similarity]
real/anna/cor_files/poly_simple.cor [moved from real/anna/poly_simple.cor with 100% similarity]
real/anna/cor_files/preludeList.cor [moved from real/anna/preludeList.cor with 100% similarity]
real/anna/cor_files/realNasties.cor [moved from real/anna/realNasties.cor with 100% similarity]
real/anna/cor_files/reverse.cor [moved from real/anna/reverse.cor with 100% similarity]
real/anna/cor_files/sebastian1.cor [moved from real/anna/sebastian1.cor with 100% similarity]
real/anna/cor_files/sebastian2.cor [moved from real/anna/sebastian2.cor with 100% similarity]
real/anna/cor_files/sets.cor [moved from real/anna/sets.cor with 100% similarity]
real/anna/cor_files/standardTest.cor [moved from real/anna/standardTest.cor with 100% similarity]
real/anna/cor_files/treeDepth.cor [moved from real/anna/treeDepth.cor with 100% similarity]
real/compress/Makefile
real/compress/compress.faststdin [moved from real/compress/compress.stdin with 100% similarity]
real/compress/compress.faststdout [moved from real/compress/compress.stdout with 100% similarity]
real/compress/compress.stdout-mingw [deleted file]
real/compress2/Makefile
real/compress2/compress2.faststdin [moved from real/compress2/compress2.stdin with 100% similarity]
real/compress2/compress2.faststdout [moved from real/compress2/compress2.stdout with 100% similarity]
real/fulsom/Makefile
real/fulsom/fulsom.faststdout [new file with mode: 0644]
real/fulsom/fulsom.stdout
real/fulsom/fulsom.stdout-mingw [deleted file]
real/gamteb/gamteb.faststdin [new file with mode: 0644]
real/gamteb/gamteb.faststdout [new file with mode: 0644]
real/gamteb/gamteb.faststdout1 [moved from real/gamteb/gamteb.stdout1 with 100% similarity]
real/gamteb/gamteb.stdin
real/gamteb/gamteb.stdout
real/grep/Makefile
real/grep/grep.stdout
real/hpg/Env.lhs
real/hpg/Makefile
real/hpg/README.nofib [new file with mode: 0644]
real/hpg/hpg.faststdout [new file with mode: 0644]
real/hpg/hpg.stderr [deleted file]
real/hpg/hpg.stdout
real/rsa/rsa.stdin
real/rsa/rsa.stdout
real/scs/Main.hs
real/scs/Makefile
real/scs/scs.stdout
real/scs/scs.stdout-x86-linux [deleted file]
real/scs/scs.stdout-x86_64 [deleted file]
real/symalg/symalg.faststdin [new file with mode: 0644]
real/symalg/symalg.faststdout [new file with mode: 0644]
real/symalg/symalg.stdin
real/symalg/symalg.stdout
shootout/binary-trees/Makefile
shootout/binary-trees/binary-trees.stdout
shootout/fasta/Makefile
shootout/k-nucleotide/Makefile
shootout/k-nucleotide/k-nucleotide.stdout
shootout/pidigits/Makefile
shootout/pidigits/pidigits.stdout
shootout/reverse-complement/Makefile
spectral/atom/Makefile
spectral/atom/atom.slowstdout [deleted file]
spectral/atom/atom.stdout
spectral/boyer/Makefile
spectral/circsim/Makefile
spectral/circsim/circsim.slowstdout [new file with mode: 0644]
spectral/circsim/circsim.stdout
spectral/clausify/Makefile
spectral/clausify/clausify.faststdout [new file with mode: 0644]
spectral/clausify/clausify.slowstdout [deleted file]
spectral/clausify/clausify.stdout
spectral/fft2/Main.lhs
spectral/fft2/Makefile
spectral/fft2/README.nofib [new file with mode: 0644]
spectral/fft2/fft2.stdout
spectral/fft2/old/Makefile [new file with mode: 0644]
spectral/fft2/old/fft2.slowstdout [moved from spectral/fft2/fft2.slowstdout with 100% similarity]
spectral/fft2/old/fft2.slowstdout-x86-linux [moved from spectral/fft2/fft2.slowstdout-x86-linux with 100% similarity]
spectral/fft2/old/fft2.slowstdout-x86-mingw [moved from spectral/fft2/fft2.slowstdout-x86-mingw with 100% similarity]
spectral/fft2/old/fft2.slowstdout-x86_64 [moved from spectral/fft2/fft2.slowstdout-x86_64 with 100% similarity]
spectral/fft2/old/fft2.stdout [new file with mode: 0644]
spectral/fft2/old/fft2.stdout-mingw [moved from spectral/fft2/fft2.stdout-mingw with 100% similarity]
spectral/fft2/old/fft2.stdout-x86_64 [moved from spectral/fft2/fft2.stdout-x86_64 with 100% similarity]
spectral/fft2/old/fft2.stdout1 [moved from spectral/fft2/fft2.stdout1 with 100% similarity]
spectral/fft2/old/fft2.stdout2 [moved from spectral/fft2/fft2.stdout2 with 100% similarity]
spectral/fft2/old/fft2.stdout3 [moved from spectral/fft2/fft2.stdout3 with 100% similarity]
spectral/fft2/old/fft2.stdout4 [moved from spectral/fft2/fft2.stdout4 with 100% similarity]
spectral/fft2/old/fft2.stdout5 [moved from spectral/fft2/fft2.stdout5 with 100% similarity]
spectral/fft2/old/fft2.stdout6 [moved from spectral/fft2/fft2.stdout6 with 100% similarity]
spectral/fft2/old/fft2.stdout7 [moved from spectral/fft2/fft2.stdout7 with 100% similarity]
spectral/fibheaps/Makefile
spectral/gcd/Makefile
spectral/gcd/gcd.faststdout [new file with mode: 0644]
spectral/gcd/gcd.slowstdout [deleted file]
spectral/gcd/gcd.stdout
spectral/hartel/comp_lab_zift/Makefile
spectral/hartel/comp_lab_zift/comp_lab_zift.faststdout [new file with mode: 0644]
spectral/hartel/comp_lab_zift/comp_lab_zift.slowstdout [deleted file]
spectral/hartel/comp_lab_zift/comp_lab_zift.stdout
spectral/hartel/event/Makefile
spectral/hartel/fft/Makefile
spectral/hartel/fft/fft.faststdout [new file with mode: 0644]
spectral/hartel/fft/fft.slowstdout [deleted file]
spectral/hartel/fft/fft.stdout
spectral/hartel/genfft/Makefile
spectral/hartel/genfft/genfft.faststdout [new file with mode: 0644]
spectral/hartel/genfft/genfft.slowstdout [deleted file]
spectral/hartel/genfft/genfft.stdout
spectral/hartel/ida/Makefile
spectral/hartel/ida/ida.faststdout [new file with mode: 0644]
spectral/hartel/ida/ida.slowstdout [deleted file]
spectral/hartel/ida/ida.stdout
spectral/hartel/sched/Makefile
spectral/hartel/sched/sched.faststdout [new file with mode: 0644]
spectral/hartel/sched/sched.slowstdout [deleted file]
spectral/hartel/sched/sched.stdout
spectral/hartel/solid/Makefile
spectral/hartel/solid/solid.faststdout [new file with mode: 0644]
spectral/hartel/solid/solid.slowstdout
spectral/hartel/solid/solid.stdout
spectral/hartel/transform/Makefile
spectral/hartel/transform/transform.faststdout [new file with mode: 0644]
spectral/hartel/transform/transform.slowstdout [deleted file]
spectral/hartel/transform/transform.stdout
spectral/hartel/typecheck/Makefile
spectral/hartel/typecheck/typecheck.faststdout [new file with mode: 0644]
spectral/hartel/typecheck/typecheck.slowstdout [deleted file]
spectral/hartel/typecheck/typecheck.stdout
spectral/hartel/wang/Makefile
spectral/hartel/wang/wang.faststdout [new file with mode: 0644]
spectral/hartel/wang/wang.slowstdout [deleted file]
spectral/hartel/wang/wang.stdout
spectral/hartel/wave4main/Makefile
spectral/hartel/wave4main/wave4main.faststdout [moved from spectral/hartel/wave4main/wave4main.stdout2 with 100% similarity]
spectral/hartel/wave4main/wave4main.slowstdout [deleted file]
spectral/hartel/wave4main/wave4main.slowstdout-x86-linux [deleted file]
spectral/hartel/wave4main/wave4main.stdout
spectral/hartel/wave4main/wave4main.stdout3 [deleted file]
spectral/knights/Makefile
spectral/knights/knights.faststdout [new file with mode: 0644]
spectral/knights/knights.slowstdout
spectral/knights/knights.stdout
spectral/lcss/Makefile
spectral/lcss/lcss.slowstdout [deleted file]
spectral/lcss/lcss.stdout
spectral/multiplier/Makefile
spectral/power/Makefile
spectral/power/power.faststdout [new file with mode: 0644]
spectral/power/power.stdout
spectral/primetest/primetest.faststdin [moved from spectral/primetest/primetest.slowstdin with 50% similarity]
spectral/primetest/primetest.faststdout [new file with mode: 0644]
spectral/primetest/primetest.slowstdout [deleted file]
spectral/primetest/primetest.stdin
spectral/primetest/primetest.stdout
spectral/rewrite/Makefile
spectral/sphere/Makefile
spectral/sphere/sphere.faststdout [new file with mode: 0644]
spectral/sphere/sphere.slowstdout [deleted file]
spectral/sphere/sphere.stdout