Remote GHCi, -fexternal-interpreter
authorSimon Marlow <marlowsd@gmail.com>
Wed, 18 Nov 2015 16:42:24 +0000 (16:42 +0000)
committerSimon Marlow <marlowsd@gmail.com>
Thu, 17 Dec 2015 09:39:52 +0000 (09:39 +0000)
commit4905b83a2d448c65ccced385343d4e8124548a3b
tree070cf9e48f6fce668cd01d888b8da8b3772d1f53
parent7221ad70daa363d77f60d96c3f6e1baa1d9bec81
Remote GHCi, -fexternal-interpreter

Summary:
(Apologies for the size of this patch, I couldn't make a smaller one
that was validate-clean and also made sense independently)

(Some of this code is derived from GHCJS.)

This commit adds support for running interpreted code (for GHCi and
TemplateHaskell) in a separate process.  The functionality is
experimental, so for now it is off by default and enabled by the flag
-fexternal-interpreter.

Reaosns we want this:

* compiling Template Haskell code with -prof does not require
  building the code without -prof first

* when GHC itself is profiled, it can interpret unprofiled code, and
  the same applies to dynamic linking.  We would no longer need to
  force -dynamic-too with TemplateHaskell, and we can load ordinary
  objects into a dynamically-linked GHCi (and vice versa).

* An unprofiled GHCi can load and run profiled code, which means it
  can use the stack-trace functionality provided by profiling without
  taking the performance hit on the compiler that profiling would
  entail.

Amongst other things; see
https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi for more details.

Notes on the implementation are in Note [Remote GHCi] in the new
module compiler/ghci/GHCi.hs.  It probably needs more documenting,
feel free to suggest things I could elaborate on.

Things that are not currently implemented for -fexternal-interpreter:

* The GHCi debugger
* :set prog, :set args in GHCi
* `recover` in Template Haskell
* Redirecting stdin/stdout for the external process

These are all doable, I just wanted to get to a working validate-clean
patch first.

I also haven't done any benchmarking yet.  I expect there to be slight hit
to link times for byte code and some penalty due to having to
serialize/deserialize TH syntax, but I don't expect it to be a serious
problem.  There's also lots of low-hanging fruit in the byte code
generator/linker that we could exploit to speed things up.

Test Plan:
* validate
* I've run parts of the test suite with
EXTRA_HC_OPTS=-fexternal-interpreter, notably tests/ghci and tests/th.
There are a few failures due to the things not currently implemented
(see above).

Reviewers: simonpj, goldfire, ezyang, austin, alanz, hvr, niteria, bgamari, gibiansky, luite

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1562
101 files changed:
.gitignore
aclocal.m4
compiler/basicTypes/BasicTypes.hs
compiler/basicTypes/Literal.hs
compiler/coreSyn/MkCore.hs
compiler/deSugar/Coverage.hs
compiler/ghc.cabal.in
compiler/ghc.mk
compiler/ghci/ByteCodeAsm.hs
compiler/ghci/ByteCodeGen.hs
compiler/ghci/ByteCodeInstr.hs
compiler/ghci/ByteCodeItbls.hs
compiler/ghci/ByteCodeLink.hs
compiler/ghci/ByteCodeTypes.hs [new file with mode: 0644]
compiler/ghci/Debugger.hs
compiler/ghci/DebuggerUtils.hs
compiler/ghci/GHCi.hs [new file with mode: 0644]
compiler/ghci/Linker.hs
compiler/ghci/RtClosureInspect.hs
compiler/main/Annotations.hs
compiler/main/DriverPipeline.hs
compiler/main/DynFlags.hs
compiler/main/DynamicLoading.hs
compiler/main/GHC.hs
compiler/main/GhcMake.hs
compiler/main/GhcPlugins.hs
compiler/main/Hooks.hs
compiler/main/HscMain.hs
compiler/main/HscTypes.hs
compiler/main/InteractiveEval.hs
compiler/main/InteractiveEvalTypes.hs
compiler/main/SysTools.hs
compiler/specialise/SpecConstr.hs
compiler/typecheck/TcRnDriver.hs
compiler/typecheck/TcRnMonad.hs
compiler/typecheck/TcRnTypes.hs
compiler/typecheck/TcSplice.hs
compiler/typecheck/TcSplice.hs-boot
compiler/utils/Binary.hs
compiler/utils/Outputable.hs
compiler/utils/Panic.hs
ghc.mk
ghc/GhciMonad.hs
ghc/InteractiveUI.hs
ghc/Main.hs
ghc/ghc-bin.cabal.in
iserv/Main.hs [new file with mode: 0644]
iserv/Makefile [new file with mode: 0644]
iserv/ghc.mk [new file with mode: 0644]
iserv/iserv-bin.cabal [new file with mode: 0644]
iserv/iservmain.c [new file with mode: 0644]
libraries/ghc-boot/GHC/LanguageExtensions.hs
libraries/ghc-boot/GHC/Serialized.hs [moved from compiler/utils/Serialized.hs with 90% similarity]
libraries/ghc-boot/ghc-boot.cabal
libraries/ghci/GHCi/CreateBCO.hs [new file with mode: 0644]
libraries/ghci/GHCi/FFI.hsc [new file with mode: 0644]
libraries/ghci/GHCi/InfoTable.hsc [new file with mode: 0644]
libraries/ghci/GHCi/Message.hs [new file with mode: 0644]
libraries/ghci/GHCi/ObjLink.hs [moved from compiler/ghci/ObjLink.hs with 72% similarity]
libraries/ghci/GHCi/RemoteTypes.hs [new file with mode: 0644]
libraries/ghci/GHCi/ResolvedBCO.hs [new file with mode: 0644]
libraries/ghci/GHCi/Run.hs [new file with mode: 0644]
libraries/ghci/GHCi/Signals.hs [new file with mode: 0644]
libraries/ghci/GHCi/TH.hs [new file with mode: 0644]
libraries/ghci/GHCi/TH/Binary.hs [new file with mode: 0644]
libraries/ghci/GNUmakefile [new file with mode: 0644]
libraries/ghci/LICENSE [new file with mode: 0644]
libraries/ghci/SizedSeq.hs [new file with mode: 0644]
libraries/ghci/ghc.mk [new file with mode: 0644]
libraries/ghci/ghci.cabal [new file with mode: 0644]
rts/Interpreter.c
rules/build-prog.mk
rules/shell-wrapper.mk
testsuite/config/ghc
testsuite/driver/testlib.py
testsuite/tests/annotations/should_run/annrun01.hs
testsuite/tests/cabal/cabal04/Makefile
testsuite/tests/cabal/cabal04/all.T
testsuite/tests/ghc-api/T4891/T4891.hs
testsuite/tests/ghci.debugger/scripts/break006.stderr
testsuite/tests/ghci.debugger/scripts/break011.script
testsuite/tests/ghci.debugger/scripts/break011.stdout
testsuite/tests/ghci.debugger/scripts/break013.stdout
testsuite/tests/ghci.debugger/scripts/break024.stdout
testsuite/tests/ghci.debugger/scripts/print019.stderr
testsuite/tests/ghci/prog001/prog001-ext.stdout [new file with mode: 0644]
testsuite/tests/ghci/prog001/prog001.T
testsuite/tests/ghci/scripts/T10110A.hs
testsuite/tests/ghci/scripts/all.T
testsuite/tests/profiling/should_run/scc003.prof.sample
testsuite/tests/rts/LinkerUnload.hs
testsuite/tests/rts/T2615.hs
testsuite/tests/th/Makefile
testsuite/tests/th/TH_Roles2.stderr
testsuite/tests/th/TH_finalizer.hs [new file with mode: 0644]
testsuite/tests/th/TH_finalizer.stderr [new file with mode: 0644]
testsuite/tests/th/TH_spliceE5_prof_ext.hs [new file with mode: 0644]
testsuite/tests/th/TH_spliceE5_prof_ext.stdout [new file with mode: 0644]
testsuite/tests/th/TH_spliceE5_prof_ext_Lib.hs [new file with mode: 0644]
testsuite/tests/th/all.T
utils/ghctags/Main.hs