sphinx-build: fix python stack overflow (Trac #10950)
authorSergei Trofimovich <slyfox@gentoo.org>
Thu, 21 Jan 2016 21:43:37 +0000 (21:43 +0000)
committerSergei Trofimovich <siarheit@google.com>
Thu, 21 Jan 2016 21:43:37 +0000 (21:43 +0000)
Summary:
commit a034031a102bc08c76a6cdb104b72922ae22c96b did not
fix problem completely. Stack overflows still occasionally
happen.

Easy to test by the following Torture Test:

  while sphinx-build -T -N -E -a -b html \
        -d docs/users_guide/.doctrees-html \
        -D latex_paper_size=letter \
        docs/users_guide docs/users_guide/build-html/users_guide
  do
    echo again
  done

sphinx build large nested data structures when parses GHC manual
(docs/users_guide/glasgow_exts.rst is 455KB in size) which
can't be serialized useing default python call stack depth of 1000
calls.

The patch increases stack depth 10 times. Survived 2 hours of
Torture Test.

Signed-off-by: Sergei Trofimovich <siarheit@google.com>
Test Plan: ran Torture Test to make sure it is stable

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1809

GHC Trac Issues: #10950

docs/users_guide/conf.py

index 119223a..f9c326e 100644 (file)
@@ -147,6 +147,8 @@ def parse_flag(env, sig, signode):
 def setup(app):
     from sphinx.util.docfields import Field, TypedField
 
+    increase_python_stack()
+
     # the :ghci-cmd: directive used in ghci.rst
     app.add_object_type('ghci-cmd', 'ghci-cmd',
                         parse_node=parse_ghci_cmd,
@@ -171,3 +173,11 @@ def setup(app):
                             Field('since', label='Introduced in GHC version', names=['since']),
                             Field('static')
                         ])
+
+def increase_python_stack():
+    # Workaround sphinx-build recursion limit overflow:
+    # pickle.dump(doctree, f, pickle.HIGHEST_PROTOCOL)
+    #  RuntimeError: maximum recursion depth exceeded while pickling an object
+    #
+    # Default python allows recursion depth of 1000 calls.
+    sys.setrecursionlimit(10000)