StringBuffer should not contain initial byte-order mark (BOM)
authorThomas Miedema <thomasmiedema@gmail.com>
Mon, 1 Sep 2014 20:11:50 +0000 (15:11 -0500)
committerAustin Seipp <austin@well-typed.com>
Mon, 1 Sep 2014 20:11:50 +0000 (15:11 -0500)
commit9e939403241b758a685834c9ff62edcd3172a2cf
treed59a43de3ff5440685c3961fb3abd145ff7fafc9
parente81e02807c7a0e723ed7b0e83418c95f99140449
StringBuffer should not contain initial byte-order mark (BOM)

Summary:
Just skipping over a BOM, but leaving it in the Stringbuffer, is not
sufficient. The Lexer calls prevChar when a regular expression starts
with '^' (which is a shorthand for '\n^'). It would never match on the
first line, since instead of '\n', prevChar would still return '\xfeff'.

Test Plan: validate

Reviewers: austin, ezyang

Reviewed By: austin, ezyang

Subscribers: simonmar, ezyang, carter

Differential Revision: https://phabricator.haskell.org/D176

GHC Trac Issues: #6016
compiler/utils/StringBuffer.lhs
testsuite/.gitignore
testsuite/tests/parser/unicode/T6016.hs [new file with mode: 0644]
testsuite/tests/parser/unicode/all.T