ApiAnnotations: Add SourceText for unicode tokens
authorAlan Zimmerman <alan.zimm@gmail.com>
Mon, 16 Nov 2015 17:43:34 +0000 (19:43 +0200)
committerAlan Zimmerman <alan.zimm@gmail.com>
Mon, 16 Nov 2015 17:43:34 +0000 (19:43 +0200)
commitfe95463bdf42651d53d1edc7723491664257cc5a
treef43452b6cbe66b0d83e0a9fa65b83a5f05989833
parent46a03fbec6a02761db079d1746532565f34c340f
ApiAnnotations: Add SourceText for unicode tokens

Summary:
At the moment there is no way to tell if a given token used its unicode
variant or its normal one, except to look at the length of the token.

This fails for the unicode '*'.

Expose the original source text for unicode variants so that API
Annotations can capture them specifically.

Test Plan: ./validate

Reviewers: mpickering, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1473

GHC Trac Issues: #11018
16 files changed:
compiler/parser/ApiAnnotation.hs
compiler/parser/Lexer.x
compiler/parser/Parser.y
testsuite/tests/ghc-api/annotations-literals/literals.stdout
testsuite/tests/ghc-api/annotations/Makefile
testsuite/tests/ghc-api/annotations/T10307.stdout
testsuite/tests/ghc-api/annotations/T10312.stdout
testsuite/tests/ghc-api/annotations/T10357.stdout
testsuite/tests/ghc-api/annotations/T10358.stdout
testsuite/tests/ghc-api/annotations/T11018.stderr [new file with mode: 0644]
testsuite/tests/ghc-api/annotations/T11018.stdout [new file with mode: 0644]
testsuite/tests/ghc-api/annotations/Test11018.hs [new file with mode: 0644]
testsuite/tests/ghc-api/annotations/all.T
testsuite/tests/ghc-api/annotations/exampleTest.stdout
testsuite/tests/ghc-api/annotations/listcomps.stdout
testsuite/tests/ghc-api/annotations/parseTree.stdout