How to scan Java source texts?
I'd like to scan Java source texts, printing one token per line.
I thought it might be possible with the compiler API, and
have read that it can return an AST, but I do not know how
to just obtain the tokens from the source code AST.
I am able to write a scanner for Java myself, but this would
take days. So I would like to shortcut it by using a Java SE
(with JDK) call. (I would not like to use a third-party
library, because when I use the Java SE compiler API, I can
be sure that this will be up-to-date with future Java-Versions.)
So, the best solution would be a short program getting this
information out of the Java compiler API. But I cannot find
an example for this in the web.
What does not seem to work is:
public class Main
{ public static void main( final java.lang.String[] args )throws java.io.IOException
{ final java.io.File javaFile = new java.io.File( "Main.java" );
final java.io.FileReader file = new java.io.FileReader( javaFile );
final java.io.StreamTokenizer streamTokenizer = new java.io.StreamTokenizer( file );
for( int i; true; )
{ i = streamTokenizer.nextToken();
if( i == java.io.StreamTokenizer.TT_EOF )break;
java.lang.System.out.println( streamTokenizer.sval ); }}}
Still, this gives the idea of what I want to accomplish.
For example, the scanner should decompose:
a+=b +"c\"d/*e"/*f*/
+g;
into
a
+=
b
+
"c\"d/*e"
/*f*/
+
g
;
(the comment ?/*f*/? can as well be deleted; also, there is
no need for any further information, such as token types.)