Re: How to strip comments out of code

Esmond Pitt wrote:


Javac's lexer do not removes comments (not all at least). Important
comments, i.e. /** ... */ must be preserver for parser because they may
contain information needed for code generation (e.g. @deprecated Javadoc
tags).

In fact, there is not clear distinction between the javac lexer, and
parser I think...

BTW, The OP may also utilize the Java Compiler API (JSR-199) and its
Tree API (the latter is still under com.sun.*, but AFAIK is "almost"
stable now...). The starting point example is below (requires
tolls.jar!). It needs more detailed scanning of source tree (extend
TreeScanner) because of current Tree.toString() implementations gives
not so exact preview of the original source code (e.g. annotations'
attribute default values are skipped from output, etc...). In the OP's
particular problem I prefer to use simplified "stripper" (the one sent
by me earlier to this thread), because everything is under "my control"
there. However, the 199 API usages are much wider than that, so its
importance is much beyond my simple approach.

piotr

import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;

import com.sun.source.tree.AnnotationTree;
import com.sun.source.tree.CompilationUnitTree;
import com.sun.source.tree.ImportTree;
import com.sun.source.tree.Tree;
import com.sun.source.tree.TreeVisitor;
import com.sun.source.util.TreeScanner;

public class JavaCBasedCommentStripper {

   public static void main(String[] args) throws Exception {
     final JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
     final StandardJavaFileManager fileManager = compiler
         .getStandardFileManager(null, null, null);
     Iterable<? extends JavaFileObject> compilationUnits = fileManager
         .getJavaFileObjects("JavaCBasedCommentStripper.java");
     com.sun.source.util.JavacTask jt = (com.sun.source.util.JavacTask)
compiler
         .getTask(null, fileManager, null, null, null, compilationUnits);
     Iterable<? extends CompilationUnitTree> ts = jt.parse();

     for (CompilationUnitTree cu : ts) {
     // System.out.println(cu); // preserves /** comments */

       for(AnnotationTree at : cu.getPackageAnnotations()) {
         System.out.println(at);
       }
       String pkg = cu.getPackageName().toString();
       if (!pkg.equals("")) {
         System.out.println("package " + pkg + ";\n");
       }
       for(ImportTree it : cu.getImports()) {
         System.out.print(it);
       }

       for(Tree td : cu.getTypeDecls()) {
         System.out.println(td); // not all details in output!

         // extend the following instead...
// TreeVisitor<Void, Void> tv = new TreeScanner<Void, Void>() {
//
// @Override
// public Void visit...
//
// };
// td.accept(tv, null);

       }
     }
   }
}