Debug Enso language in ChromeDev tools with --inspect option (#3432)

Finally this pull request proposes `--inspect` option to allow [debugging of `.enso`](e948f2535f/docs/debugger/README.md) in Chrome Developer Tools:

```bash
enso$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --inspect --run ./test/Tests/src/Data/Numbers_Spec.enso
Debugger listening on ws://127.0.0.1:9229/Wugyrg9Nm4OUL9YhzdcElmLft71ayZW3LMUPCdPyNAY
For help, see: https://www.graalvm.org/tools/chrome-debugger
E.g. in Chrome open: devtools://devtools/bundled/js_app.html?ws=127.0.0.1:9229/Wugyrg9Nm4OUL9YhzdcElmLft71ayZW3LMUPCdPyNAY
```
copy the printed URL into chrome browser and you should see:

![obrazek](https://user-images.githubusercontent.com/26887752/167235327-8ad15fb2-96d4-4a0c-9e31-ed67ab46578b.png)

One can also debug the `.enso` files in NetBeans or [VS Code with Apache Language Server extension](https://cwiki.apache.org/confluence/display/NETBEANS/Apache+NetBeans+Extension+for+Visual+Studio+Code) just pass in special JVM arguments:
```bash
enso$ JAVA_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,address=8000 ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run ./test/Tests/src/Data/Numbers_Spec.enso
Listening for transport dt_socket at address: 8000
```
and then _Debug/Attach Debugger_. Once connected choose the _Toggle Pause in GraalVM Script_ button in the toolbar (the "G" button):

![obrazek](https://user-images.githubusercontent.com/26887752/167235598-98266c7e-beb5-406b-adc6-8167b3d1b453.png)

and your execution shall stop on the next `.enso` line of code. This mode allows to debug both - the Enso code as well as Java code.

Originally started as an attempt to write test in Java:

* test written in Java
* support for JUnit in `build.sbt`
* compile Java with `-g` - so it can be debugged
* Implementation of `StatementNode` - only gets created when `materialize` request gets to `BlockNode`

CHANGELOG.md

@@ -193,6 +193,7 @@
 - [Provide `tagValues` for function arguments in the language server][3422]
 - [Delay construction of Truffle nodes to speed initialization][3429]
 - [Frgaal compiler integration to allow for latest Java constructs][3421]
+- [Support for Chrome developer tools --inspect option][3432]
 - [Move Builtin Types and Methods definitions to stdlib][3363]
 
 [3227]: https://github.com/enso-org/enso/pull/3227
@@ -209,6 +210,7 @@
 [3422]: https://github.com/enso-org/enso/pull/3422
 [3429]: https://github.com/enso-org/enso/pull/3429
 [3421]: https://github.com/enso-org/enso/pull/3421
+[3432]: https://github.com/enso-org/enso/pull/3432
 [3363]: https://github.com/enso-org/enso/pull/3363
 
 # Enso 2.0.0-alpha.18 (2021-10-12)

build.sbt

@@ -155,7 +155,8 @@ Global / onChangedBuildSource := ReloadOnSourceChanges
 ThisBuild / javacOptions ++= Seq(
   "-encoding",   // Provide explicit encoding (the next line)
   "UTF-8",       // Specify character encoding used by Java source files.
-  "-deprecation" // Shows a description of each use or override of a deprecated member or class.
+  "-deprecation",// Shows a description of each use or override of a deprecated member or class.
+  "-g"           // Include debugging information
 )
 
 ThisBuild / scalacOptions ++= Seq(
@@ -1152,6 +1153,8 @@ lazy val runtime = (project in file("engine/runtime"))
       "org.graalvm.truffle" % "truffle-api"           % graalVersion      % Benchmark,
       "org.typelevel"      %% "cats-core"             % catsVersion,
       "eu.timepit"         %% "refined"               % refinedVersion,
+     "junit" % "junit" % "4.12" % Test,
+     "com.novocode" % "junit-interface" % "0.11" % Test exclude("junit", "junit-dep"),
       // This dependency is needed only so that developers don't download Frgaal manually.
       // Sadly it cannot be placed under plugins either because meta dependencies are not easily
       // accessible from the non-meta build definition.

docs/CONTRIBUTING.md

@@ -356,6 +356,14 @@ withDebug run --dumpGraphs --printAssembly -- --run MyFile.enso
 withDebug benchOnly --showCompilations -- RecursionBenchmark
 ```
 
+Step by step debugging can be triggered as
+
+```
+withDebug testOnly --debugger -- *FavoriteTest*
+```
+
+read more about [debugging Java & Enso code](debugger/README.md).
+
 #### Working with Assembly
 
 In order to examine the assembly generated by GraalVM and HotSpot you need to
@@ -480,9 +488,8 @@ filing an issue with us.
   `sbt` invocation.
 - **Debugging Not Working:** The sbt tasks run the invoked programs in a forked
   JVM. This means that to attach a debugger to it you need to use the JVM remote
-  debugging support. We cannot support all possible configurations for this, but
-  if you use IntelliJ please see the [Using IntelliJ](#using-intellj) section
-  above for instructions.
+  debugging support. Follow [Enso debugging instructions](debugger/README.md) or
+  see the [Using IntelliJ](#using-intellj) section for instructions.
 
 If your problem was not listed above, please
 [file a bug report](https://github.com/enso-org/enso/issues/new?assignees=&labels=Type%3A+Bug&template=bug-report.md&title=)
@@ -659,8 +666,11 @@ The project manager will look for the appropriate subdirectory in the _engines_
 directory of the distribution folder. Distribution paths are printed when you
 run project manager with `-v` verbose logging.
 
+Btw. you can specify `ENSO_JVM_OPTS` to turn
+[debugging of the Engine runtime](debugger/README.md) on:
+
 ```bash
-$ export ENSO_JVM_OPTS=-agentlib:jdwp=transport=dt_socket,address=5005 # to turn debugging of Engine runtime on
+$ export ENSO_JVM_OPTS=-agentlib:jdwp=transport=dt_socket,address=5005
 $ ./built-distribution/enso-project-manager-0.0.0-dev-linux-amd64/enso/bin/project-manager --no-log-masking -v
 [info] [2021-06-16T11:49:33.639Z] [org.enso.projectmanager.boot.ProjectManager$] Starting Project Manager...
 [debug] [2021-06-16T11:49:33.639Z] [org.enso.runtimeversionmanager.distribution.DistributionManager] Detected paths: DistributionPaths(

docs/debugger/README.md

@@ -6,7 +6,7 @@ tags: [debugger, repl, readme]
 order: 0
 ---
 
-# Debugger
+# Enso Debugger
 
 The Enso Debugger allows amongst other things, to execute arbitrary expressions
 in a given execution context - this is used to implement a debugging REPL. The
@@ -16,3 +16,55 @@ This folder contains all documentation pertaining to the REPL and the debugger,
 which is broken up as follows:
 
 - [**The Enso Debugger Protocol:**](./protocol.md) The protocol for the Debugger
+
+# Chrome Developer Tools Debugger
+
+As a well written citizen of the [GraalVM](http://graalvm.org) project the Enso
+language can be used with existing tools available for the overall platform. One
+of them is
+[Chrome Debugger](https://www.graalvm.org/22.1/tools/chrome-debugger/) and Enso
+language is fully integrated with it. Launch the `bin/enso` executable with
+additional `--inspect` option and debug your Enso programs in _Chrome Developer
+Tools_.
+
+```bash
+enso$ ./built-distribution/enso-engine-*/enso-*/bin/enso --inspect --run ./test/Tests/src/Data/Numbers_Spec.enso
+Debugger listening on ws://127.0.0.1:9229/Wugyrg9
+For help, see: https://www.graalvm.org/tools/chrome-debugger
+E.g. in Chrome open: devtools://devtools/bundled/js_app.html?ws=127.0.0.1:9229/Wugyrg9
+```
+
+copy the printed URL into chrome browser and you should see:
+
+![Chrome Debugger](chrome-debugger.png)
+
+Step in, step over, set breakpoints, watch values of your variables as execution
+of your Enso program progresses.
+
+# Debugging Enso and Java Code at Once
+
+Enso libraries are written in a mixture of Enso code and Java libraries.
+Debugging both sides (the Java as well as Enso code) is possible with a decent
+IDE.
+
+Get [NetBeans](http://netbeans.apache.org) version 13 or newer or
+[VS Code with Apache Language Server extension](https://cwiki.apache.org/confluence/display/NETBEANS/Apache+NetBeans+Extension+for+Visual+Studio+Code)
+and just pass in special JVM arguments when launching the `bin/enso` launcher:
+
+```bash
+enso$ JAVA_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,address=8000 ./built-distribution/enso-engine-*/enso-*/bin/enso --run ./test/Tests/src/Data/Numbers_Spec.enso
+Listening for transport dt_socket at address: 8000
+```
+
+and then _Debug/Attach Debugger_. Once connected suspend the execution and (if
+the Enso language has already been started) choose the _Toggle Pause in GraalVM
+Script_ button in the toolbar:
+
+![NetBeans Debugger](java-debugger.png)
+
+and your execution shall stop on the next `.enso` line of code. This mode allows
+to debug both - the Enso code as well as Java code. The stack traces shows a
+mixture of Java and Enso stack frames by default. Right-clicking on the thread
+allows one to switch to plain Java view (with a way more stack frames) and back.
+Analyzing low level details as well as Enso developer point of view shall be
+simple with such tool.

engine/runner/src/main/scala/org/enso/runner/ContextFactory.scala

@@ -26,6 +26,7 @@ class ContextFactory {
     * @param strictErrors whether or not to use strict errors
     * @param useGlobalIrCacheLocation whether or not to use the global IR cache
     *                                 location
+    * @param options additional options for the Context
     * @return configured Context instance
     */
   def create(
@@ -36,9 +37,10 @@ class ContextFactory {
     logLevel: LogLevel,
     logMasking: Boolean,
     enableIrCaches: Boolean,
-    strictErrors: Boolean             = false,
-    useGlobalIrCacheLocation: Boolean = true,
-    enableAutoParallelism: Boolean    = false
+    strictErrors: Boolean                  = false,
+    useGlobalIrCacheLocation: Boolean      = true,
+    enableAutoParallelism: Boolean         = false,
+    options: java.util.Map[String, String] = java.util.Collections.emptyMap
   ): PolyglotContext = {
     val context = Context
       .newBuilder()
@@ -54,6 +56,7 @@ class ContextFactory {
       .option(RuntimeOptions.DISABLE_IR_CACHES, (!enableIrCaches).toString)
       .option(DebugServerInfo.ENABLE_OPTION, "true")
       .option(RuntimeOptions.LOG_MASKING, logMasking.toString)
+      .options(options)
       .option(
         RuntimeOptions.ENABLE_AUTO_PARALLELISM,
         enableAutoParallelism.toString

engine/runner/src/main/scala/org/enso/runner/Main.scala

@@ -22,11 +22,13 @@ import scala.Console.err
 import scala.jdk.CollectionConverters._
 import scala.util.{Failure, Success, Try}
 import scala.util.control.NonFatal
+import java.util.Collections
 
 /** The main CLI entry point class. */
 object Main {
 
   private val RUN_OPTION                     = "run"
+  private val INSPECT_OPTION                 = "inspect"
   private val HELP_OPTION                    = "help"
   private val NEW_OPTION                     = "new"
   private val PROJECT_NAME_OPTION            = "new-project-name"
@@ -88,6 +90,10 @@ object Main {
       .longOpt(RUN_OPTION)
       .desc("Runs a specified Enso file.")
       .build
+    val inspect = CliOption.builder
+      .longOpt(INSPECT_OPTION)
+      .desc("Start the Chrome inspector when --run is used.")
+      .build
     val docs = CliOption.builder
       .longOpt(DOCS_OPTION)
       .desc("Runs the Enso documentation generator.")
@@ -312,6 +318,7 @@ object Main {
       .addOption(help)
       .addOption(repl)
       .addOption(run)
+      .addOption(inspect)
       .addOption(docs)
       .addOption(preinstall)
       .addOption(newOpt)
@@ -468,6 +475,7 @@ object Main {
     * @param logLevel       log level to set for the engine runtime
     * @param logMasking     is the log masking enabled
     * @param enableIrCaches are IR caches enabled
+    * @param inspect        shall inspect option be enabled
     */
   private def run(
     path: String,
@@ -475,7 +483,8 @@ object Main {
     logLevel: LogLevel,
     logMasking: Boolean,
     enableIrCaches: Boolean,
-    enableAutoParallelism: Boolean
+    enableAutoParallelism: Boolean,
+    inspect: Boolean
   ): Unit = {
     val file = new File(path)
     if (!file.exists) {
@@ -506,7 +515,10 @@ object Main {
       logMasking,
       enableIrCaches,
       strictErrors          = true,
-      enableAutoParallelism = enableAutoParallelism
+      enableAutoParallelism = enableAutoParallelism,
+      options =
+        if (inspect) Collections.singletonMap("inspect", "")
+        else Collections.emptyMap
     )
     if (projectMode) {
       PackageManager.Default.loadPackage(file) match {
@@ -948,7 +960,8 @@ object Main {
         logLevel,
         logMasking,
         shouldEnableIrCaches(line),
-        line.hasOption(AUTO_PARALLELISM_OPTION)
+        line.hasOption(AUTO_PARALLELISM_OPTION),
+        line.hasOption(INSPECT_OPTION)
       )
     }
     if (line.hasOption(REPL_OPTION)) {

engine/runtime/src/main/java/org/enso/interpreter/Language.java

@@ -53,6 +53,7 @@ import org.graalvm.options.OptionDescriptors;
   DebuggerTags.AlwaysHalt.class,
   StandardTags.CallTag.class,
   StandardTags.ExpressionTag.class,
+  StandardTags.StatementTag.class,
   StandardTags.RootTag.class,
   StandardTags.TryBlockTag.class,
   IdentifiedTag.class

engine/runtime/src/main/java/org/enso/interpreter/node/callable/function/BlockNode.java

@@ -1,8 +1,12 @@
 package org.enso.interpreter.node.callable.function;
 
 import com.oracle.truffle.api.frame.VirtualFrame;
+import com.oracle.truffle.api.instrumentation.InstrumentableNode;
+import com.oracle.truffle.api.instrumentation.StandardTags;
+import com.oracle.truffle.api.instrumentation.Tag;
 import com.oracle.truffle.api.nodes.ExplodeLoop;
 import com.oracle.truffle.api.nodes.NodeInfo;
+import java.util.Set;
 import org.enso.interpreter.node.ExpressionNode;
 
 /**
@@ -48,4 +52,16 @@ public class BlockNode extends ExpressionNode {
     }
     return returnExpr.executeGeneric(frame);
   }
+
+  @Override
+  public InstrumentableNode materializeInstrumentableNodes(
+      Set<Class<? extends Tag>> materializedTags) {
+    if (materializedTags.contains(StandardTags.StatementTag.class)) {
+      for (int i = 0; i < statements.length; i++) {
+        statements[i] = insert(StatementNode.wrap(statements[i]));
+      }
+      this.returnExpr = insert(StatementNode.wrap(returnExpr));
+    }
+    return this;
+  }
 }

engine/runtime/src/main/java/org/enso/interpreter/node/callable/function/StatementNode.java

@@ -0,0 +1,47 @@
+package org.enso.interpreter.node.callable.function;
+
+import com.oracle.truffle.api.frame.VirtualFrame;
+import com.oracle.truffle.api.instrumentation.StandardTags;
+import com.oracle.truffle.api.instrumentation.Tag;
+import com.oracle.truffle.api.source.SourceSection;
+import org.enso.interpreter.node.ExpressionNode;
+
+/**
+ * Node tagged with {@link StandardTags.StatementTag}. Inserted by {@link BlockNode} into the AST
+ * when debugger is connected.
+ */
+final class StatementNode extends ExpressionNode {
+  @Child ExpressionNode node;
+
+  private StatementNode(ExpressionNode node) {
+    this.node = node;
+  }
+
+  static StatementNode wrap(ExpressionNode node) {
+    if (node instanceof StatementNode statement) {
+      return statement;
+    } else {
+      return new StatementNode(node);
+    }
+  }
+
+  @Override
+  public SourceSection getSourceSection() {
+    return node.getSourceSection();
+  }
+
+  @Override
+  public boolean isInstrumentable() {
+    return getSourceSection() != null && node.isInstrumentable();
+  }
+
+  @Override
+  public Object executeGeneric(VirtualFrame frame) {
+    return node.executeGeneric(frame);
+  }
+
+  @Override
+  public boolean hasTag(Class<? extends Tag> tag) {
+    return StandardTags.StatementTag.class == tag;
+  }
+}

engine/runtime/src/test/java/org/enso/interpreter/test/DebuggingEnsoTest.java

@@ -0,0 +1,78 @@
+package org.enso.interpreter.test;
+
+import com.oracle.truffle.api.debug.DebugException;
+import com.oracle.truffle.api.debug.DebugValue;
+import com.oracle.truffle.api.debug.Debugger;
+import com.oracle.truffle.api.debug.SuspendedEvent;
+import java.net.URI;
+import java.nio.file.Paths;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeSet;
+import org.enso.polyglot.RuntimeOptions;
+import org.graalvm.polyglot.Context;
+import org.graalvm.polyglot.Engine;
+import org.graalvm.polyglot.Language;
+import org.graalvm.polyglot.Source;
+import org.junit.Assert;
+import static org.junit.Assert.assertEquals;
+import org.junit.Test;
+
+public class DebuggingEnsoTest {
+  @Test
+  public void evaluation() throws Exception {
+    Engine eng = Engine.newBuilder()
+      .allowExperimentalOptions(true)
+      .option(
+        RuntimeOptions.LANGUAGE_HOME_OVERRIDE,
+        Paths.get("../../distribution/component").toFile().getAbsolutePath()
+      ).build();
+    Context ctx = Context.newBuilder()
+      .engine(eng)
+      .allowIO(true)
+      .build();
+    final Map<String, Language> langs = ctx.getEngine().getLanguages();
+    org.junit.Assert.assertNotNull("Enso found: " + langs, langs.get("enso"));
+
+    final URI facUri = new URI("memory://fac.enso");
+    final Source facSrc = Source.newBuilder("enso", """
+    fac : Number -> Number
+    fac n =
+        facacc : Number -> Number -> Number
+        facacc n accumulator =
+                stop = n <= 1
+                if stop then accumulator else @Tail_Call facacc n-1 n*accumulator
+
+        facacc n 1
+    """, "fac.enso")
+            .uri(facUri)
+            .buildLiteral();
+
+    var module = ctx.eval(facSrc);
+    var facFn = module.invokeMember("eval_expression", "here.fac");
+    final var dbg = Debugger.find(eng);
+    final var values = new TreeSet<Integer>();
+    try (var session = dbg.startSession((event) -> {
+      final DebugValue accumulatorValue = findDebugValue(event, "accumulator");
+      if (accumulatorValue != null) {
+        final int accumulator = accumulatorValue.asInt();
+        values.add(accumulator);
+      }
+      event.getSession().suspendNextExecution();
+    })) {
+      session.suspendNextExecution();
+      var fac5 = facFn.execute(5);
+      Assert.assertEquals("5!", 120, fac5.asInt());
+    }
+    assertEquals("Accumulator gets following values one by one", Set.of(1, 5, 20, 60, 120), values);
+  }
+
+  private static DebugValue findDebugValue(SuspendedEvent event, final String n) throws DebugException {
+    for (var v : event.getTopStackFrame().getScope().getDeclaredValues()) {
+      if (v.getName().contains(n)) {
+        return v;
+      }
+    }
+    return null;
+  }
+}