roc/editor/editor-ideas.md

(For background, this talk has an overview of the design goals for the editor.)

Editor Ideas

Here are some ideas and interesting resources for the editor. Feel free to make a PR to add more!

Sources of Potential Inspiration

These are potentially inspirational resources for the editor's design.

Nice collection of research on innovative editors, link.

Package-specific editor integrations

(Or possibly module-specific integrations, type-specific integrations, etc.)

• What FP can learn from Smalltalk by Aditya Siram
• Moldable development by Tudor Gîrba
• Unity game engine
  • Scripts can expose values as text inputs, sliders, checkboxes, etc or even generate custom graphical inputs
  • Drag-n-drop game objects and component into script interfaces
• How to Visualize Data Structures in VS Code

Live Interactivity

• Up and Down the Ladder of Abstraction by Bret Victor
• 7 Bret Victor talks
• Against the Current by Chris Granger
• Sketch-n-Sketch: Interactive SVG Programming with Direct Manipulation by Ravi Chugh
• Xi modern text editor with concurrent editing (related to Druid)
• Self programming language
• Primitive code exploration in Virtual Reality
• Luna language for interactive data processing and visualization

Debugging

• VS code debug visualization
• Algorithm visualization for javascript
• godbolt.org Compiler Explorer
• whitebox debug visualization
• Hest tool for making highly interactive simulations.
• Say you have a failing test that used to work, it would be very valuable to see all code that was changed that was used only by that test. e.g. you have a test calculate_sum_test that only uses the function add, when the test fails you should be able to see a diff showing only what changed for the function add. It would also be great to have a diff of expression values Bret Victor style. An ambitious project would be to suggest or automatically try fixes based on these diffs.
• I think it could be possible to create a minimal reproduction of a program / block of code / code used by a single test. So for a failing unit test I would expect it to extract imports, the platform, types and functions that are necessary to run only that unit test and put them in a standalone roc project. This would be useful for sharing bugs with library+application authors and colleagues, for profiling or debugging with all "clutter" removed.

Cool regular editors

• Helix modal (terminal, for now) editor in rust. Good UX.

Structured Editing

• Greenfoot

• Deuce (videos on the right) by Ravi Chugh and others

• Fructure: A Structured Editing Engine in Racket by Andrew Blinn

• Hazel: A Live FP Environment with Typed Holes by Cyrus Omar

• Dark Demo by Ellen Chisa

• Introduction to JetBrains MPS by Kolja Dummann

• Eve

  • code editor as prose writer
  • live preview
  • possible inspiration for live interactivity as well

• Unreal Engine 4

  • Blueprints visual scripting (not suggesting visual scripting for Roc)

• Live Programing by [Microsoft Research] it contains many interesting research papers.

• Math Inspector, github

• Lamdu live functional programming.

• Sourcetrail nice tree-like source explorer.

• Unisonweb, definition based editor as opposed to file based.

Voice Interaction Related

• We should label as many things as possible and expose jumps to those labels as shortkeys.
• Update without user interaction. e.g. autosave.
• Could be efficient way to communicate with smart assistant.
• You don't have to remember complex keyboard shortcuts if you can describe actions to execute them. Examples:
  • Add latest datetime package to dependencies.
  • Generate unit test for this function.
  • Show edit history for this function.
  • Adjusting settings: switch to light theme, increase font size...
• Use (context specific) voice command state machine to assist Machine Learning voice recognition model.
• Nice special use case: using voice to code while on treadmill desk.

Inspiration

• Voice control and eye tracking with Talon
• Seminar about programming by voice
• Talon voice commands in elm
• Mozilla DeepSpeech model runs fast, works pretty well for actions but would need additional training for code input. Possible to reuse Mozilla common voice for creating more "spoken code" data.

Beginner-focused Features

• Show Roc cheat sheet on start-up.
• Plugin that translates short pieces of code from another programming language to Roc. Relevant research. Someone who only knows the R language could get started with Roc with less friction if they could quickly define a list R style (lst <- c(1,2,3)) and get it translated to Roc.
• Being able to asses or ask the user for the amount of experience they have with Roc would be a valuable feature for recommending plugins, editor tips, recommending tutorials, automated error search (e.g searching common beginner errors first), ... .

Productivity features

• When refactoring;
  • Cutting and pasting code to a new file should automatically add imports to the new file and delete them from the old file.
  • Ability to link e.g. variable name in comments to actual variable name. Comment is automatically updated when variable name is changed.
  • When updating dependencies with breaking changes; show similar diffs from github projects that have succesfully updated that dependency.
  • AST backed renaming, changing variable/function/type name should change it all over the codebase.
• Automatically create all "arms" when pattern matching after entering when var is based on the type.
  • All when ... is should be updated if the type is changed, e.g. adding Indigo to the Color type should add an arm everywhere where when color is is used.
• When a function is called like foo(false), the name of the boolean argument should be shown automatically; foo(is_active:false). This should be done for booleans and numbers.
• Suggest automatically creating a function if the compiler says it does not exist.
• Integrated search:
  • Searchbar for examples/docs. With permission search strings could be shared with the platform/package authors so they know exactly what their users are struggling with.
• Show productivity/feature tips on startup. Show link to page with all tips. Allow not seeing tips next time.
• Search friendly editor docs inside the editor. Offer to send search string to Roc maintainers when no results, or if no results were clicked.
• File history timeline view. Show timeline with commits that changed this file, the number of lines added and deleted as well as which user made the changes. Arrow navigation should allow you to quickly view different versions of the file.
• Suggested quick fixes should be directly visible and clickable. Not like in vs code where you put the caret on an error until a lightbulb appears in the margin which you have to click for the fixes to apppear, after which you click to apply the fix you want :( .
• Regex-like find and substitution based on plain english description and example (replacement). i.e. replace all [ between double quotes with {. Inspiration.
• Show productivity tips based on behavior. i.e. if the user is scrolling through the error bar and clicking on the next error several times, show a tip with "go to next error" shortcut.
• Command to "benchmark this function" or "benchmark this test" with flamegraph and execution time per line.
• Instead of going to definition and having to navigate back and forth between files, show an editable view inside the current file. See this video
• When encountering an unexpected error in the user's program we show a button at the bottom to start an automated search on this error. The search would:
  • look for similar errors in github issues of the relevant libraries
  • search stackoverflow questions
  • search a local history of previously encountered errors and fixes
  • search through a database of our zullip questions
  • ...

Autocomplete

• Use more space for autocomplete options:
  • Multiple columns. Columns could have different sources, i.e. middle column suggests based on current folder, left column on whole project, right column on github.
  • show cell with completion + auto import suggestion
• Webcam based eye tracking for quick selection.
• Machine Learning:
  • GPT-3 can generate correct python functions based on a comment describing the functionality, video here. It's possible that training a model using ast's may lead to better results than text based models.
• Current autocomplete lacks flow, moving through suggestions with arrows is slow. Being able to code by weaving together autocomplete suggestions layed out in rows using eye tracking, that could flow.

Productivity Inspiration

• Kite AI autocomplete and doc viewer.
• Tabnine AI autocomplete.
• Codota AI autocomplete and example searching.
• Aroma showing examples similar to current code.
• MISM neural network based code similarity scoring.
• Inquisitive code editor Interactive bug detection with doc+test generation.
• NextJournal Discoverable commands and shortcuts.

Non-Code Related Inspiration

• Scrivner writing app for novelists, screenwriters, and more
• Word processors (Word, Google Docs, etc)
  • Comments that are parallel to the text of the document.
  • Comments can act as discussions and not just statements.
  • Easy tooling around adding tables and other stylised text
• Excel and Google Sheets
  • Not sure, maybe something they do well that we (code editors) could learn from

Machine Learning Ideas

• Ability to record all changes to abstract syntax tree with user permission.
  • I think it is possible to create powerful automatic error resolution by having a dataset available of ast's with a specific error and the subsequent transformation that fixed the error.
  • Users with large private code bases could (re)train a publicly available error recovery model to experience benefits without having to share their code.
  • It could be useful to a user who is creating a function to show them the most similar function (type signature, name, comment) in a public+their private database. Say I was using a web framework and I just created a function that has a multipart form as argument, it would be great to have an example instantly available.
    • A simpler start for this idea without user data gathering: how the user a code snippet that is most similar to what they are currently writing. Snippets can be aggregated from examples, tests, docstrings at zero cost to the package/platform authors.
    • See codata for inspiration on a snippet/example finder.
• Fuzzy natural language based setting adjustment in search bar or with voice input: increase font size, enable autosave, switch to light theme...
• Detect deviation of best practices, example case: alert developer when they are defining a color inline (rgb(30,30,30)) while all colors have been previously imported from a single file. See also Codota.
• It would be valuable to record the user's interactions with the editor when debugging as well as the AST. On enough data we could train a model to perform a bunch of debugging steps and show values of the most important variables in relation to the bug. Having assistance in finding the exact code that causes the problem could be super valuable. There could be sensitive data, so it should only be recorded and or shared for open source codebases with explicit user permission.

Testing

• From Google Docs' comments, adding tests in a similar manner, where they exists in the same "document" but parallel to the code being written
  • Makes sense for unit tests, keeps the test close to the source
  • Doesn't necessarily make sense for integration or e2e testing
  • Maybe easier to manually trigger a test related to exactly what code you're writing
• Ability to generate unit tests for a selected function in context menu
  • A table should appear to enter input and expected output pairs quickly
• Ability to "record" unit tests
  • Select a function to record.
  • Do a normal run, and save the input and output of the selected function.
  • Generate a unit test with that input-output pair

Inspiration

• Haskell language server plugin evaluate code in comments, to test and document functions and to quickly evaluate small expressions.

Documentation

• Ability to see module as it would be presented on a package website.
  • Modern editors may guide developers to the source code too easily. The API and documentation are meant to interface with humans.

General Thoughts/Ideas

Thoughts and ideas possibly taken from above inspirations or separate.

• ACCESSIBILITY === EMPATHY

  • Visual Imapirments No Animation is most benign form of cognitive disabity but really important base line of people with tense nerve system. Insensitivity to certain or all colors. Need of highcontrast Or Everything Magnified for me with no glasses. Or Total blindness where we need to trough sound to communicate to the user Screen readers read trees of labeled elements. Each platform has different apis, but I think they are horrible. Just close your eyes and imagine listening to screen reader all day while you are using this majectic machines called computers. But blind people walk with a tool and they can react much better to sound/space relations than full on visal majority does. They are acute to sound as a spatial hint. And a hand for most of them is a very sensitive tool that can make sounds in space. Imagine if everytime for the user doesnt want to rely on shining rendered pixels on the screen for a feedback from machine, we make a accoustic room simulation, where with moving the "stick", either with mouse or with key arrows, we bump into one of the objects and that produces certain contextually appropriate sound (clean)ding

    On the each level of abstraction they can make sounds more deeper, so then when you type letters you feel like you are playing with the sand (soft)shh. We would need help from some sound engeneer about it, but imagine moving down, which can be voice trigered command for motion impaired, you hear (soft)pup and the name of the module, and then you have options and commands appropriate for the module, they could map to those basic 4 buttons that we trained user on, and he would shortcut all the soft talk with click of a button. Think of the satisfaction when you can skip the dialog of the game and get straight into action. (X) Open functions! each function would make a sound and say its name, unless you press search and start searching for a specific function inside module, if you want one you select or move to next.

  • Motor impariments [rant]BACKS OF CODERS ARE NOT HEALTHY! We need to change that![/neverstop] Too much mouse waving and sitting for too long is bad for humans. Keyboard is basic accessability tool but Keyboard is also optional, some people have too shaky hands even for keyboard. They rely on eye tracking to move mouse cursor arond. If we employ some voice recognition functions we could make same interface as we could do for consoles where 4+2 buttons and directional pad would suffice. That is 10 phrases that need to be pulled trough as many possible translations so people don't have to pretend that they are from Maine or Texas so they get voice recognition to work. Believe me I was there with Apple's Siri :D That is why we have 10 phrases for movement and management and most basic syntax.

• Nice backtraces that highlight important information

• Ability to show import connection within project visually

  • This could be done by drawing connections between files or functions in the tree view. This would make it easier for people to get their bearings in new big projects.

• Connections could also be drawn between functions that call each other in the tree view. The connections could be animated to show the execution flow of the program.

• Ability to inline statements contained in called functions into the callee function for debugging.

  • The value of expressions can be shown at the end of the line like in the Inventing on Principle talk
  • This would give a clear overview of the execution and should make it easy to pinpoint the line where the bug originates.
  • That specific line can then be right clicked to go to the actual function.
  • Having to jump around between different functions and files is unnecessary and makes it difficult to see the forest through the trees.

• "Error mode" where the editor jumps you to the next error

  • Similar in theory to diff tools that jump you to the next merge conflict

• dependency recommendation