mal/docs/step_notes.txt

Step Notes:

- step0_repl
    - prompt, input, READ, EVAL, PRINT, output
    - readline module
        - display prompt, read line of input

- use native eval in EVAL if available

- libedit/GNU readline:
    - use existing lib, wrap shell call or implement
    - load history file on first call
    - add non-blank lines to history
    - append to history file

- step1_read_print
    - types module:
        - add boxed types if no language equivalent:
            - nil, true, false, symbol, integer, string, list
    - reader module:
        - stateful reader object
            - alternative: mutate token list
        - tokenize (if regex available)
            - standard regex pattern: "/[\s,]*(~@|[\[\]{}()'`~^@]|\"(?:\\.|[^\\\"])*\"|;.*|[^\s\[\]{}('\"`,;)]*)/"
        - read_str
            - read_form(new Reader(tokenize(str)))
        - read_form
            - detect errors
            - call read_list or read_atom
        - read_list
            - read_form until ')'
            - return array (boxed)
        - read_atom (not atom type)
            - return scalar boxed type:
                - nil, true, false, symbol, integer, string
    - printer module:
        - _pr_str:
            - stringify boxed types to their Mal representations
            - list/array is recursive
    - repl loop
        - catch errors, print them and continue
        - impls without exception handling will need to have a global
          variable with checks for it at the beginning of critical
          code sections

- comments

- vectors
    - Basically: two array types that retain their boxed types, can be
      challenging depending on the language (e.g. JS, PHP: no clean
      way to derive new array types).
    - types module:
        - add vector boxed type
            - derived from array if possible
        - pr_str:
            - vector is recursive
        - sequential?
    - reader module:
        - read_vector:
            - re-use read_list but with different constructor, delims

- hash-maps
    - reader module:
        - re-use read_list function and apply that using hash-map
          constructor
    - types module:
        - pr_str addition
        - hash-map, map?, assoc, dissoc, get, contains?, keys,
          vals (probably assoc! and dissoc! for internal)
    - eval_map: eval the keys and values of hash_maps
    - EVAL:
        - if hash_map, call eval_map on it

- step2_eval
    - types module:
        - symbol?, list? (if no simple idiomatic impl type check)
        - first, rest, nth on list
    - eval_ast:
        - if symbol, return value of looking up in env
        - if list, eval each item, return new list
        - otherwise, just return unchanged ast
    - EVAL/apply:
        - if not a list, call eval_ast
        - otherwise, apply first item to eval_ast of (rest ast)
    - repl_env as simple one level assoc. array (or hash_map)
        - store function as hash_map value

- vectors
    - eval each item, return new vector

- hash-maps
    - eval each value, return new hash_map

- step3_env
    - types module:
        - may need function type if HashMap is strongly typed (e.g. Java)
    - env type:
        - find, set, get (no binds/exprs in constructor yet)
    - EVAL/apply:
        - def! - mutate current environment
        - let* - create new environment with bindings

- step4_if_fn_do
    - types module:
        - function type if no closures in impl language
        - _equal_Q function (recursive)
    - reader module
        - string unescaping
    - printer module
        - print_readably option for pr_str
        - add function printing to pr_str
        - string escaping in pr_str
    - core module (export via core_ns):
        - export equal_Q from types as =
        - move arith operations here
        - add arith comparison functions
        - pr_str, str, prn, println
        - list, list?, count, empty?
    - env module:
        - add binds/exprs handling to Env constructor with variable arity
    - EVAL:
        - do:
        - if:
        - fn*:
            - simple if language supports closures
            - otherwise needs a way of representing functions that can
              have associated metadata
    - define "not" using REP/RE

- step5_tco
    - types module:
        - mal function type:
            - stores: func, exp, env, params
            - func is EVAL in native mal case, otherwise reference to
              platform function
            - if metadata support, then store exp, env, params as
              metadata
    - printer
        - add printing of mal function type
    - EVAL:
        - while loop around whole thing
        - cases where we directly return result of EVAL, instead set
          ast and env to what would be put in the EVAL, then loop.
            - do, if, "apply"
            - for "apply" case, set env to new Env based on properties
              on the function

- step6_file
    - core module:
        - read-string, slurp functions
    - define eval and load-file functions
    - set *ARGV*
    - if files on command line, use load-file to run first argument
      using rest as arguments

- step7_quote
    - add is_pair and quasiquote functions
        - rewrite ast using cons/concat functions
        - if vectors, use sequential? instead of list? in is_pair
    - EVAL:
        - add 'quote', 'quasiquote' cases
    - core module:
        - add cons and concat functions
    - reader module:
        - add reader macros to read_form for quote, unquote,
          splice-unquote and quasiquote

- step8_macros
    - types
        - capability to store ismacro property in function
    - core module:
        - add first, rest, nth functions
    - add is_macro_call and macroexpand
        - recursively macroexpand lists
        - if applying a macro function, run it on the ast first before
          continuing
    - call macroexpand apply in EVAL before apply
    - EVAL:
        - add 'defmacro!' and 'macroexpand'
        - set ismacro property on function

- step9_interop
    - convert returned data to mal data
        - recursive, similar to pr_str

- stepA_more
    - core module:
        - throw function
        - apply, map functions: should not directly call EVAL, which
          requires the function object to be runnable
        - readline
        - nil?, true?, false? 
    - EVAL:
        - try*/catch*: for normal exceptions, extracts string
          otherwise extracts full value
    - set and print *host-language*
    - define cond and or macros using REP/RE

- Extra defintions needed for self-hosting
    - core module:
        - symbol?, sequential? (if not already)
        - vector, vector?


- Other misc:
    - conj function

- atoms
    - reader module:
        - @a reader macro -> (deref a)
    - core module:
        - pr_str case
        - atom type, atom, atom?, deref, reset!, swap!

- metadata
    - reader module:
        - ^ reader macro reads ^meta obj -> (with-meta obj meta)
    - types module:
        - support meta property on collections: lists, vectors,
          hash-maps, functions, atoms
            - clone/copy of collections
    - core module:
        - add with-meta, meta functions