Language guide.

docs/Contributing.md

@@ -0,0 +1,24 @@
+Contributing
+============
+
+Thank you for considering contributing to Carp.
+
+This document currently focuses on developer contributions but we welcome all kinds of contributions.
+
+
+Community
+---------
+The best place to start is to join the Carp Gitter channel over at
+[https://gitter.im/carp-lang/Carp](https://gitter.im/carp-lang/Carp)
+
+
+License
+-------
+Carp is currently released under the terms of the the ASL 2.0 license.
+
+
+Code
+----
+For tips on how to navigate the codebase please see [DeveloperTips.md](DeveloperTips.md) 
+in this same directory.
+

docs/Install.md

@@ -0,0 +1,6 @@
+# Installation
+
+1. Make sure you have [Stack](https://docs.haskellstack.org/en/stable/README/) installed
+2. Clone this repo
+3. Run ```stack build``` in the root
+4. Stack install will install the Carp command line tool for easy access on your system

docs/LanguageGuide.md

@@ -0,0 +1,139 @@
+## The Language
+
+Carp borrows its looks from Clojure but the runtime semantics are much closer to those of ML or Rust. Here's a sample program:
+
+```clojure
+(defn say-hi [text]
+  (while true
+    (if (< (strlen text) 10)
+      (println &"Too short!")
+      (println text))))
+```
+
+This compiles to the following C program:
+```C
+void say_MINUS_hi(string* text) {
+  bool _55 = true;
+  while (_55) {
+    int _62 = String_count(text);
+    bool _60 = Int__LT_(_62, 10);
+    if (_60) {
+      string _69 = strdup("Too short!");
+      string* _68 = &_69; // ref
+      IO_println(_68);
+    } else {
+      IO_println(text);
+    }
+    _55 = true;
+  }
+}
+```
+
+If-statements are kind of tricky in regards to memory management:
+```clojure
+(defn say-what [text]
+  (let [manage-me (copy text)]
+    (if (< (count text) 10)
+      (copy "Too short")
+      manage-me)))
+```
+
+The 'manage-me' variable is the return value in the second branch, but should get freed if "Too short" is returned.
+The output is a somewhat noisy C program:
+```C
+string say_MINUS_what(string text) {
+    string _78;
+    string* _84 = &text; // ref
+    int _82 = String_count(_84);
+    bool _80 = Int__LT_(_82, 10);
+    if (_80) {
+        string _90 = strdup("Too short");
+        string* _89 = &_90; // ref
+        string _87 = String_copy(_89);
+        String_delete(text);
+        _78 = _87;
+    } else {
+        _78 = text;
+    }
+    return _78;
+}
+```
+
+The most important thing in Carp is to work with arrays of data. Here's an example of how that is supposed to look: (NOT FULLY IMPLEMENTED YET)
+
+```clojure
+(defn weird-sum [nums]
+  (reduce + 0 (map inc (filter even? nums))))
+```
+
+All the array modification functions like 'map', 'filter', etc. use C-style mutation of the array and return the same data structure back afterwards, no allocation or deallocation needed. The lifetime analyzer ("borrow checker" in [Rust](https://www.rust-lang.org) parlance) makes sure that the same data structure isn't used in several places.
+
+To know whether a function takes over the responsibility of freeing some memory (through its args) or generates some new memory that the caller has to handle (through the return value), just look at the type of the function (right now the easiest way to do that is with the ```(env)``` command). If the value is a simple type like String, Vector3, or similar, it means that the memory ownership gets handed over. If it's a reference signature (i.e. ```(Ref String)```), the memory is just temporarily lended out and someone else will make sure it gets deleted. When interoping with existing C code it's often correct to send your data structures to C as refs or pointers (using ```(address <variable>)```), keeping the memory management inside the Carp section of the program.
+
+### Data Literals
+```clojure
+100 ; Int
+3.14f ; Float
+10.0 ; Double
+true ; Bool
+"hello" ; String
+\e ; Char
+[1 2 3] ; (Array Int)
+```
+
+### Dynamic-only Data Literals
+Right now the following data types are only available for manipulation in non-compiled code.
+
+```clojure
+(1 2 3) ; list
+foo ; symbol
+```
+
+### Special Forms
+```clojure
+(def variable-name value)
+(defn function-name (arg1 arg2 ...) (function-body ...))
+(let [var1 expr1, var2 expr2, ...] body)
+(do expr1 expr2 ...)
+(if expression true-branch false-branch)
+(while expression body)
+(ref x) ;; Turns an owned value into an unowned one
+(address x) ;; Takes the memory address of a value, returns a C-style pointer
+(set! variable value)
+(the Int x) ;; explicitly tell the type of a
+```
+
+### Named holes
+```clojure
+(String.append ?w00t "!") ;; Will generate a type error telling you that the type of 'w00t' is String
+```
+
+### Reader macros
+```clojure
+&x ; same as (ref x)
+@x ; same as (copy x)
+```
+
+### Dynamic-only Special Forms
+```
+(import <module>) ;; Make it possible to access members of a module without qualifying them with "ModuleName."
+```
+
+### Structs
+```clojure
+(deftype Vector2 [x Float, y Float])
+
+(let [my-pos (Vector2.init 102.2f 210.3f)]
+  ...)
+
+;; A 'lens' is automatically generated for each member:
+(Vector2.x my-pos) ;; => 102.2f
+(Vector2.set-x my-pos 3.0f) ;; => (Vector2 10.2f 3.0f)
+(Vector2.update-x my-pos inc) ;; => (Vector2 10.2f 4.0f)
+```
+
+### C interop
+```clojure
+(register blah "foo" (Fn (Int Int) String)) ;; will register the function 'foo' that takes two ints and returns a string
+```
+

docs/Libraries.md

@@ -0,0 +1,17 @@
+# Core Libraries
+See [lisp/core.carp](../lisp/core.carp), proper docs are coming soon!
+
+# The C standard library (wrapped)
+See [lisp/builtins.carp](../lisp/builtins.carp)
+
+# OpenGL
+See [lisp/gl.carp](../lisp/gl.carp)
+
+# The '*' macros
+Since the functions in Carp can't accept a variable number of args there are a bunch of helper macros that allows you to circumvent this limitation. Here are some examples:
+
+```clojure
+(str* "This string " "and this string, here's a number " 123 ", etc...")
+(println* "X = " x ", Y = " y)
+(and* true false false true false)
+```

docs/Manual.md

@@ -0,0 +1,36 @@
+## The Compiler
+The Carp language is very tightly integrated with its compiler which itself is written in a dynamic version of Carp (implemented in C). To work on a Carp program you run ```carp``` (first making sure it's in your $PATH, see installation instructions below) which starts the REPL. Everything you want to do to your program can be controlled from here.
+
+For example, to compile a function named 'fib' you enter the following:
+```clojure
+λ> (bake fib)
+```
+
+This results in the compiler analyzing the code form for 'fib' and compiling it to (hopefully very fast) binary code, immediately loading this back into the REPL so that it can be called from there. The resulting C-code, AST and type signature are bound to the three variables 'c', 'ast' and 's', respectively. Inspecting their contents will teach you more about the innards of the Carp language, for sure!
+
+From the REPL you can also inspect your the state of variables, extend the compiler, script the build process of your project, or statically analyze its code. All these operations should be really quick to execute and easy to remember so you can focus on developing your program.
+
+To start the Carp compiler in development mode (which will run its test suite), invoke it like this instead:
+
+```CARP_DEV=1 carp```
+
+### Compiler Variables
+* ```carp-dir``` The root folder of the Carp compiler, should be the same folder as the one where the README.md file resides.
+* ```out-dir``` A string with the name of the folder where build artifacts should be put. Standard value is the 'out' folder in the carp directory.
+* ```exe-out-dir``` Where the exe:s produced by (bake-exe ...) should be placed. Standard value is "./" (working directory)
+* ```echo-signature-after-bake``` If this is true the type signature of freshly baked functions will be printed in the REPL.
+* ```prompt``` The prompt displayed in the repl
+* ```profile-infer-time``` Set to true if you want to know the time it takes to infer the types for each function
+* ```profile-external-compiler-time``` Set to true if you want to know the time it takes to run the external C compiler
+* ```log-unloading-of-dylibs``` Should the compiler log when it unloads dynamic libraries?
+* ```log-deps-when-baking-ast``` Should the compiler log the libraries it links to?
+
+### Special Files
+If a file called ```user.carp``` is placed in the folder ```~/.carp/```, that file will get loaded after the compiler has started. This file is meant for user specific settings that you want in all your projects, like little helper functions and other customizations.
+
+If a file called ```project.carp``` is placed in the folder where you invoke the ```carp``` command this file will get loaded after the compiler has started (and after 'user.carp' has loaded). This files is intended for setting up the build process of this particular project, for example by loading the correct source files, configuring the compiler variables, etc.
+
+### Recovering from errors
+If an error occurs at the REPL, Carp will intercept the error signal and try to recover. Sometimes this does not work (because of memory corruption or similar) and your only option is to restart the process. Quite often it works though, so make sure to try it before resorting to a hard reset.
+
+When working with glfw windows a crash will not close the window, and creating a new one will not work either. To be able to continue, call ```(glfwTerminate)``` first. This will clear everything related to the window and allow you to start anew. A similar process is probably worth working out for other kind or resources that gets lost when a crash happens. Also make sure you fix the problematic code or state that caused the error or you will just crash again immediately.

docs/Research.md

@@ -0,0 +1,83 @@
+https://news.ycombinator.com/item?id=12039268
+
+# C PROGRAMMING, LLVM, AND DEBUGGING
+
+- http://clang.llvm.org/docs/AddressSanitizer.html
+- https://www.iar.com/support/resources/articles/advanced-preprocessor-tips-and-tricks/
+- http://jvns.ca/blog/2016/03/01/a-few-notes-on-the-stack/
+- https://www.recurse.com/blog/7-understanding-c-by-learning-assembly
+- http://msm.runhello.com/p/1003
+- http://stackoverflow.com/questions/2505385/classes-and-static-variables-in-shared-libraries
+- http://www.gameaipro.com/GameAIPro/GameAIPro_Chapter15_Runtime_Compiled_C++_for_Rapid_AI_Development.pdf
+- https://fsharpforfunandprofit.com/posts/fsharp-decompiled/
+- http://kristerw.blogspot.se/2016/05/type-based-aliasing-in-c.html
+
+# OWNERSHIP & BORROWING
+
+- http://blog.piston.rs/2016/01/23/dynamo/
+- http://evincarofautumn.blogspot.se/2016/01/thoughts-on-using-fractional-types-to.html
+- http://andrewbrinker.github.io/blog/2016/03/27/string-types-in-rust/
+- http://www.pipeline.com/~hbaker1/LinearLisp.html
+- http://smallcultfollowing.com/babysteps/blog/2016/04/27/non-lexical-lifetimes-introduction/
+- https://doc.rust-lang.org/book/deref-coercions.html
+- http://ticki.github.io/blog/lambda_crabs_1/
+- https://gankro.github.io/blah/linear-rust/
+
+# MODULES & FUNCTORS
+
+- http://homepages.inf.ed.ac.uk/mfourman/teaching/mlCourse/notes/sml-modules.html
+- https://www.cse.unsw.edu.au/~chak/papers/modules-classes.pdf
+
+# TYPES
+
+- https://lambdacube3d.wordpress.com/2016/03/03/tuples-as-heterogeneous-lists/
+- https://jeltsch.wordpress.com/2016/02/22/generic-programming-in-haskell/
+- http://okmij.org/ftp/ML/generalization.html
+- http://www.cs.cornell.edu/courses/cs312/2005sp/lectures/rec22.asp
+- http://www.cs.cornell.edu/courses/cs3110/2011sp/lectures/lec26-type-inference/type-inference.htm
+- http://www.scheme.com/tspl4/examples.html#./examples:h10
+
+# CSP
+
+- http://reaktor.com/blog/why-csp-matters-ii-how-do-i-know-sync-works/
+
+# GC
+
+- http://prl.ccs.neu.edu/blog/2016/05/24/measuring-gc-latencies-in-haskell-ocaml-racket/
+
+# OTHER LANGUAGES
+
+- https://wingolog.org/archives/2016/02/08/a-lambda-is-not-necessarily-a-closure
+- http://alex-charlton.com/posts/Prototype_to_polish_Making_games_in_CHICKEN_Scheme_with_Hypergiant/
+- http://beautifulracket.com/first-lang.html
+- http://stackoverflow.com/questions/4899113/fixed-point-combinator-for-mutually-recursive-functions/5272086#5272086
+- http://klisp.org
+- http://gliese1337.blogspot.se/2012/04/schrodingers-equation-of-software.html
+- https://github.com/kiselgra/c-mera
+- https://github.com/wolfgangj/bone-lisp/
+- https://github.com/akkartik/mu
+- https://github.com/haskell-lisp/liskell
+- https://github.com/PistonDevelopers/dyon/pull/318
+
+# LANGUAGE DESIGN
+
+- http://www.complang.tuwien.ac.at/kps2015/proceedings/KPS_2015_submission_29.pdf
+- http://www.lihaoyi.com/post/WhatsinaBuildTool.html
+- http://prog21.dadgum.com/136.html
+- https://existentialtype.wordpress.com/2011/03/19/dynamic-languages-are-static-languages/
+
+# GRAPHS
+
+- https://en.wikipedia.org/wiki/Strongly_connected_component
+- https://en.wikipedia.org/wiki/Transitive_closure#In_graph_theory
+
+# GRAPHICS
+
+- https://medium.com/@evanwallace/easy-scalable-text-rendering-on-the-gpu-c3f4d782c5ac#.ajyuy8weu
+- https://pomax.github.io/bezierinfo/
+
+# TO TRY
+
+- let fun ident(x) => x
+- in ident(ident)(2) end
+

docs/Todo.md

@@ -0,0 +1,103 @@
+# The Big 'ref' debacle - alternatives:
+  1. Allow ref:ed value types to be coerced into non-ref:ed types (best solution, if it works)
+  2. A deref function that can remove the ref from primitive types
+  3. A 'map-primitive' that can map a function that takes non-refs as argument 
+     since it's annying to not be able to use functions like 'itos' directly with 'map-copy' 
+     (it requires a fn of type &a -> b)
+
+# Compiler Big Features
+  - Live Reloading (requires threads and bytecode interpreter)
+  - Special handling of POD structs (stack allocated, referenced by pointer)
+  - Compile match statements (could it be a macro?)
+  - Compile modules (when they exist in the dynamic runtime...)
+  - Compile dictionaries (requires hashing function for all types that can be keys)
+  - Lambdas
+  - get / set special forms? Would enable shorter names, and sharing names for members between different structs
+
+# Bytecode
+  - Make bytecode 'match' use labels and gotos instead of recursive calls to eval
+
+# Compiler Small Features
+  - Shorter names for concrete versions of generic functions (don't duplicate types like this: 'atan2_Int_Int')
+  - All types should have capital first letter?
+  - Be able to save concretized struct types for type checking etc
+
+# Compiler Correctness
+  - Variables/functions named the same thing as a struct can override the dylib generated for the struct group
+  - Must unload all concretized structs when the parent struct is redefined
+  - Compiler doesn't catch when a let-binding refers to a variable that's defined later (in the same let binding)
+  - Avoid problems with name shadowing when freeing a local variable (is this possible? disallow shadowing instead?)
+  - Complete type constraints for binops, check for "numeric" types (use a union type of some sort?). Or turn binops into normal funcs?
+  
+# Compiler efficiency / beauty
+  - Avoid creating unique typevars for multiple calls with the same types to a generic function?
+  - Use 'sicp unifier' instead of the current mess
+  - Use 'generic-name' when concretizing generic primops
+  - Rewrite a bunch of functions in the compiler passes using pipe operator and update-in
+  - Speed up some passes by mutating a single variable instead of copying immutable versions around
+  - Use the new key-is-true function instead of has-key? in lots of places
+  - Calls back to the compiler from runtime should be minimized and only require a single call, not two or three like it often is now
+
+# Dynamic Runtime Big Features
+  - Desugar [...] to (array ...) in reader
+  - Macro splicing
+  - Modules 
+  - A Set-type with reader syntax #{}
+  - Instantiate generic functions like '=' for primitive types when calling them
+  - Line numbers for dictionary literals
+  
+# Modules
+  - Name
+  - List of imported modules (with the name used for importation)
+  - List of opened modules
+  - Environment (with all the bindings)
+
+# Dynamic Runtime Small Features
+  - Delete content of global var when resetting from repl
+  - Be able to mark symbols/modules as "frozen" (with meta data) so that they can't be overriden by user
+  - Better error handling and input validation for primops, clean up the C error/assertion macros
+  - ONLY allow [] in parameter list for function definitions
+  - Use size_t where approperiate
+  - Make the reader warn when the text to read is too big (repl.c)
+  - Resetting a global variable pointing to an array can be fooled by using wrong kind of array (anything goes at the moment)
+  
+# Dynamic Runtime Optimization
+
+# Bugs
+  - Don't allow sending compiled functions of wrong type to ffi functions (check their types with 'signature')
+  - assert-eq shows wrong result when the assertion fails? (in ffi situations, the wrong type is produced and compared to something else)
+  
+# Sanity checks
+  - Ensure correctness of GC (run at every step)
+  - Don't leak values returned from calling ffi functions via non-compiled code
+  
+# Lisp Core Libs
+  - 'import' function that searches paths for carp files
+  - shuffle (for lists)
+  - Conversions between a list of pairs and dictionaries
+  - 'for' macro with multiple bindings (i, j, etc...)
+
+# Maybes
+  - Compile keywords?
+  - Add void constraints for (do ...) statements ?
+  - Add proper no-op :node for () ?
+  - Polymorphic math operators?
+  - Matching/destructuring in let statements and function arguments too?
+  - Reading of dotted pairs?
+  - Not possible to write an 'eat-void' function: (register-builtin "eat_void" '(:void) :void), need a proper unit type for that
+  - :when clauses in match?
+  - Use a more generalized method for generating 'str' function when inspecting ptr:s at the REPL (some kind of "hook" system)
+  - Ownership tracking to enable returning refs from functions (it's forbidden at the moment)
+  - Reorder arguments to "set"/"update"-lens to make them less problematic for borrow checking (the main structure is given away to the first argument) ?
+  - Use modules to solve problem of using same name for members in different structs?
+  - Create a carp_bool type with defined size
+
+# Niceties
+  - Built in tutorial for the language
+  - Built in manual
+
+# Gotchas
+  - Unloading of function/dylib doesn't work after another function has linked to it during its compilation.
+  - Variable shadowing doesn't work properly when referencing itself
+  - Size of bool is undefined
+  - Must mark global variables on windows as __declspec(dllimport) when using them and __declspec(dllexport) when providing for others

examples/temp.carp

@@ -27,3 +27,14 @@
 
 (defn main []
   (f))
+
+(defn say-hi [text]
+  (while true
+    (if (< (count text) 10)
+      (println &"Too short!")
+      (println text))))
+
+(defn say-what [text]
+  (if (< (count &text) 10)
+    (copy &"Too short")
+    text))