Carp/docs/LanguageGuide.md

## The Language

### Introduction
Carp borrows its looks from Clojure but the runtime semantics are much closer to those of ML or Rust.
Types are inferred but can be annoted for readability using the ```the``` keyword (see below).

Memory management is handled by static analysis, a value is owned by the function where it was created.
When a value is returned or passed to another function the initial function will give up ownership of it
and any subsequent use will lead to a compiler error. To temporarily lend a value to another function
(for example to print it) a reference must be created, using the ```ref``` special form (or the ```&``` reader macro).

To learn more about the details of memory management, check out [Memory.md](https://github.com/carp-lang/Carp/blob/master/docs/Memory.md)

### Comments
```clojure
;; Comments begin with a semicolon and continue until the end of the line.
```

### Data Literals
```clojure
100     ;; Int
1500l   ;; Long
3.14f   ;; Float
10.0    ;; Double
true    ;; Bool
"hello" ;; &String
\#"hello" ;; &Pattern
\e      ;; Char
[1 2 3] ;; (Array Int)
{1 1.0 2 2.0} ;; (Map Int Double)
```

### Type Literals
```clojure
t ;; Type variables begin with a lowercase letter
Int
Long
Float
Double
Bool
String
Pattern
Char
(Array t)
(Map <key-type> <value-type>)
(Fn [<arg-type1> <arg-type2> ...] <return-type>) ;; Function type
```

### Dynamic-only Data Literals
Right now the following data types are only available for manipulation in non-compiled code.

```clojure
(1 2 3) ; list
foo ; symbol
```

### Defining things
```clojure
(defn function-name [<arg1> <arg2> ...] <body>) ;; Define a function (will be compiled, can't be called at the REPL)
(definterface interface-name (Fn [<t1> <t2>] <return>)) ;; Define a generic function that can have multiple implementations
(def variable-name value) ;; Define a global variable (only handles primitive constants for the moment)
(defmacro <name> [<arg1> <arg2> ...] <macro-body>) ;; Define a macro, its argument will not be evaluated when called
(defdynamic <name> [<arg1> <arg2> ...] <function-body>) ;; A function that can only be used at the REPL or during compilation
(defmodule <name> <definition1> <definition2> ...) ;; The main way to organize your program into smaller parts
```

### Special Forms
The following forms can be used in Carp source code and will be compiled to C after type checking
and other static analysis. The first three of them are also available in dynamic functions.

```clojure
(let [<var1> <expr1> <var2> <expr2> ...] <body>) ;; Create local bindings
(do <expr1> <expr2> ... <return-expression>) ;; Perform side-effecting functions, then return a value
(if <expression> <true-branch> <false-branch>) ;; Branching
(while <expression> <body>) ;; Loop until expression is false
(ref <expression>) ;; Borrow an owned value
(address <expression>) ;; Takes the memory address of a value, returns a C-style pointer
(set! <variable> <expression>) ;; Mutate a variable
(the <type> <expression>) ;; Explicitly declare the type of an expression
```

Here's an example of how to use the `the` form to make an identity function that only accepts Integers:

```clojure
(defn f [x]
  (the Int x))
```

### Reader Macros
```clojure
&x ;; same as (ref x)
@x ;; same as (copy x)
```

### Named Holes
When using a statically typed language like Carp it can sometimes be hard to know what value should
be used at a specific point in your program. In such cases the concept of 'holes' can be useful. Just
add a hole in your source code and reload (":r") to let the Carp compiler figure out what type goes there.

```clojure
(StringCopy.append ?w00t @"!") ;; Will generate a type error telling you that the type of '?w00t' is String
```

### Special forms during evaluation of dynamic code
```clojure
(quote <expression>) ;; Avoid further evaluation of the expression
(and) (or) (not) ;; Logical operators
```

### Dynamic functions
These can only be used at the REPL and during macro evaluation. Here's a subset with some of the most commonly used ones:

```clojure
(car <collection>) ;; Return the first element of a list or array
(cdr <collection>) ;; Return all but the first element of a list or array
(cons <expr> <list>) ;; Add the value of <expr> as the first element the <list>
(cons-last <expr> <list>) ;; Add the value of <expr> as the last element the <list>
(list <expr1> <expr2> ...) ;; Create a list from a series of evaluated expressions
(array <expr1> <expr2> ...) ;; Create an array from a series of evaluated expressions
```

To see all functions available in the `Dynamic` module, enter `(info Dynamic)` at the REPL.

### Modules and Name Lookup
Functions and variables can be stored in modules which are named and can be nested. To use a symbol inside a module
you need to qualify it with the module name, like this: ```Float.cos```.

*Using* a module makes it possible to access its members without qualifying them:

```clojure
(use Float)

(defn f []
  (cos 3.2f))
```

If there are several used modules that contain symbols with the same name, the type inferer will try to figure
out which one of the symbols you really mean (based on the types in your code). If it can't, it will display an error.
For example, both the module ```String``` and ```Array``` contain a function named 'length'. In the following code it's
possible to see that it's the array version that is needed, and that one will be called:

```clojure
(use String)
(use Array)

(defn f []
  (length [1 2 3 4 5]))
```

In the following example it's not possible to figure out which type is intended:
```clojure
(use String)
(use Array)

(defn f [x]
  (length x))
```

Specifying the type solves this error:
```clojure
(use String)
(use Array)

(defn f [x]
  (String.length x))
```

### Structs
```clojure
(deftype Vector2 [x Int, y Int])

(let [my-pos (Vector2.init 10 20)]
  ...)

;; A 'lens' is automatically generated for each member:
(Vector2.x my-pos) ;; => 10
(Vector2.set-x my-pos 30) ;; => (Vector2 30 20)
(Vector2.update-x my-pos inc) ;; => (Vector2 11 20)
```

### C Interop
```clojure
(system-include "math.h") ;; compiles to #include <math.h>
(local-include "math.h") ;; compiles to #include "math.h"

(register blah (Fn [Int Int] String)) ;; Will register the function 'blah' that takes two Int:s and returns a String
(register pi Double) ;; Will register the global variable 'pi' of type Double

(register blah (Fn [Int Int] String) "exit") ;; Will register the function 'blah' but use the name 'exit' in the emitted C code.

(register-type Apple) ;; Register an opaque C type
(register-type Banana [price Double, size Int]) ;; Register an external C-structs, this will generate getters, setters and updaters.
```

### Patterns

Patterns are similar to, but not the same as, Regular Expressions. They were
derived from [Lua](http://lua-users.org/wiki/PatternsTutorial), and are useful
whenever you want to find something within or extract something from strings.

They are simpler than Regular Expressions, as they do not provide alternation.
Nonetheless, they are often very useful and, because they are simpler, also
faster and more predictable.

Here is a little overview of the API:

```clojure
; you can initialize a pattern with a literal or create one from a string
#"[a-z]"
(Pattern.init "[a-z]")

; you can also get a string back from it
(str #"[a-z]")
(prn #"[a-z]")

; you can find things in strings by index
(Pattern.find #"[a-z]" "1234a") ; => 4
(Pattern.find #"[a-z]" "1234")  ; => -1

; also multiple things at once!
(Pattern.find-all #"[a-z]" "1234a b") ; => [4 6]

; matches? checks whether a string matches a pattern
(Pattern.matches? #"(\d+) (\d+)" "  12 13") ; => true

; match returns all match groups of the first match
(Pattern.match #"(\d+) (\d+)" "  12 13") ; => ["12" "13"]

; match-str returns the whole string of the first match
(Pattern.match #"(\d+) (\d+)" "  12 13") ; => "12 13"

; global-match gets all match groups of all matches
(Pattern.global-match #"(\d+) (\d+)" "  12 13 14 15") ; => [["12" "13"] ["14" "15"]]

; substitute helps you replace patterns in a string n times
(Pattern.substitute #"sub-me" "sub-me sub-me sub-me" "replaced" 1) ; => "replaced sub-me sub-me"

; if you want to replace every occurrence, use -1
(Pattern.substitute #"sub-me" "sub-me sub-me sub-me" "replaced" -1) ; => "replaced replaced replaced"
```
Language guide. 2017-06-26 12:45:18 +03:00			`## The Language`

More explanation. 2017-06-26 16:58:23 +03:00			`### Introduction`
Improved language guide. 2017-06-26 16:50:51 +03:00			`Carp borrows its looks from Clojure but the runtime semantics are much closer to those of ML or Rust.`
			Types are inferred but can be annoted for readability using the ```the``` keyword (see below).
Language guide. 2017-06-26 12:45:18 +03:00
More explanation. 2017-06-26 16:58:23 +03:00			`Memory management is handled by static analysis, a value is owned by the function where it was created.`
			`When a value is returned or passed to another function the initial function will give up ownership of it`
			`and any subsequent use will lead to a compiler error. To temporarily lend a value to another function`
English. 2019-03-07 16:14:43 +03:00			(for example to print it) a reference must be created, using the ```ref``` special form (or the ```&``` reader macro).
More explanation. 2017-06-26 16:58:23 +03:00
Link. 2017-10-17 23:48:02 +03:00			`To learn more about the details of memory management, check out [Memory.md](https://github.com/carp-lang/Carp/blob/master/docs/Memory.md)`
Link to Memory.md 2017-10-17 23:47:21 +03:00
More guide. 2017-06-26 17:09:12 +03:00			`### Comments`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Fix. 2017-06-26 17:14:10 +03:00			`;; Comments begin with a semicolon and continue until the end of the line.`
More guide. 2017-06-26 17:09:12 +03:00			```

Language guide. 2017-06-26 12:45:18 +03:00			`### Data Literals`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
More guide. 2017-06-26 17:09:12 +03:00			`100 ;; Int`
docs: updated language guide 2017-12-18 19:27:51 +03:00			`1500l ;; Long`
More guide. 2017-06-26 17:09:12 +03:00			`3.14f ;; Float`
			`10.0 ;; Double`
			`true ;; Bool`
Correction about &String. 2017-11-28 10:02:11 +03:00			`"hello" ;; &String`
core: rename regex to pattern, as @RyanSquared suggested in #192 2018-03-02 17:29:29 +03:00			`\#"hello" ;; &Pattern`
More guide. 2017-06-26 17:09:12 +03:00			`\e ;; Char`
			`[1 2 3] ;; (Array Int)`
docs: add map to std types 2018-08-30 22:21:45 +03:00			`{1 1.0 2 2.0} ;; (Map Int Double)`
Language guide. 2017-06-26 12:45:18 +03:00			```

Types. 2017-06-26 17:12:41 +03:00			`### Type Literals`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Whitespace. 2017-06-26 17:13:24 +03:00			`t ;; Type variables begin with a lowercase letter`
Types. 2017-06-26 17:12:41 +03:00			`Int`
docs: updated language guide 2017-12-18 19:27:51 +03:00			`Long`
Types. 2017-06-26 17:12:41 +03:00			`Float`
			`Double`
			`Bool`
			`String`
core: rename regex to pattern, as @RyanSquared suggested in #192 2018-03-02 17:29:29 +03:00			`Pattern`
Types. 2017-06-26 17:12:41 +03:00			`Char`
			`(Array t)`
docs: add map to std types 2018-08-30 22:21:45 +03:00			`(Map <key-type> <value-type>)`
Types. 2017-06-26 17:12:41 +03:00			`(Fn [<arg-type1> <arg-type2> ...] <return-type>) ;; Function type`
			```

Language guide. 2017-06-26 12:45:18 +03:00			`### Dynamic-only Data Literals`
			`Right now the following data types are only available for manipulation in non-compiled code.`

language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Language guide. 2017-06-26 12:45:18 +03:00			`(1 2 3) ; list`
			`foo ; symbol`
			```

Reorganised the language guide a bit. 2017-12-22 17:09:53 +03:00			`### Defining things`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Reorganised the language guide a bit. 2017-12-22 17:09:53 +03:00			`(defn function-name [<arg1> <arg2> ...] <body>) ;; Define a function (will be compiled, can't be called at the REPL)`
			`(definterface interface-name (Fn [<t1> <t2>] <return>)) ;; Define a generic function that can have multiple implementations`
			`(def variable-name value) ;; Define a global variable (only handles primitive constants for the moment)`
			`(defmacro <name> [<arg1> <arg2> ...] <macro-body>) ;; Define a macro, its argument will not be evaluated when called`
			`(defdynamic <name> [<arg1> <arg2> ...] <function-body>) ;; A function that can only be used at the REPL or during compilation`
			`(defmodule <name> <definition1> <definition2> ...) ;; The main way to organize your program into smaller parts`
			```

Language guide. 2017-06-26 12:45:18 +03:00			`### Special Forms`
Correction about &String. 2017-11-28 10:02:11 +03:00			`The following forms can be used in Carp source code and will be compiled to C after type checking`
Reorganised the language guide a bit. 2017-12-22 17:09:53 +03:00			`and other static analysis. The first three of them are also available in dynamic functions.`
Disclaimer. 2017-06-26 17:17:25 +03:00
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Reorganised the language guide a bit. 2017-12-22 17:09:53 +03:00			`(let [<var1> <expr1> <var2> <expr2> ...] <body>) ;; Create local bindings`
More guide. 2017-06-26 17:09:12 +03:00			`(do <expr1> <expr2> ... <return-expression>) ;; Perform side-effecting functions, then return a value`
			`(if <expression> <true-branch> <false-branch>) ;; Branching`
			`(while <expression> <body>) ;; Loop until expression is false`
			`(ref <expression>) ;; Borrow an owned value`
Improved language guide. 2017-06-26 16:50:51 +03:00			`(address <expression>) ;; Takes the memory address of a value, returns a C-style pointer`
			`(set! <variable> <expression>) ;; Mutate a variable`
Example of ´the´ 2019-03-07 16:16:31 +03:00			`(the <type> <expression>) ;; Explicitly declare the type of an expression`
			```

			Here's an example of how to use the `the` form to make an identity function that only accepts Integers:

			```clojure
			`(defn f [x]`
			`(the Int x))`
Language guide. 2017-06-26 12:45:18 +03:00			```

More explanation. 2017-06-26 16:58:23 +03:00			`### Reader Macros`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
More guide. 2017-06-26 17:09:12 +03:00			`&x ;; same as (ref x)`
			`@x ;; same as (copy x)`
More explanation. 2017-06-26 16:58:23 +03:00			```

			`### Named Holes`
Improved language guide. 2017-06-26 16:50:51 +03:00			`When using a statically typed language like Carp it can sometimes be hard to know what value should`
			`be used at a specific point in your program. In such cases the concept of 'holes' can be useful. Just`
			`add a hole in your source code and reload (":r") to let the Carp compiler figure out what type goes there.`

language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
String.append is now by reference, StringCopy.append is by copy/linear Adding memory leak tests to String.append and StringCopy.append Issue #94 2018-05-20 07:04:47 +03:00			`(StringCopy.append ?w00t @"!") ;; Will generate a type error telling you that the type of '?w00t' is String`
Language guide. 2017-06-26 12:45:18 +03:00			```

Docs. 2017-12-23 08:38:45 +03:00			`### Special forms during evaluation of dynamic code`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
More guide. 2017-06-26 17:09:12 +03:00			`(quote <expression>) ;; Avoid further evaluation of the expression`
Docs. 2017-12-23 08:38:45 +03:00			`(and) (or) (not) ;; Logical operators`
			```

			`### Dynamic functions`
			`These can only be used at the REPL and during macro evaluation. Here's a subset with some of the most commonly used ones:`

language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
More guide. 2017-06-26 17:09:12 +03:00			`(car <collection>) ;; Return the first element of a list or array`
			`(cdr <collection>) ;; Return all but the first element of a list or array`
			`(cons <expr> <list>) ;; Add the value of <expr> as the first element the <list>`
docs: updated language guide 2017-12-18 19:27:51 +03:00			`(cons-last <expr> <list>) ;; Add the value of <expr> as the last element the <list>`
More guide. 2017-06-26 17:09:12 +03:00			`(list <expr1> <expr2> ...) ;; Create a list from a series of evaluated expressions`
			`(array <expr1> <expr2> ...) ;; Create an array from a series of evaluated expressions`
Modules docs. 2017-06-26 14:45:51 +03:00			```

Reorganised the language guide a bit. 2017-12-22 17:09:53 +03:00			To see all functions available in the `Dynamic` module, enter `(info Dynamic)` at the REPL.

More explanation. 2017-06-26 16:58:23 +03:00			`### Modules and Name Lookup`
Modules docs. 2017-06-26 14:45:51 +03:00			`Functions and variables can be stored in modules which are named and can be nested. To use a symbol inside a module`
			you need to qualify it with the module name, like this: ```Float.cos```.

Fixed imports in docs. 2017-09-06 15:43:41 +03:00			`Using a module makes it possible to access its members without qualifying them:`
Modules docs. 2017-06-26 14:45:51 +03:00
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Fixed imports in docs. 2017-09-06 15:43:41 +03:00			`(use Float)`
Modules docs. 2017-06-26 14:45:51 +03:00
			`(defn f []`
			`(cos 3.2f))`
			```

Fixed imports in docs. 2017-09-06 15:43:41 +03:00			`If there are several used modules that contain symbols with the same name, the type inferer will try to figure`
Modules docs. 2017-06-26 14:45:51 +03:00			`out which one of the symbols you really mean (based on the types in your code). If it can't, it will display an error.`
Renaming: String.count -> String.length, Array.count -> Array.length Issue #236 2018-05-20 10:57:51 +03:00			For example, both the module ```String``` and ```Array``` contain a function named 'length'. In the following code it's
Modules docs. 2017-06-26 14:45:51 +03:00			`possible to see that it's the array version that is needed, and that one will be called:`

language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Fixed imports in docs. 2017-09-06 15:43:41 +03:00			`(use String)`
			`(use Array)`
Modules docs. 2017-06-26 14:45:51 +03:00
			`(defn f []`
Renaming: String.count -> String.length, Array.count -> Array.length Issue #236 2018-05-20 10:57:51 +03:00			`(length [1 2 3 4 5]))`
Modules docs. 2017-06-26 14:45:51 +03:00			```

			`In the following example it's not possible to figure out which type is intended:`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Fixed imports in docs. 2017-09-06 15:43:41 +03:00			`(use String)`
			`(use Array)`
Modules docs. 2017-06-26 14:45:51 +03:00
			`(defn f [x]`
Renaming: String.count -> String.length, Array.count -> Array.length Issue #236 2018-05-20 10:57:51 +03:00			`(length x))`
Modules docs. 2017-06-26 14:45:51 +03:00			```

			`Specifying the type solves this error:`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Fixed imports in docs. 2017-09-06 15:43:41 +03:00			`(use String)`
			`(use Array)`
Modules docs. 2017-06-26 14:45:51 +03:00
			`(defn f [x]`
Renaming: String.count -> String.length, Array.count -> Array.length Issue #236 2018-05-20 10:57:51 +03:00			`(String.length x))`
Language guide. 2017-06-26 12:45:18 +03:00			```

			`### Structs`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
Improved language guide. 2017-06-26 16:50:51 +03:00			`(deftype Vector2 [x Int, y Int])`
Language guide. 2017-06-26 12:45:18 +03:00
Improved language guide. 2017-06-26 16:50:51 +03:00			`(let [my-pos (Vector2.init 10 20)]`
Language guide. 2017-06-26 12:45:18 +03:00			`...)`

			`;; A 'lens' is automatically generated for each member:`
Improved language guide. 2017-06-26 16:50:51 +03:00			`(Vector2.x my-pos) ;; => 10`
			`(Vector2.set-x my-pos 30) ;; => (Vector2 30 20)`
			`(Vector2.update-x my-pos inc) ;; => (Vector2 11 20)`
Language guide. 2017-06-26 12:45:18 +03:00			```

More explanation. 2017-06-26 16:58:23 +03:00			`### C Interop`
language guide: add section on patterns 2018-12-15 17:17:05 +03:00			```clojure
docs: updated language guide 2017-12-18 19:27:51 +03:00			`(system-include "math.h") ;; compiles to #include <math.h>`
			`(local-include "math.h") ;; compiles to #include "math.h"`

Corrected example of registering external function. 2017-11-30 23:49:24 +03:00			`(register blah (Fn [Int Int] String)) ;; Will register the function 'blah' that takes two Int:s and returns a String`
Improved language guide. 2017-06-26 16:50:51 +03:00			`(register pi Double) ;; Will register the global variable 'pi' of type Double`
Show 'register-type' in language guide. 2017-12-22 17:01:00 +03:00
Can register a different name to use in emitted C code. 2018-02-26 21:53:23 +03:00			`(register blah (Fn [Int Int] String) "exit") ;; Will register the function 'blah' but use the name 'exit' in the emitted C code.`

Show 'register-type' in language guide. 2017-12-22 17:01:00 +03:00			`(register-type Apple) ;; Register an opaque C type`
			`(register-type Banana [price Double, size Int]) ;; Register an external C-structs, this will generate getters, setters and updaters.`
Language guide. 2017-06-26 12:45:18 +03:00			```
language guide: add section on patterns 2018-12-15 17:17:05 +03:00
			`### Patterns`

			`Patterns are similar to, but not the same as, Regular Expressions. They were`
			`derived from [Lua](http://lua-users.org/wiki/PatternsTutorial), and are useful`
			`whenever you want to find something within or extract something from strings.`

			`They are simpler than Regular Expressions, as they do not provide alternation.`
			`Nonetheless, they are often very useful and, because they are simpler, also`
			`faster and more predictable.`

			`Here is a little overview of the API:`

			```clojure
			`; you can initialize a pattern with a literal or create one from a string`
			`#"[a-z]"`
			`(Pattern.init "[a-z]")`

			`; you can also get a string back from it`
			`(str #"[a-z]")`
			`(prn #"[a-z]")`

			`; you can find things in strings by index`
			`(Pattern.find #"[a-z]" "1234a") ; => 4`
			`(Pattern.find #"[a-z]" "1234") ; => -1`

			`; also multiple things at once!`
			`(Pattern.find-all #"[a-z]" "1234a b") ; => [4 6]`

			`; matches? checks whether a string matches a pattern`
			`(Pattern.matches? #"(\d+) (\d+)" " 12 13") ; => true`

			`; match returns all match groups of the first match`
			`(Pattern.match #"(\d+) (\d+)" " 12 13") ; => ["12" "13"]`

			`; match-str returns the whole string of the first match`
			`(Pattern.match #"(\d+) (\d+)" " 12 13") ; => "12 13"`

			`; global-match gets all match groups of all matches`
			`(Pattern.global-match #"(\d+) (\d+)" " 12 13 14 15") ; => [["12" "13"] ["14" "15"]]`

			`; substitute helps you replace patterns in a string n times`
			`(Pattern.substitute #"sub-me" "sub-me sub-me sub-me" "replaced" 1) ; => "replaced sub-me sub-me"`

			`; if you want to replace every occurrence, use -1`
			`(Pattern.substitute #"sub-me" "sub-me sub-me sub-me" "replaced" -1) ; => "replaced replaced replaced"`
			```