on 2d3bbcc help sign in
Idris-dev/docs/tutorial/miscellany.rst
554 lines 18 kB history raw
1.. _sect-misc:
3**********
4Miscellany
5**********
7In this section we discuss a variety of additional features:
9+ auto, implicit, and default arguments;
10+ literate programming;
11+ interfacing with external libraries through the foreign function
12+ interface;
13+ type providers;
14+ code generation; and
15+ the universe hierarchy.
17Auto implicit arguments
18=======================
20We have already seen implicit arguments, which allows arguments to be
21omitted when they can be inferred by the type checker, e.g.
23.. code-block:: idris
25 index : {a:Type} -> {n:Nat} -> Fin n -> Vect n a -> a
27In other situations, it may be possible to infer arguments not by type
28checking but by searching the context for an appropriate value, or
29constructing a proof. For example, the following definition of ``head``
30which requires a proof that the list is non-empty:
32.. code-block:: idris
34 isCons : List a -> Bool
35 isCons [] = False
36 isCons (x :: xs) = True
38 head : (xs : List a) -> (isCons xs = True) -> a
39 head (x :: xs) _ = x
41If the list is statically known to be non-empty, either because its
42value is known or because a proof already exists in the context, the
43proof can be constructed automatically. Auto implicit arguments allow
44this to happen silently. We define ``head`` as follows:
46.. code-block:: idris
48 head : (xs : List a) -> {auto p : isCons xs = True} -> a
49 head (x :: xs) = x
51The ``auto`` annotation on the implicit argument means that Idris
52will attempt to fill in the implicit argument by searching for a value
53of the appropriate type. It will try the following, in order:
55- Local variables, i.e. names bound in pattern matches or ``let`` bindings,
56 with exactly the right type.
57- The constructors of the required type. If they have arguments, it will
58 search recursively up to a maximum depth of 100.
59- Local variables with function types, searching recursively for the
60 arguments.
61- Any function with the appropriate return type which is marked with the
62 ``%hint`` annotation.
64In the case that a proof is not found, it can be provided explicitly as normal:
66.. code-block:: idris
68 head xs {p = ?headProof}
70More generally, we can fill in implicit arguments with a default value
71by annotating them with ``default``. The definition above is equivalent
72to:
74.. code-block:: idris
76 head : (xs : List a) ->
77 {default proof { trivial; } p : isCons xs = True} -> a
78 head (x :: xs) = x
80Implicit conversions
81====================
83Idris supports the creation of *implicit conversions*, which allow
84automatic conversion of values from one type to another when required to
85make a term type correct. This is intended to increase convenience and
86reduce verbosity. A contrived but simple example is the following:
88.. code-block:: idris
90 implicit intString : Int -> String
91 intString = show
93 test : Int -> String
94 test x = "Number " ++ x
96In general, we cannot append an ``Int`` to a ``String``, but the
97implicit conversion function ``intString`` can convert ``x`` to a
98``String``, so the definition of ``test`` is type correct. An implicit
99conversion is implemented just like any other function, but given the
100``implicit`` modifier, and restricted to one explicit argument.
102Only one implicit conversion will be applied at a time. That is,
103implicit conversions cannot be chained. Implicit conversions of simple
104types, as above, are however discouraged! More commonly, an implicit
105conversion would be used to reduce verbosity in an embedded domain
106specific language, or to hide details of a proof. Such examples are
107beyond the scope of this tutorial.
109Literate programming
110====================
112Like Haskell, Idris supports *literate* programming. If a file has
113an extension of ``.lidr`` then it is assumed to be a literate file. In
114literate programs, everything is assumed to be a comment unless the line
115begins with a greater than sign ``>``, for example:
117.. code-block:: idris
119 > module literate
121 This is a comment. The main program is below
123 > main : IO ()
124 > main = putStrLn "Hello literate world!\n"
126An additional restriction is that there must be a blank line between a
127program line (beginning with ``>``) and a comment line (beginning with
128any other character).
130Foreign function calls
131======================
133For practical programming, it is often necessary to be able to use
134external libraries, particularly for interfacing with the operating
135system, file system, networking, *et cetera*. Idris provides a
136lightweight foreign function interface for achieving this, as part of
137the prelude. For this, we assume a certain amount of knowledge of C and
138the ``gcc`` compiler. First, we define a datatype which describes the
139external types we can handle:
141.. code-block:: idris
143 data FTy = FInt | FFloat | FChar | FString | FPtr | FUnit
145Each of these corresponds directly to a C type. Respectively: ``int``,
146``double``, ``char``, ``char*``, ``void*`` and ``void``. There is also a
147translation to a concrete Idris type, described by the following
148function:
150.. code-block:: idris
152 interpFTy : FTy -> Type
153 interpFTy FInt = Int
154 interpFTy FFloat = Float
155 interpFTy FChar = Char
156 interpFTy FString = String
157 interpFTy FPtr = Ptr
158 interpFTy FUnit = ()
160A foreign function is described by a list of input types and a return
161type, which can then be converted to an Idris type:
163.. code-block:: idris
165 ForeignTy : (xs:List FTy) -> (t:FTy) -> Type
167A foreign function is assumed to be impure, so ``ForeignTy`` builds an
168``IO`` type, for example:
170.. code-block:: idris
172 Idris> ForeignTy [FInt, FString] FString
173 Int -> String -> IO String : Type
175 Idris> ForeignTy [FInt, FString] FUnit
176 Int -> String -> IO () : Type
178We build a call to a foreign function by giving the name of the
179function, a list of argument types and the return type. The built in
180construct ``mkForeign`` converts this description to a function callable
181by Idris:
183.. code-block:: idris
185 data Foreign : Type -> Type where
186 FFun : String -> (xs:List FTy) -> (t:FTy) ->
187 Foreign (ForeignTy xs t)
189 mkForeign : Foreign x -> x
191Note that the compiler expects ``mkForeign`` to be fully applied to
192build a complete foreign function call. For example, the ``putStr``
193function is implemented as follows, as a call to an external function
194``putStr`` defined in the run-time system:
196.. code-block:: idris
198 putStr : String -> IO ()
199 putStr x = mkForeign (FFun "putStr" [FString] FUnit) x
201Include and linker directives
202-----------------------------
204Foreign function calls are translated directly to calls to C functions,
205with appropriate conversion between the Idris representation of a
206value and the C representation. Often this will require extra libraries
207to be linked in, or extra header and object files. This is made possible
208through the following directives:
210- ``%lib target x`` — include the ``libx`` library. If the target is
211 ``C`` this is equivalent to passing the ``-lx`` option to ``gcc``. If
212 the target is Java the library will be interpreted as a
213 ``groupId:artifactId:packaging:version`` dependency coordinate for
214 maven.
216- ``%include target x`` — use the header file or import ``x`` for the
217 given back end target.
219- ``%link target x.o`` — link with the object file ``x.o`` when using
220 the given back end target.
222- ``%dynamic x.so`` — dynamically link the interpreter with the shared
223 object ``x.so``.
225Testing foreign function calls
226------------------------------
228Normally, the Idris interpreter (used for typechecking and at the REPL)
229will not perform IO actions. Additionally, as it neither generates C
230code nor compiles to machine code, the ``%lib``, ``%include`` and
231``%link`` directives have no effect. IO actions and FFI calls can be
232tested using the special REPL command ``:x EXPR``, and C libraries can
233be dynamically loaded in the interpreter by using the ``:dynamic``
234command or the ``%dynamic`` directive. For example:
236.. code-block:: idris
238 Idris> :dynamic libm.so
239 Idris> :x unsafePerformIO ((mkForeign (FFun "sin" [FFloat] FFloat)) 1.6)
240 0.9995736030415051 : Float
242Type Providers
243==============
245Idris type providers, inspired by F#’s type providers, are a means of
246making our types be “about” something in the world outside of Idris. For
247example, given a type that represents a database schema and a query that
248is checked against it, a type provider could read the schema of a real
249database during type checking.
251Idris type providers use the ordinary execution semantics of Idris to
252run an IO action and extract the result. This result is then saved as a
253constant in the compiled code. It can be a type, in which case it is
254used like any other type, or it can be a value, in which case it can be
255used as any other value, including as an index in types.
257Type providers are still an experimental extension. To enable the
258extension, use the ``%language`` directive:
260.. code-block:: idris
262 %language TypeProviders
264A provider ``p`` for some type ``t`` is simply an expression of type
265``IO (Provider t)``. The ``%provide`` directive causes the type checker
266to execute the action and bind the result to a name. This is perhaps
267best illustrated with a simple example. The type provider ``fromFile``
268reads a text file. If the file consists of the string ``Int``, then the
269type ``Int`` will be provided. Otherwise, it will provide the type
270``Nat``.
272.. code-block:: idris
274 strToType : String -> Type
275 strToType "Int" = Int
276 strToType _ = Nat
278 fromFile : String -> IO (Provider Type)
279 fromFile fname = do str <- readFile fname
280 return (Provide (strToType (trim str)))
282We then use the ``%provide`` directive:
284.. code-block:: idris
286 %provide (T1 : Type) with fromFile "theType"
288 foo : T1
289 foo = 2
291If the file named ``theType`` consists of the word ``Int``, then ``foo``
292will be an ``Int``. Otherwise, it will be a ``Nat``. When Idris
293encounters the directive, it first checks that the provider expression
294``fromFile theType`` has type ``IO (Provider Type)``. Next, it executes
295the provider. If the result is ``Provide t``, then ``T1`` is defined as
296``t``. Otherwise, the result is an error.
298Our datatype ``Provider t`` has the following definition:
300.. code-block:: idris
302 data Provider a = Error String
303 | Provide a
305We have already seen the ``Provide`` constructor. The ``Error``
306constructor allows type providers to return useful error messages. The
307example in this section was purposefully simple. More complex type
308provider implementations, including a statically-checked SQLite binding,
309are available in an external collection [1]_.
311C Target
312========
314The default target of Idris is C. Compiling via :
316::
318 $ idris hello.idr -o hello
320is equivalent to :
322::
324 $ idris --codegen C hello.idr -o hello
326When the command above is used, a temporary C source is generated, which
327is then compiled into an executable named ``hello``.
329In order to view the generated C code, compile via :
331::
333 $ idris hello.idr -S -o hello.c
335To turn optimisations on, use the ``%flag C`` pragma within the code, as
336is shown below :
338.. code-block:: idris
340 module Main
341 %flag C "-O3"
343 factorial : Int -> Int
344 factorial 0 = 1
345 factorial n = n * (factorial (n-1))
347 main : IO ()
348 main = do
349 putStrLn $ show $ factorial 3
351JavaScript Target
352=================
354Idris is capable of producing *JavaScript* code that can be run in a
355browser as well as in the *NodeJS* environment or alike. One can use the
356FFI to communicate with the *JavaScript* ecosystem.
358Code Generation
359---------------
361Code generation is split into two separate targets. To generate code
362that is tailored for running in the browser issue the following command:
364::
366 $ idris --codegen javascript hello.idr -o hello.js
368The resulting file can be embedded into your HTML just like any other
369*JavaScript* code.
371Generating code for *NodeJS* is slightly different. Idris outputs a
372*JavaScript* file that can be directly executed via ``node``.
374::
376 $ idris --codegen node hello.idr -o hello
377 $ ./hello
378 Hello world
380Take into consideration that the *JavaScript* code generator is using
381``console.log`` to write text to ``stdout``, this means that it will
382automatically add a newline to the end of each string. This behaviour
383does not show up in the *NodeJS* code generator.
385Using the FFI
386-------------
388To write a useful application we need to communicate with the outside
389world. Maybe we want to manipulate the DOM or send an Ajax request. For
390this task we can use the FFI. Since most *JavaScript* APIs demand
391callbacks we need to extend the FFI so we can pass functions as
392arguments.
394The *JavaScript* FFI works a little bit differently than the regular
395FFI. It uses positional arguments to directly insert our arguments into
396a piece of *JavaScript* code.
398One could use the primitive addition of *JavaScript* like so:
400.. code-block:: idris
402 module Main
404 primPlus : Int -> Int -> IO Int
405 primPlus a b = mkForeign (FFun "%0 + %1" [FInt, FInt] FInt) a b
407 main : IO ()
408 main = do
409 a <- primPlus 1 1
410 b <- primPlus 1 2
411 print (a, b)
413Notice that the ``%n`` notation qualifies the position of the ``n``-th
414argument given to our foreign function starting from 0. When you need a
415percent sign rather than a position simply use ``%%`` instead.
417Passing functions to a foreign function is very similar. Let’s assume
418that we want to call the following function from the *JavaScript* world:
420.. code-block:: idris
422 function twice(f, x) {
423 return f(f(x));
424 }
426We obviously need to pass a function ``f`` here (we can infer it from
427the way we use ``f`` in ``twice``, it would be more obvious if
428*JavaScript* had types).
430The *JavaScript* FFI is able to understand functions as arguments when
431you give it something of type ``FFunction``. The following example code
432calls ``twice`` in *JavaScript* and returns the result to our Idris
433program:
435.. code-block:: idris
437 module Main
439 twice : (Int -> Int) -> Int -> IO Int
440 twice f x = mkForeign (
441 FFun "twice(%0,%1)" [FFunction FInt FInt, FInt] FInt
442 ) f x
444 main : IO ()
445 main = do
446 a <- twice (+1) 1
447 print a
449The program outputs ``3``, just like we expected.
451Including external *JavaScript* files
452-------------------------------------
454Whenever one is working with *JavaScript* one might want to include
455external libraries or just some functions that she or he wants to call
456via FFI which are stored in external files. The *JavaScript* and
457*NodeJS* code generators understand the ``%include`` directive. Keep in
458mind that *JavaScript* and *NodeJS* are handled as different code
459generators, therefore you will have to state which one you want to
460target. This means that you can include different files for *JavaScript*
461and *NodeJS* in the same Idris source file.
463So whenever you want to add an external *JavaScript* file you can do
464this like so:
466For *NodeJS*:
468.. code-block:: idris
470 %include Node "path/to/external.js"
472And for use in the browser:
474.. code-block:: idris
476 %include JavaScript "path/to/external.js"
478The given files will be added to the top of the generated code.
480Including *NodeJS* modules
481--------------------------
483The *NodeJS* code generator can also include modules with the ``%lib``
484directive.
486.. code-block:: idris
488 %lib Node "fs"
490This directive compiles into the following *JavaScript*
492.. code-block:: javascript
494 var fs = require("fs");
496Shrinking down generated *JavaScript*
497-------------------------------------
499Idris can produce very big chunks of *JavaScript* code. However, the
500generated code can be minified using the ``closure-compiler`` from
501Google. Any other minifier is also suitable but ``closure-compiler``
502offers advanced compilation that does some aggressive inlining and code
503elimination. Idris can take full advantage of this compilation mode
504and it’s highly recommended to use it when shipping a *JavaScript*
505application written in Idris.
507Cumulativity
508============
510Since values can appear in types and *vice versa*, it is natural that
511types themselves have types. For example:
513::
515 *universe> :t Nat
516 Nat : Type
517 *universe> :t Vect
518 Vect : Nat -> Type -> Type
520But what about the type of ``Type``? If we ask Idris it reports
522::
524 *universe> :t Type
525 Type : Type 1
527If ``Type`` were its own type, it would lead to an inconsistency due to
528`Girard’s paradox <http://www.cs.cmu.edu/afs/cs.cmu.edu/user/kw/www/scans/girard72thesis.pdf>`_ , so internally there is a
529*hierarchy* of types (or *universes*):
531.. code-block:: idris
533 Type : Type 1 : Type 2 : Type 3 : ...
535Universes are *cumulative*, that is, if ``x : Type n`` we can also have
536that ``x : Type m``, as long as ``n < m``. The typechecker generates
537such universe constraints and reports an error if any inconsistencies
538are found. Ordinarily, a programmer does not need to worry about this,
539but it does prevent (contrived) programs such as the following:
541.. code-block:: idris
543 myid : (a : Type) -> a -> a
544 myid _ x = x
546 idid : (a : Type) -> a -> a
547 idid = myid _ myid
549The application of ``myid`` to itself leads to a cycle in the universe
550hierarchy — ``myid``\ ’s first argument is a ``Type``, which cannot be
551at a lower level than required if it is applied to itself.
553.. [1]
554 https://github.com/david-christiansen/idris-type-providers