@@ -1,4 +1,4 @@
1−# Internal vs External
1+# Internal vs External Modules
3We keep all modules exposed to facilitate convenient exposure of experimental
3We keep all modules exposed to facilitate convenient exposure of experimental
4APIs and constructors to users. It allows users of the library to experiment
4APIs and constructors to users. It allows users of the library to experiment
@@ -33,27 +33,102 @@ expose an API.
34We choose the first approach.
34We choose the first approach.
36−# Name Spaces
36+# Module Name Spaces
38We use the "Streamly" prefix to all the module names so that they do not
38We use the "Streamly" prefix to all the module names so that they do not
39conflict with any other module on Hackage.
39conflict with any other module on Hackage.
41−We have the following division under Streamly:
41+We have the following module hierarchy under Streamly:
43−* Data: All the data structures that are immutable and make use of the GC
43+* Data: All the data structures that make use of the unpinned GC memory to
44− memory to store data. These data structures are mainly useful for stream
44+ store data. These data structures are suitable for stream processing but
45− processing and not for storing data in memory for longer durations.
45+ may not be suitable for storing large amounts of data in memory for longer
46+ durations. These are suitable for short lived and smaller structures
47+ because they can be moved by the GC to defragment the heap.
47* Memory: This name space is for data structures that make use of the memory as
49* Memory: This name space is for data structures that make use of the memory as
48− a storage device. This is not GC allocated memory but rather pinned memory.
50+ a persistent storage device. The memory may be allocated by foreign
49− The memory in this case is treated just like any other storage device e.g.
51+ allocators or pinned memory allocated by GHC. Because the memory is pinned it
50− disk drives, but faster. The data is serialized into memory. Unlike the data
52+ can be used for interfacing with the system/kernel. These structures are
51− structure under "Data" these data structures have no impact on GC. Many of
53+ efficient for storing large amounts of data for longer durations because it
52− the data structures in this category are parallel to the data structures
54+ does not have to be copied by the GC. These structures may not be suitable
53− under "FileSystem".
55+ for small, short lived data because that is likely to fragment the heap.
55* FileSystem: This name space is for data structures that reside in files
57* FileSystem: This name space is for data structures that reside in files
56 provided by a file system interface on top of storage devices.
58 provided by a file system interface on top of storage devices.
58* Network: This name space is for APIs that access data from remote computers
60* Network: This name space is for APIs that access data from remote computers
59 over the network.
61 over the network.
65+As explained above, we distinguish two types of data structures under "Data"
66+and "Memory". Alternatively, we could have used a "Memory" namespace under
67+each data structure e.g. "Streamly.Data.Array.Memory" instead of using a top
68+level "Streamly.Memory", however, we chose to distinguish such data structures
69+using a top level "Memory" name space because it enforces consistent naming by
70+fitting all such data structures under this top level hierarchy. It also makes
71+it easier to find out what all data structures fall in this category.
73+# Module Types and Naming
77+Abstract modules are meant to represent an abstract interface (e.g. a type
78+class). Concrete modules can make use of this interface and possibly
79+extend it to provide concrete functionality.
81+The general convention in the Haskell ecosystem for naming an abstract
82+interface module is to name it as "Module.Class" (e.g. Control.Monad.IO.Class).
83+An alternative name could be "Module.Interface".
85+In some other cases such modules are named after the class name (e.g. see the
86+array package for an example). This is more appropriate when there is no single
87+hierarchy where we can place the ".Class" module. For example, we have arrays
88+in Data.Array, Memory.Array, we have to choose one over the other to place the
89+".Class" module for an array abstraction. Alternatively, we can choose
90+"Data.IsArray" instead.
92+Yet another way could be to use the parent module as an interface module and
93+the child modules as concrete modules. For example, "Streamly.Data.Stream"
94+module could provide the common "Stream" type and the "IsStream" type class.
95+The submodule "Streamly.Data.Stream.Serial" can provide a concrete "Serial"
96+stream type importing the "Streamly.Data.Stream" abstract module.
100+Some modules represent common types or utility functions that are shared across
101+multiple modules. The general convention is to name such modules as
102+"Module.Types", "Module.Common", or "Module.Core".
104+## Constrained Modules
106+Some modules represent operations on a type which constrain a type using a type
107+class or a specific instance of a general type. For example, we may have a
108+module representing operations on a stream of any type and another module that
109+specifically deals with operations on a Char stream. There are two ways to deal
112+First is to use a submodule for the constrained type. For example,
113+`Streamly.Data.Stream` represents a general stream type whereas
114+`Streamly.Data.Stream.Char` represents operations on a stream of Char type.
115+This makes sense where the type we are constraining to is a specific type
116+rather than a type constrained using a type class.
118+Second is to use a separate hierarchy for the constrained type. For example, we
119+could use `Streamly.Data.Array` for a general array and `Streamly.Prim.Array`
120+for an array that works on `Prim` types. This makes sense when the type is
121+constrained by a type class, we may have more data structures for that
122+constrained type to be bundled under that hierarchy.
126+In some cases we may want to aggregate the functionality of several small
127+modules in a combined aggregate module. In many cases, the aggregate module is
128+made a parent module of the constituent modules. The parent module depends on
129+the child modules and exposes the functionality from the constituent modules.
131+## Placeholder Modules
133+In some cases a parent module is just a placeholder in the namespace and does
134+not export any functionality.