2020-02-27 06:00:59 +03:00
|
|
|
|
# `with-utf8`
|
2020-02-16 03:08:31 +03:00
|
|
|
|
|
2020-02-18 04:00:13 +03:00
|
|
|
|
Get your IO right on the first try.
|
2020-02-16 03:08:31 +03:00
|
|
|
|
|
|
|
|
|
Reading files in Haskell is trickier than it could be due to the non-obvious
|
|
|
|
|
interactions between file encodings and system locale. This library is meant
|
|
|
|
|
to make it easy once and for all by providing “defaults” that make more sense
|
|
|
|
|
in the modern world.
|
|
|
|
|
|
2020-02-18 04:00:13 +03:00
|
|
|
|
See [this blog post][blog:post] for more details on why this library needs to
|
2020-02-16 03:08:31 +03:00
|
|
|
|
exists and an explanation of some of the opinionated decisions it is based on.
|
|
|
|
|
|
2020-02-27 06:00:59 +03:00
|
|
|
|
[blog:post]: https://serokell.io/blog/haskell-with-utf8
|
2020-02-16 03:08:31 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Use
|
|
|
|
|
|
2020-02-18 04:00:13 +03:00
|
|
|
|
See the documentation on Hackage for details, this is a quick summary.
|
|
|
|
|
|
2020-02-16 03:15:52 +03:00
|
|
|
|
### Step 1: Get it
|
|
|
|
|
|
2020-02-27 06:00:59 +03:00
|
|
|
|
The [library is on Hackage][hackage:with-utf8],
|
2020-02-18 04:00:13 +03:00
|
|
|
|
go ahead and add it to the dependencies of your project.
|
2020-02-16 03:15:52 +03:00
|
|
|
|
|
2020-02-27 06:00:59 +03:00
|
|
|
|
[hackage:with-utf8]: https://hackage.haskell.org/package/with-utf8
|
2020-02-16 03:15:52 +03:00
|
|
|
|
|
|
|
|
|
### Step 2: Wrap your `main`
|
|
|
|
|
|
2020-02-18 04:00:13 +03:00
|
|
|
|
Import `withUtf8` from `Main.Utf8` and wrap it around your `main`:
|
2020-02-16 03:15:52 +03:00
|
|
|
|
|
|
|
|
|
```haskell
|
2020-02-18 04:00:13 +03:00
|
|
|
|
import Main.Utf8 (withUtf8)
|
|
|
|
|
|
2020-02-16 03:15:52 +03:00
|
|
|
|
main :: IO ()
|
2020-02-18 04:00:13 +03:00
|
|
|
|
main = withUtf8 $
|
|
|
|
|
{- ... your main function ... -}
|
2020-02-16 03:15:52 +03:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This will make sure that if your program reads something from `stdin` or
|
|
|
|
|
outputs something to `stdout`/`stderr`, it will not fail with a runtime
|
|
|
|
|
error due to encoding issues.
|
|
|
|
|
|
|
|
|
|
### Step 3: Read files using UTF-8
|
|
|
|
|
|
|
|
|
|
If you are going to read a text file (to be precise, if you are going to open
|
|
|
|
|
a file in text mode), you’ll probably use `withFile`, `openFile`, or `readFile`.
|
|
|
|
|
Grab the first two from `System.IO.Utf8` or the latter from `Data.Text.IO.Utf8`.
|
|
|
|
|
|
2020-02-18 04:00:13 +03:00
|
|
|
|
_Note: it is best to import these modules qualified._
|
2020-02-16 03:15:52 +03:00
|
|
|
|
|
|
|
|
|
_Note: there is no `System.IO.Utf8.readFile` because it’s 2020 and
|
|
|
|
|
you should not read `String`s from files._
|
|
|
|
|
|
|
|
|
|
All these functions will make sure that the content will be treated as if it
|
2020-02-18 04:00:13 +03:00
|
|
|
|
was encoded in UTF-8.
|
2020-02-16 03:15:52 +03:00
|
|
|
|
|
|
|
|
|
If, for some reason, you really need to use `withFile`/`openFile` from `base`,
|
2020-02-17 05:49:07 +03:00
|
|
|
|
or you got your file handle from somewhere else, wrap the code that works
|
2020-02-18 04:00:13 +03:00
|
|
|
|
with it in a call to `withHandle` from `System.IO.Utf8`:
|
2020-02-16 03:15:52 +03:00
|
|
|
|
|
|
|
|
|
```haskell
|
|
|
|
|
import qualified System.IO as IO
|
|
|
|
|
import qualified System.IO.Utf8 as Utf8
|
|
|
|
|
|
2020-02-17 05:49:07 +03:00
|
|
|
|
doSomethingWithAFile :: IO.Handle -> IO ()
|
2020-02-18 04:00:13 +03:00
|
|
|
|
doSomethingWithAFile h = Utf8.withhandle h $ do
|
2020-02-17 05:49:07 +03:00
|
|
|
|
{- ... work with the file ... -}
|
2020-02-16 03:15:52 +03:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Step 4: Write files using UTF-8
|
|
|
|
|
|
|
|
|
|
When writing a file either open it using `withFile`/`openFile` from
|
|
|
|
|
`System.IO.Utf8` or write to it directly with `writeFile` from
|
|
|
|
|
`Data.Text.IO.Utf8`.
|
|
|
|
|
|
2020-02-18 04:00:13 +03:00
|
|
|
|
_Note: it is best to import these modules qualified._
|
2020-02-16 03:15:52 +03:00
|
|
|
|
|
2020-02-18 04:00:13 +03:00
|
|
|
|
_Note: there is no `System.IO.Utf8.writeFile`._
|
2020-02-16 03:15:52 +03:00
|
|
|
|
|
|
|
|
|
If, for some reason, you really need to use `withFile`/`openFile` from `base`,
|
|
|
|
|
do the same as in the previous step.
|
2020-02-16 03:08:31 +03:00
|
|
|
|
|
2020-04-21 01:38:55 +03:00
|
|
|
|
## Troubleshooting
|
|
|
|
|
|
|
|
|
|
Locales are pretty straightforward, but some people might have their terminals
|
|
|
|
|
misconfigured for various reasons. To help troubleshoot any potential issues,
|
|
|
|
|
this package comes with a tool called `utf8-troubleshoot`.
|
|
|
|
|
|
|
|
|
|
This tool outputs some basic information about locale settings in the OS and
|
|
|
|
|
what they end up being mapped to in Haskell. If you are looking for help,
|
|
|
|
|
please, provide the output of this tool, or if you are helping someone,
|
|
|
|
|
ask them to run this tool and provide the output.
|
2020-02-16 03:08:31 +03:00
|
|
|
|
|
|
|
|
|
## Contributing
|
|
|
|
|
|
|
|
|
|
If you encounter any issues when using this library or have improvement ideas,
|
|
|
|
|
please open report in issue on GitHub. You are also very welcome to submit
|
|
|
|
|
pull request, if you feel like doing so.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## License
|
|
|
|
|
|
|
|
|
|
[MPL-2.0] © [Serokell]
|
|
|
|
|
|
|
|
|
|
[MPL-2.0]: https://spdx.org/licenses/MPL-2.0.html
|
|
|
|
|
[Serokell]: https://serokell.io/
|