A couple of months ago something rather unusual happened:\nI got permission to rewrite an old production codebase.\nNothing wild inside, just a few services bouncing messages\naround and calling other places. I wrote most of it years\nago, and most of it was god awful on account of the fact that\nI had no idea what I was doing at the time.
\nLike any real codebase written in Haskell, it relied on\nan effect system to do… well, things.
\nIndeed, like most of the community, I couldn't properly\nformulate what an effect system does. Yes, I know there are\nat least\nten of them,\nall at odds with one another, yet seemingly completely\ninterchangeable. Smarter people have narrowed the goals down to\ntracking effects, mocking and internal consistency,\nwhich strongly implies that plain IO is incapable of\nthese things, and that's something I could neither confirm nor deny.
\nSo now, being able to reassemble an entire system from\nthe ground up, the question I got to ask was…
\nNo, I can do that manually.
\nNo, tracking effects does not preclude anyone\nfrom adding bad IO to an effect\nimplementation, from importing\nunsafePerformIO,\nor from finding a sum of an infinite\nlist.1
\nNo, for pure functions that do mutation\nthe ST monad works just fine.
\n**Is there any benefit to tracking IO as a
separate effect?**
No, and in fact the opposite: there are small effects\nall around the codebase that I'd rather not\ntrack.
\nFor example, random number generation is commonplace,\nbut the steps necessary to generate a random value\nare different every time. The only shared behavior is\nthe use of generator state, and even then it's unclear\nif passing it around has any benefits over using the\nglobal generator.
\nNo, none of the effects I'm using need to overlap or\nnest. I'm not precisely doing rocket science over here;\nI'd prefer that whatever I'm using doesn't come with a\nwhole separate manual.
\nAfter throwing out pretty much every single feature, I finally\nfound the one thing I was using the effect system for:\nerror handling. Which upon closer inspection turned out\nto be the execution of some set of other effects followed\nby…
\nThis may well be the most embarrassing problem in all of\nHaskell.\nOver\nand\nover\nagain people waltz in with the exact same basic question:\n"How do I return from a function early?".
\nAnd time and time again they're hit with the same three\noptions:
\nMake it so that the error is the last statement in
the function, using Either or continuations.
Code looks awful and constantly drifts to the right.
\n*Roughly equivalent to burying a landmine\nin your backyard.*
\nUse an effect system.
???
\nThe catch is that effect systems don't have some special\nthird way of handling errors, they merely wrap the other\ntwo approaches. Notably,\nExceptT\nis a faithful implementation of the first approach,\nthreading an Either through every action. That's\nobviously very inefficient and is known to not\ncompose well, so I'd prefer the second option.
\nTo do this we'll need some way to carry the knowledge that\na specific resource (in our case an exception) may only be\nused (thrown) within a specific function. This,\nunsurprisingly, is a problem that has been solved decades ago\nfor file handles and raw pointers through the use of\nbracket.\nThough in our case there's nothing to allocate, the\nvalue is on the type level:
\n{-# LANGUAGE RoleAnnotations #-}\n\nmodule Early\n ( Early\n , leave\n , runEarly\n ) where\n\nimport Control.Exception\nimport Data.Typeable\n\n\ntype role Early nominal\ndata Early a = Early\n\n\ndata ReturningEarly a = ReturningEarly a\n\ninstance Typeable e => Show (ReturningEarly e) where\n show = displayException\n\ninstance Typeable e => Exception (ReturningEarly e) where\n displayException (ReturningEarly _) = "Early return exception"\n\n\nleave :: Typeable e => Early e -> e -> IO a\nleave _Early e = throwIO $ ReturningEarly e\n\n\nrunEarly :: Typeable e => (Early e -> IO a) -> IO (Either e a)\nrunEarly f = catch (Right <$> f Early) (\\(ReturningEarly e) -> pure $ Left e)And the module can then be used like this:
\nimport Control.Monad\nimport Early\n\n\nexample :: Early () -> IO Bool\nexample early = do\n putStrLn "Ran this action"\n when True $ do\n leave early ()\n\n putStrLn "Didn't run this action"\n pure Trueghci> runEarly example\nRan this action\nLeft ()There are no caveats to this code beyond those that come\nwith using bracket.
\nAnd just like that I ran out of reasons to use an\neffect system.
\nThere's not much of a story to tell from this point on:\nI shaped every other effect much like I had shaped\nEarly and everything fell into place nicely.\nThe rest of the post are my findings, structured to the\nbest of my ability.
\nFinding a definition for the word "effect" is unfortunately\nas tedious as finding one for "effect system".\nIf I am to trust\nsome people on Reddit,\nan "effect" is a convention to only access specific\nside effects through a common interface tracked on\nthe type level.
\nBoth Early and "handles" from the\n"handle pattern"\nfit this definition:
\nThey have interfaces (leave,
createUser) and implementations\n(runEarly, withHandle). Compare to\nthe similar\neffect/handler\nseparation in\neff.
createUser :: Handle -> Text -> IO User\n\nlistDirectory :: FileSystem :> es => FilePath -> Eff es [FilePath]\n\nleave :: Early e -> e -> IO a\n\nthrowError :: Error e :> es => e -> Eff es aMore generally, an effect in plain IO is a\ndata type that serves as a bridge between interface\nand implementation functions. The data type's\nconstructor is as such an implementation detail and\nshould not be exported.
\nEffects are tracked as function arguments; this is\nquite different from effect systems, which prefer\nconstraints. One benefit of this is that we don't have\nto deal with the complexities of\ndisambiguating,\nreordering\nor\nreinterpreting\neffects.
\nEffect systems tend to put all of their definitions\ninto a single module, but there is no hard requirement\nfor this. For example, Early can be broken into
\nmodule Effect.Early.Leave (Early, leave) wheremodule Effect.Early.Runner (Early, runEarly) wheremodule Effect.Early.Internal (Early, leave, runEarly) whereThis can be used to track function access at module\nlevel; particularly useful if an effect has multiple\ninterfaces (say, multiple programs rely on the\nsame effect data type) and/or multiple implementations\n(say, mocking).
\nEffects may throw exceptions, but if they do they're also\nresponsible for catching them. Much like the data\ntype constructors, exceptions are an implementation detail.
\nEffect implementations are generally not invoked alone\nhowever, they're stacked into a\nterrifyingly large pile\nat the very edge of the program. In our case stacking\ndoes work out of the box, but each layer pushes the\nsuccessful case deeper into the chain:
\nrunFoo :: (Foo -> IO a) -> IO (Either FooError a)\n\nstack\n :: (Foo -> Bar -> Qux -> IO a)\n -> IO (Either FooError (Either BarError (Either QuxError a)))\nstack f =\n runFoo $ \\foo ->\n runBar $ \\bar ->\n runQux $ \\qux ->\n f foo bar quxThis can be solved rather nicely by using\nslightly more complicated\ntypes:3
\nrunFoo :: (Foo -> IO (Either e a)) -> IO (Either (Either e FooError) a)\n\nlift :: IO a -> IO (Either Void a)\nlift = fmap Right\n\nstack\n :: (Foo -> Bar -> Qux -> IO a)\n -> IO (Either (Either (Either (Either Void QuxError) BarError) FooError) a)\nstack f =\n runFoo $ \\foo ->\n runBar $ \\bar ->\n runQux $ \\qux ->\n lift $\n f foo bar quxPassing around two effects with the same name would\nbe wildly confusing; passing something like a\nTagged "helper" Database would be a massive\nnuisance. If a user needs two of the same effect,\nthey should wrap the functions on their side into nice\nnewtyped names (e.g. HelperDatabase).
\nDuplicate exception handling in implementation functions\ncan be addressed in a similar way by providing an extra\nargument and matching on that, essentially letting the\nuser call one effect "database 1" and the other "database 2".
\nLet's look at all of the categories of effects\nI ended up with (or without) in my codebase.
\nConfiguration data is generally read at application start\nbefore invoking the implementation stack, and is passed\ninto it as either function arguments or "reader" effects.
\nTrying to implement a "reader" outside of an effect\nsystem results in a bunch of redundant wrapping:
\ndata Conf =\n Conf\n { bar :: Int\n , baz :: Bool\n , qux :: String\n }\n\ngetBar :: Conf -> Int\ngetBar = bar\n\nrunConf :: Conf -> Conf\nrunConf = idI thus prefer to keep configuration data types in\nseparate modules and to pass them around as arguments.
\nAn effect that can be mocked is simply a product\nof functions:
\ndata Database =\n Database\n { createUser :: Text -> IO (Maybe User)\n , getUserMail :: User -> IO [Mail]\n }\n\nrunRealDatabase :: RealDatabase -> Database\nrunRealDatabase db =\n Database\n { createUser = Database.createUser db\n , getUserMail = Database.getUserMail db\n }Effects that never fail—like logging and metrics\ncollection—are special cases of mocking:
\nimport Data.ByteString.Builder\nimport Data.Text (Text)\nimport Data.Text.Encoding\nimport System.IO\n\n\nnewtype Logger = Logger (Builder -> IO ())\n\nnote :: Logger -> Text -> IO ()\nnote (Logger f) = f . encodeUtf8Builder\n\nrunLogger :: Handle -> Logger\nrunLogger handle =\n Logger $ \\msg ->\n hPutBuilder handle $ msg <> "\\n"I'll use the database effect as an example.\nHere's a rough outline of the implementation function:
\nimport Control.Exception\nimport Database.PostgreSQL.Simple\nimport GHC.Stack\nimport System.IO\n\n\nnewtype Database = Database Connection\n\nrunDatabase\n :: ConnectInfo\n -> (Database -> IO (Either e a))\n -> IO (Either (Either e DatabaseError) a)\nrunDatabase connInfo f =\n mask $ \\unmask -> do\n conn <- connect connInfo\n\n let cleanup = close conn\n\n ei <- unmask (Right <$> f (Database conn))\n `catch` \\ex ->\n case fromException ex of\n Just dbEx -> pure $ Left (dbEx :: DatabaseException)\n Nothing -> _ -- (4) Failure above.\n case ei of\n Right (Right a) -> _ -- (1) Success.\n Right (Left e) -> _ -- (2) Failure below.\n Left ex -> _ -- (3) Failure at this level.For the effect to behave the same regardless of its\nposition within the implementation stack cases (1),\n(2) and (4) should all use the same cleanup\nfunction.
\nInterface functions may use the resource directly,\nalthough in libraries that use exceptions liberally it's\nmore convenient to funnel all uses through a helper function:
\ndata DatabaseException = DatabaseException CallStack DatabaseExceptionKind\n\ndata DatabaseExceptionKind = DatabaseSqlException SqlError\n | _\n\nhandlesPostgres :: HasCallStack => Database -> (Connection -> IO a) -> IO a\nhandlesPostgres (Database conn) f =\n let rethrow :: (e -> DatabaseExceptionKind) -> e -> IO a\n rethrow kind = throwIO . DatabaseException callStack . kind\n\n in f conn\n `catches`\n [ Handler $ rethrow DatabaseSqlException\n-- , -- however many more of these are necessary --\n ]\n\n\ncreateUser :: Database -> Text -> IO (Maybe User)\ncreateUser db =\n handlesPostgres db $ \\conn ->\n _ connTaking argument passing to the extreme, one will\ninevitably end up with a function that looks like
\nendpoint\n :: AuthConf -> ServiceConf\n -> Logger -> Metrics -> Early ServerError -> Database -> Cache -> Messaging\n -> AuthHeader -> EndpointRequest -> IO EndpointResponse\nendpoint authConf serviceConf logger metrics early database cache messaging\n authHeader EndpointRequest {..} = doTo combat this the "handle pattern" post proposes\nnesting effects; the\n"ReaderT design pattern"\npost instead suggests carrying all the effects in a\nrecord. Both of these solutions are wrong.
\nGHC has warnings for\nunused arguments, and since we\ntreat effects as arguments, we can use it to point out\nunused effects. To leverage this, the code must be\nstructured in such a way that each statement uses at\nmost one effect. A bunch of statements then bundle into\na function, functions bundle into larger functions,\nuntil we reach the aforementioned endpoint.
\nAs an example, creating a user actually requires at least\nthree effects:
\nnewUser :: Database -> Text -> IO (Maybe User)\n\ncreateUser :: Logger -> Early ServerError -> Database -> Text -> IO User\ncreateUser logger early database qux = do\n mayUser <- newUser database qux\n case mayUser of\n Just user -> pure user\n Nothing -> do\n note logger "Could not create a user entry"\n leave early err500This approach works in the opposite direction too:\nif we need to find out what caused a\nDatabaseError, we only need to track which\nfunctions are passed the Database argument.
\nRunning the implementation stack is as straightforward\nas in any effect system:
\nmain :: IO ()\nmain = do\n conf <- readConfiguration\n\n let logger = runLogger stderr\n\n metrics <- runMetrics (getMetricsConf conf)\n\n withDatabasePool (getDatabaseConf conf) $ \\dbPool -> do\n\n withMessagingEnv (getMessagingConf conf) $ \\msgEnv -> do\n\n note logger "Initialization complete"\n\n consumeMessagesForever msgEnv $ \\newMsg ->\n runStack logger metrics dbPool msgEnv $ \\early database messaging ->\n process logger metrics early database messaging newMsg\n\n\nrunStack\n :: Logger -> Metrics -> DatabasePool -> MessagingEnv\n -> (Early () -> Database -> Messaging -> IO ())\n -> IO ()\nrunStack logger metrics dbPool msgEnv f = do\n ei <- runEarly $ \\early ->\n runDatabase logger metrics dbPool $ \\database ->\n runMessaging logger metrics msgEnv $ \\msg ->\n lift $\n f early database msg\n\n case ei of\n Right () -> _ -- Success\n Left err ->\n case err of\n Left (Left (Right msgError)) -> _ -- Messaging error \n Left (Right dbError) -> _ -- Database error \n Right () -> _ -- Early return\n\n\nprocess\n :: Logger -> Metrics -> Early () -> Database -> Messaging\n -> NewMessage -> IO ()The shape remains almost the same when using\nservant.\nThe only quirk is that argument passing renders all\nthe fancy\nhoisting functionality completely\nunusable, so the stack has to be invoked inside each\nendpoint manually.
\nrunStack\n :: Logger -> Metrics -> DatabasePool -> MessagingEnv\n -> (Early ServerError -> Database -> Messaging -> IO a)\n -> Handler a\nrunStack logger metrics dbPool msgEnv f =\n MkHandler $ do\n ei <- runEarly $ \\early ->\n runDatabase logger metrics dbPool $ \\database ->\n runMessaging logger metrics msgEnv $ \\msg ->\n lift $\n f early database msg\n\n case ei of\n Right a -> pure $ Right a\n Left err ->\n case err of\n Left (Left (Right msgError)) -> _ -- Messaging error \n Left (Right dbError) -> _ -- Database error \n Right srvError -> pure $ Left srvError\n\n\ntype API = CountAPI :<|> _\n\nserver\n :: Logger -> Metrics -> DatabasePool -> MessagingEnv\n -> Server API\nserver logger metrics dbPool msgEnv =\n countServer logger metrics dbPool msgEnv\n :<|> _\n\n\ntype CountAPI = "count"\n :> ReqBody '[JSON] CountRequest\n :> Get '[JSON] CountResponse\n\ncountServer\n :: Logger -> Metrics -> Early ServerError -> DatabasePool -> MessagingEnv\n -> Server CountAPI\ncountServer logger metrics early dbPool msgEnv request =\n runStack logger metrics dbPool msgEnv $ \\early database messaging ->\n countEndpoint logger metrics early database messaging request\n\ncountEndpoint\n :: Logger -> Metrics -> Early ServerError -> Database -> Messaging\n -> CountRequest -> IO CountResponseThere is only one unique feature effect\nsystems—or, more generally, custom monads designed\nto supercede IO—provide: they can do\nanything in between the lines [of code]. And I don't\nthink that's a good thing.
\nHere are the advantages of running effects\nin plain IO:
\nHere are the disadvantages:
\nI hope this knowledge is used for things.
","headings":[{"level":1,"text":"You don't need an effect system","id":"you-don-t-need-an-effect-system"},{"level":2,"text":"How we got here","id":"how-we-got-here"},{"level":3,"text":"Am I using the effect system for anything?","id":"am-i-using-the-effect-system-for-anything"},{"level":2,"text":"Early return","id":"early-return"},{"level":3,"text":"Type-safe exceptions","id":"type-safe-exceptions"},{"level":2,"text":"Intermission","id":"intermission"},{"level":2,"text":"Effects in plain IO","id":"effects-in-plain-io"},{"level":3,"text":"Module structure","id":"module-structure"},{"level":3,"text":"Error handling","id":"error-handling"},{"level":3,"text":"Duplicate effects","id":"duplicate-effects"},{"level":2,"text":"Real-world effects","id":"real-world-effects"},{"level":3,"text":"Argument passing, fancier","id":"argument-passing-fancier"},{"level":3,"text":"Mocking","id":"mocking"},{"level":3,"text":"Tracking resources","id":"tracking-resources"},{"level":2,"text":"Tying everything together","id":"tying-everything-together"},{"level":3,"text":"Interfaces","id":"interfaces"},{"level":3,"text":"Implementations","id":"implementations"},{"level":2,"text":"Conclusion","id":"conclusion"}]}}