# Handling failures in pipelines with funflow

Many pipelines are configurable, and different parameter sets may be nonsensical. Furthermore, differences in input data or the state of a network or other resources may lead to an less obvious state of error. This is to say that pipelines are somewhat inherently susceptible to encountering an exceptional condition.

As such, funflow provides some facility, which we present in this tutorial, with which a pipeline may be programmed to handle errors gracefully.

## Requirements

This tutorial will use the following language extensions...

:opt no-lint
{-# LANGUAGE Arrows #-}               -- pipeline flow syntax
{-# LANGUAGE TypeApplications #-}     -- declaring exception types to handle
{-# LANGUAGE OverloadedStrings #-}    -- polymorphic String typing

...and imports...

import Control.Exception.Safe (SomeException)
import qualified Data.CAS.ContentStore as CS
import Funflow (Flow, pureFlow, ioFlow, dockerFlow, throwStringFlow, returnFlow, tryE)
import Funflow.Tasks.Docker (DockerTaskConfig (DockerTaskConfig), DockerTaskInput (DockerTaskInput), args, argsVals, command, image, inputBindings, Arg(Placeholder))

## Handle a task that can fail

Let's write a flow, based on a DockerTask, which will fail:

someFlowThatFails :: Flow DockerTaskInput CS.Item
someFlowThatFails = dockerFlow DockerTaskConfig{ image = "badImageName", command = "badCommand", args = [Placeholder "missingArgument"] }

As may be clear, this flow demonstrates several reasons a flow may fail:

• the Docker image name does not exist or cannot be pulled
• improper argument configuration (e.g., a placeholder isn't filled)
• the command does not exist
• command execution fails

In fact, if we were to try to run this flow directly with runFlow or runFlowWithConfig, nothing in a container would ever be done. An attempt to fetch "badImageName" would fail, but execution of the above flow would actually never proceed to the fetch attempt. Instead, the flow would fail at configuration time, as the task was being "interpreted," since we've not defined how to fill the placeholder "missingArgument".

To prepare a flow for exception(s), we can use funflow's tryE function, which broadens the output type to accommodate potential for specific exception types. Specifically, tryE transforms a Flow i o that can throw an exception of type ex into a flow of type Flow i (Either ex o). This means that the result of the flow is either the exception on the left or the result on the right. This transformation means that occurrence of some types of exception will no longer be fatal! Instead, the flow will be able to produce an exception of the declared type as a value, allowing downstream logic to process it and act accordingly.

flow :: Flow () String
flow = proc () -> do
  -- Try to run a flow that fails, receive the result in an @Either SomeException ()@
  result <- tryE @SomeException someFlowThatFails -< DockerTaskInput {inputBindings = mempty, argsVals = mempty}
  case result of
    Left _ ->
      returnFlow -< "The task failed"
    Right _ ->
      returnFlow -< "The task succeeded"

In this example, to catch any exception, we use tryE to wrap execution of the Docker task. We can then decide how the rest of the pipeline behaves, since we've catpured the result--exceptional or not--in a value of a broadened result type. We could, for instance, in case of an exception, write logging information to a file and/or run cleanup operations.

Notice that we had to specify which type of exceptions will be handled: tryE @SomeException.... As such, tryE requires explicit specification regarding the extent of exception types the pipeline will handle; in turn, this feature requires the TypeApplications extension.

Also note that the several potential failure causes mentioned above are ordered chronologically by when each may be encountered during an run attempt. Because the first two--unknown image name and illegal configuration--occur during task interpretation, before actual execution begins, those error causes cannot be protected by tryE. They may be handled with ordinary machinery for exceptional control flow, though, e.g. using the Control.Exception API. In addition to more standard exceptions relating to things like filesystem permissions and network requests, funflow provides some custom error types around which you may wish to program a flow, particularly those in Data.CAS.ContentStore, Data.CAS.RemoteCache, and Docker.API.Client.

## Throwing exceptions in a workflow

A task will may fail on its own, when an exception arises. Sometimes, however, you may want to decide that a task has failed, based on the actual value of a computed result.

In funflow, it is possible for you to throw an exception manually, using throwStringFlow...

flowThatFails :: Flow () ()
flowThatFails = proc () -> do
  -- Do some computations first (this is a dummy example)
  result <- pureFlow id -< ()
  -- Depending on the result, throw an exception
  if result == ()
    then throwStringFlow -< "Nothing has been done (as expected for this example)"
    else returnFlow -< ()

...you can then handle the failure as previously demonstrated...

flow' :: Flow () ()
flow' = proc () -> do
  -- Try to run a flow that fails, receive the result in an @Either SomeException ()@
  result <- tryE @SomeException flowThatFails -< ()
  -- Handle result as previously shown
  case result of
    Left exception ->
      (ioFlow $ \exception -> putStrLn $ "Exception caught: " ++ show exception) -< exception
    Right () ->
      ioFlow $ const $ error "Exception not caught" -< ()