perl

Migrating To async/await (part 2)

Tech@Deriv

Mar 10, 2020 • 6 min read

Things to consider when rewriting code to use the new async/await syntax provided by Future::AsyncAwait.

In part 1 we looked at how to rewrite code that previously used plain Future on its own for sequencing operations, into using the neater async/await syntax provided by Future::AsyncAwait. In this part, we'll take a look at how to handle the various forms of repeating and iteration provided by Future::Utils::repeat. We'll conclude by taking a look at how to handle conditionals and concurrency structures.

`repeat` as a `while` loop

Using plain Future the structure of repeating loops such as while loops can be hard to express, and so the Future::Utils module provides a handy utility, repeat, to help write these. Now that we are able to use full async/await syntax within the code, we don't have to use this, and instead we can write a regular while loop in regular Perl code, with await expressions inside it.

A regular Perl while loop has its condition test at the start as compared to the repeat loop testing the condition after the first iteration, so it is often neatest to write it using a while(1) loop and an explicit last to break out of the loop once the condition is satisfied.

# previously
use Future::Utils 'repeat';
sub example {
  repeat {
    FIND_NEXT_THING();
  } while => sub { defined $_[0] };
}

# becomes
async sub example {
  while(1) {
    my $thing = await FIND_NEXT_THING();
    last if !$thing;
  }
}

In a repeat loop, it is common to use the "previous attempt future" passed in as the first argument in order to detect whether this attempt is the first one or not, to decide whether to apply a delay time before a subsequent attempt. In this case, using the last loop control of a regular while(1) loop allows you to just skip over such a delay:

# previously
use Future::Utils 'repeat';
sub example {
  return repeat {
    my $prevf = shift;
    ($prevf ? $loop->delay_future(after => 2) : Future->done)
      ->then(sub {
        return TRY_DO_TASK();
      })
  } until => sub { $_[0]->get->is_success };
}

# becomes
async sub example {
  while(1) {
    my $result = await TRY_DO_TASK();
    last if $result->is_success;
    
    await $loop->delay_future(after => 2);
  }
}

`repeat` as a `foreach` loop

Another use for repeat is to create an iteration loop over a set of values. As before, we can write this using an await expression inside a regular Perl foreach loop.

# previously
use Future::Utils 'repeat';
sub example {
  my @parts = ...;
  return repeat {
    my $idx = shift;
    return PUT_PART($idx, $parts[$idx]);
  } foreach => [0 .. $#parts];
}

# becomes
async sub example {
  my @parts = ...;
  foreach my $idx (0 .. $#parts) {
    await PUT_PART($idx, $parts[$idx]);
  }
}

Sometimes the repeat ... foreach loop is accompanied by an otherwise argument to give some code for what happens when the loop finishes - perhaps it generates some sort of error condition. Plain Perl foreach loops don't have an exact equivalent, but in the likely case that the loop is the final piece of code in the function, a return expression can be used to provide the result, and the failure handling code can be put after the loop.

# previously
use Future::Utils 'repeat';
sub example {
  return repeat {
    return TRY_DO_THING();
  } foreach => [1 .. 10],
    while => sub { !$_[0]->get->is_success },
    otherwise => sub {
      return Future->fail("Unable to do the thing in 10 attempts");
    };
}

# becomes
async sub example {
  foreach my $attempt (1 .. 10) {
    my $result = await TRY_DO_THING();
    return $result if $result->is_success;
  }
  
  Future::Exception->throw("Unable to do the thing in 10 attempts");
}

The `fmap` family of functions

The fmap function and similarly-named functions provide a concurrent-capable version of the map Perl operator. Because it takes an extra argument to control the level of concurrency, and as currently await expressions don't work inside Perl's map (RT129748), it is best to continue to use fmap and similar where possible.

Don't forget that since fmap returns a Future you can still await on the result, and that the loop body can be an async sub, allowing you to use await expressions inside it.

# previously
use Future::Utils 'fmap1';
use List::Util 'sum';
sub example {
  my @uids = ...;
  return (fmap1 {
    my $uid = shift;
    return GET_USER($uid)->then(sub {
      my ($info) = @_;
      return GET_USER_BALANCE($info);
    });
  } foreach => [@uids], concurrent => 10)->then(sub {
    my @balances = @_;
    return Future->done(sum @balances);
  });
}

# becomes
use Future::Utils 'fmap1';
use List::Util 'sum';
async sub example {
  my @uids = ...;
  my @balances = await fmap1(async sub {
    my $uid = shift;
    my $info = await GET_USER($uid);
    return await GET_USER_BALANCE($info);
  }, foreach => [@uids], concurrent => 10);
  
  return sum @balances;
}

# previously
sub example {
  return Future->wait_any(
    $loop->timeout_future(after => 10),
    GET_THING(),
  )->then(sub {
    my ($thing) = @_;
    ...
  });
}

# becomes
async sub example {
  my ($thing) = await Future->wait_any(
    $loop->timeout_future(after => 10),
    GET_THING(),
  );
  ...
}

Sometimes the "main" path of a call to Future->wait_any is not just one function call, but composed of multiple operations, perhaps in a sequence of ->then chains, or a repeat loop. In this case it is a little more difficult to rewrite that path into async/await syntax because an await expression would cause the entire containing function to pause, and in any case does not yield a Future value suitable to pass into the ->wait_any.

In this case, we can use an inner async sub to contain the main path of code with await expressions, and immediately invoke it.

# previously
use Future::Utils 'repeat';
sub example {
  return Future->wait_any(
    $loop->delay_future(after => 10)
      ->then_fail("Failed to get item in 10 seconds"),
      
    (repeat {
      return GET_ITEM()
    } until => sub { $_[0]->get->is_success }),
  );
}

# becomes
async sub example {
  await Future->wait_any(
    $loop->delay_future(after => 10)
      ->then_fail("Failed to get item in 10 seconds"),
      
    (async sub {
      while(1) {
        my $result = await GET_ITEM();
        return $result if $result->is_success;
      }
    })->(),
  );
}

Remember that the return inside the foreach loop makes its containing function return, i.e. the anonymous async sub that is invoked as the second argument to Future->wait_any. This makes it convenient to finish the loop there.

Conditionals and Concurrency structures

Conditionals

One particularly awkward code structure to write using simple Future is how to perform a future-returning action conditionally in a sequence of other steps. The usual workaround often involves testing if the condition holds and, if not, waiting on a "dummy" immediate future to cover the case that the conditional code isn't being invoked. Because await can be placed anywhere inside an async sub, including inside a regular if block, these awkward structures can be avoided and much simpler code written instead.

# previously
sub example1 {
  return ( $COND ? MAYBE_DO_THIS() : Future->done )->then(sub {
    return DO_SECOND();
  });
}

sub example2 {
  return DO_FIRST()->then(sub {
    return Future->done unless $COND;
    OTHER_STEPS();
    return MAYBE_DO_THIS();
  })->then(sub {
    DO_SECOND();
  });
}

# becomes
async sub example1 {
  await MAYBE_DO_THIS() if $COND;
  return await DO_SECOND();
}

async sub example2 {
  await DO_FIRST();
  if ($COND) {
    OTHER_STEPS();
    await MAYBE_DO_THIS();
  }
  return await DO_SECOND();
}

The await keyword can also be used inside the condition test for an if block:

# previously
sub example {
  my ($uid) = @_;
  
  return GET_USER_INFO($uid)->then(sub {
    my ($info) = @_;
    return Future->done if !$info;
    return PROVISION_USER($info);
  });
}

# becomes
async sub example {
  my ($uid) = @_;
  
  if(my $info = await GET_USER_INFO($uid)) {
    await PROVISION_USER($info);
  }
}

Convergent Flow

Often in future-based code, there is some concurrency of multiple operations converging in one place by using Future->needs_all or related functions. Since this constructor returns a future, we can use await on it much like any other. Since the results are returned in a same-ordered list as the futures it is waiting on, it is simple enough to unpack these in a list assignment:

# previously
sub example {
  return Future->needs_all(
    GET_X(),
    GET_Y(),
  )->then(sub {
    my ($x, $y) = @_;
    ...
  });
}

# becomes
async sub example {
  my ($x, $y) = await Future->needs_all(
    GET_X(), GET_Y(),
  );
  ...
}

Similarly, uses of Future->wait_any as for example used to implement a timeout, can use await on that:

# previously
sub example {
  return Future->wait_any(
    $loop->timeout_future(after => 10),
    GET_THING(),
  )->then(sub {
    my ($thing) = @_;
    ...
  });
}

# becomes
async sub example {
  my ($thing) = await Future->wait_any(
    $loop->timeout_future(after => 10),
    GET_THING(),
  );
  ...
}

In this case, we can use an inner async sub to contain the main path of code with await expressions, and immediately invoke it.

# previously
use Future::Utils 'repeat';
sub example {
  return Future->wait_any(
    $loop->delay_future(after => 10)
      ->then_fail("Failed to get item in 10 seconds"),
      
    (repeat {
      return GET_ITEM()
    } until => sub { $_[0]->get->is_success }),
  );
}

# becomes
async sub example {
  await Future->wait_any(
    $loop->delay_future(after => 10)
      ->then_fail("Failed to get item in 10 seconds"),
      
    (async sub {
      while(1) {
        my $result = await GET_ITEM();
        return $result if $result->is_success;
      }
    })->(),
  );
}

Note: This post has been ported from https://tech.binary.com/ (our old tech blog). Author of this post is https://metacpan.org/author/PEVANS

repeat as a while loop

repeat as a foreach loop

The fmap family of functions