future methods on iterator helpers

[bakkot]

@michaelficarra and I have been thinking about what other iterator helper methods we'd like to pursue, and have the following extremely tentative list. This doesn't represent the opinion of TC39 as a whole - it's just what we are personally interested in. This is not necessarily actionable for you; I just wanted to give a heads up.

• zip / zipWith / (maybe) zipLongest
• chunks / windows (chunks would group outputs into a chunk every n items, windows would give you a sliding window of the last n items)
• concat
• join (like on Array.prototype)
• tap
• some sort of cleanup (analogous to this proposal's finally; ideally these would be designed in concert)
• of (like on Array)
• takeWhile / dropWhile
• scan
• into (i.e. function(arg) { return arg(this); } - handy for chaining / custom operators; RxJS spells this pipe)

[benlesh]

So I can give you some information about what I've seen at scale, with some anecdotes from Google's HUGE number of build targets using RxJS.

Observable should also have catch, I think that's an oversight @domfarolino. Observable<C> catch((Error) => Observable<C>)

From the above list

Some are popular some are not.

Very Popular

• tap
• of
• scan
• concat

Somewhat Popular

• takeWhile

Rarely Used

• zip: Its sister operation combineLatest is MUCH more popular...

Almost never used

• window: For whatever reason, window wasn't used at all across thousands and thousands of build targets. However buffer is... which is basically: source.window().flatMap(window => window.toArray())

Async Temporal Operators

There's a bunch of operations that can be done over async collections (AsyncIterator or Observable) that can't be done on sync collections like arrays or iterables.

These are the most popular by far:

• combineLatest
• debounce
• throttle
• buffer(time)
• timeout

Other popular operations:

There's some other very popular operations that might work on any of the types in question, in particular Observable and AsyncIterator but maybe also Iterator. Although the fact that the TC39 hinged their implementation on the iterator and not on the iterable might hinder this a bit:

• retry: If there's an error, reset and start over.
• repeat: When you're done, reset and start over.

@bakkot I'm happy to help with your AsyncIterator helpers proposal... if it's not too late.

[bakkot]

Thanks for the info @benlesh! Some thoughts:

zip: It's sister operation combineLatest is MUCH more popular...

That was initially surprising to me, but thinking about it more it makes sense because observables are so often treated as being essentially unordered. For iterators (async or otherwise) I think it's more usual to rely on the order, so zip makes more sense there.

window

Yeah, I also don't use this much.

Async Temporal Operators

I do like these, but I'd like to start by getting generic function-based throttle and debounce in the language, for regular functions, and then worry about adding them to Observable specifically.

retry/repeat

Yeah, I can't think of how those could possibly make sense on iterators. Seperately, they're also kind of scary - I'm not sure I'd want to make those operations easy to reach for.

---

I'm happy to help with your AsyncIterator helpers proposal... if it's not too late.

It's definitely not too late for input on details, though for the initial version we're pretty settled on the high-level design and the set of combinators (i.e., just those in the initial version of the iterator helpers proposal, plus something like bufferAhead and maybe something to help limit concurrency). I've appreciated the feedback you've given in the issue tracker so far.

[benlesh]

retry/repeat
Yeah, I can't think of how those could possibly make sense on iterators. Seperately, they're also kind of scary - I'm not sure I'd want to make those operations easy to reach for.

So, the uses for retry and repeat are interesting and different.

retry is VERY common and useful. If you have a websocket wrapped in an Observable, or an HTTP request, then retry, particularly with some arguments can allow all sorts of valid retrying behaviors.

repeat has a few use cases where you'd combine it with takeUntil or take, etc. This allows the user to compose pretty complex operations and have them repeat over and over.

Between the two, retry is easily the most common and well used. In particular for fetching things over a network. timeout is another that's pretty common for that scenario.

[a-laughlin]

into

+1 on into/pipe. It reduces and simplifies the api surface for learnability, reduces future name conflicts, enables custom operators (e.g., combineLatest), simplifies operator composition, and enables other utility libraries to fill in api gaps without waiting for language additions.

Separately, if I understand correctly, the promisifying methods combine concerns that result in redundant functionality and preventable naming challenges. Consider separating the concerns of promisification from the concerns of what gets promisified? e.g., with a method called toPromise. Examples:

first (or nth)

element.on('click').into(
  take(1),
).toPromise();

composing custom operations like preventDefault

element.on('click').into(
  tap(e=>{e.preventDefault()}),
  take(1),
).toPromise();

creating custom operators like find (using flow forward composition semantics)

const find = (predicate)=>flow(
  filter(predicate),
  take(1),
);

element.on('click').into(
  find(...),
).toPromise();

[benlesh]

In a (probably poorly conducted) poll of ~800 respondents on what RxJS operators they use, The top twenty are:

Operator	Percentage (%)
map	83.5443038
switchMap	71.39240506
filter	69.36708861
combineLatest	66.32911392
tap	64.30379747
takeUntil	58.48101266
catchError	49.11392405
debounceTime	41.01265823
mergeMap	38.48101266
take	38.35443038
forkJoin	33.41772152
concatMap	28.10126582
merge	23.29113924
first	21.64556962
concat	17.21518987
firstValueFrom	16.32911392
finalize	15.56962025
lastValueFrom	13.67088608
scan	12.40506329

Overall, this surprises me a little bit in some cases.

I knew combineLatest was popular, but I didn't think quite so many people were using it. combineLatest simply combines the latest values from all observables passed to it and emits them, only emitting once it has at least one value from each.

forkJoin is also interesting... for context here, it's basically the Observable equivalent of Promise.all. It emits an array of the last values of all observables passed to it when all observables complete.

firstValueFrom and lastValueFrom are both functions for converting an Observable to a Promise, by taking the first value or the last value respectively.

[mfp22]

forkJoin is also interesting... for context here, it's basically the Observable equivalent of Promise.all. It emits an array of the last values of all observables passed to it when all observables complete.

forkJoin is really popular with Angular devs because they don't know what they're doing. They actually want combineLatest but they have a thing where they want to sometimes not worry about having to unsubscribe if the only sources come from HTTP requests, which is a losing battle and a generally unsafe habit, probably supporting imperative code as well (from manual subscriptions), and makes the code annoying to refactor if a data source changes to emitting more than 1 value (auto-refreshing data source could be one example). I don't know why it became popular, but probably an influential article or two early in Angular 2 days probably started it.