1.x: observeOn + immediate scheduler to be a request rebatcher

[akarnokd]

This PR removes the immediate() scheduler "optimization" from observeOn and treats it as a common scheduler. Since observeOn has a stable request pattern, this turns it into a rebatching operator. No matter what the downstream requests, the upstream will requests of the specified size (with 25% as low water mark; i.e., replenishment after 75%). Since immediate is synchronous, this will run the drain loop, non-reentrant, on the caller thread.

I found this mode of operation very handy in my Reactive-RPC prototype and a simple streaming echo RPC call; it prevents going unbounded and bloating the message sender threads:

// remote
public Observable<Integer> echo(RpcStreamContext<?> ctx, Observable<Integer> in) {
   return in.observeOn(Schedulers.immediate(), 16);
}

// client
api.echo(Observable.range(1, 100_000)).observeOn(Schedulers.immediate(), 32)
.subscribe(System.out::println, Throwable::printStackTrace);

[davidmoten]

Great! What about adding a method to Observable that calls it so this functionality is discoverable? Say Observable.batchRequests(n)?

[akarnokd]

That could be a follow-up PR if this gets accepted.

[zsxwing]

Why not just adding a batchRequests operator instead of changing the existing behavior? This probably affects some users' applications if they depends on the optimization.

Secondly, using observeOn(Schedulers.immediate(), 16) and assuming how it implements is not reliable. This is not an API contract and it could be changed in any time.

[akarnokd]

The alternative requires duplicating code, which triggers more outcry usually...

[zsxwing]

I think you can add a flag (e.g., disallowOptimization) to OperatorObserveOn and use it for batchRequests.