Inadvertent exporting of Wasm functions

[fgmccabe]

The anyref/anyfunc allows a function to exported from a WASM module dynamically - as part of the returned value of another exported function. Similarly for imports.
This effectively blows apart any discipline for what is exported from a WASM module. It also complicates implementation because potentially every WASM function may become accessible from JS.

The table.set operator in conjunction with indirect calling of functions from any table allows monkey-patching of functions.

[lukewagner]

Technically it's not any function; just functions that are statically referrenced by ref.func. Tables are already readable/writable from JS (via Table.prototype.get and set), so once a ref.funcd function is stored in a table, it's no different than if it was added to the table via elem segment.

Maybe the issue you're getting at, though, which I do think we should consider, is that, in the MVP, we know all the external (exported or stored in an external table) functions before the code section starts, whereas with ref.func, we'll only know this by the end of the code section. This could negatively impact streaming parallel compilation. Given that we explicitly moved the elem section from after to before the code section (way back when) specifically so that it was known before the code section, we might want to consider adding some sort of "aliasable" section which declares those functions (and later, other kinds of definitions too!) which were allowed to be ref.funcd.

[titzer]

I agree with @lukewagner here. It'd be nice to know up front which functions are used in a first-class way, before the code section.

[rossberg]

So, if I was a lazy producer that simply put all their functions into this section, what would be the penalty?

[titzer]

On V8, the penalty is that we generate wrapper code per signature (essentially a machine code stub whose complexity is proportional to the number of arguments) plus a JavaScript function per Wasm function (minimum two objects--a JSFunction and a SharedFunctionInfo). We're working to make the generation of those wrappers cheaper (and happen in parallel), but the JSFunctions we'd need to do lazily to avoid the upfront cost.

[rossberg]

Hm, how difficult would a lazy approach be for an implementation?

An extra section adds somewhat non-trivial language complexity in this case: we would need to decode that section, validate it, extend static function contexts to track the information whether a func is "first-class", employ that to validate uses of func.ref, etc. All doable, but somewhat ugly and ad-hoc.

Shame, I suppose a custom section for this sort of (primarily JS-relevant?) optimisation hint would not cut it, since you cannot rely on its being accurate.

[lukewagner]

Indirect callability could end up influencing codegen for streaming non-JS hosts as well. I would think, spec-wise, this could be handled as a purely binary encoding detail, similar to how we did with the data segments up-front size declaration.

[rossberg]

Yeah, I'm not sure. The "binary encoding detail" approach already became slightly hacky for the simple data count section. I suspect it would be borderline for something like this, which requires more involved validation. I'll think about it.

[lukewagner]

I'm about to implement this; which means either harvesting ref.funcd func-indices after-the-fact or adding a new declarative section, so it'd be nice to figure this out.

I still think we should maintain the general property that all aliased functions are known before the code section. I realized that aliased functions are not just externally callable (requiring entry stubs, which should be lazily generated for a couple of reasons) but also indirectly callable (since any funcref can be put into a table). For some impls, indirect calls may have a slightly different ABI with a second prologue that immediately precedes the normal prologue of indirectly-callable functions. Without an up-front declaration, streaming/parallel compilation would need to conservatively give every function an indirect call prologue, increasing codegen size. It's a modestly-compelling example, but given that we've already ensured this property in the MVP, I think it makes sense to continue to ensure it when adding features.

Any new thoughts @titzer @rossberg ?

[rossberg]

Ah, thanks for the reminder. I have been throwing this around in my head for a while and so far think that a clean and forward-looking solution involves the following:

• Adding a "reference section" to the binary format. Its structure follows that of the export section, in that it can (potentially) list entities of different sort. For now we would only allow functions.

• Extending the abstract syntax with "reference declarations", again similar to export declarations.

• Validation would check that (a) all reference declarations refer to valid indices, and (b) that every use of ref.func (or similar instructions in the future) only mentions indices declared as references. To that end, validation would require to track the set of declared reference indices as an additional context component.

• There would probably be a bunch of ad-hoc exceptions for uses of ref.func outside function bodies, i.e., in element segments, which don't require a priori declaration?

• Since they become part of the AST, reference declarations would also need to occur in the text format, which, among other things, allows writing proper tests for this feature. We could add the same shortcuts as for exports.

I also have considered the idea of only making it a detail of the binary format. But it seems that would be stretching it too far in this case, since the validation conditions are substantially more extensive than just comparing two numbers.

WDYT?

[rossberg]

FWIW, another option might be to piggyback on (and abuse) segments and the validation exception that we'll probably need to introduce for them anyway. That is:

• Validation requires that ref.func only mentions functions that are also referenced inside an element segment. (For ref.func occurring in segments this condition holds vacuously.)

• To avoid redundant allocation for auxiliary forward declaration segments, introduce a third kind of segment, unused (or declared), which is like a passive segment in that it does not perform any initialisation, but like an active one in that it is dropped implicitly.

This solution is a bit more hacky, but would require fewer extensions to AST, binary, and text format (only adding the new segment kind).

[lars-t-hansen]

I think using element segments came up before but was shot down as too hacky. The declared bit (expressible more to the point?) reduces the hackiness by quite a lot, I think. If other kinds of values require a reference section in the future so be it, but until then we keep the complexity down.

[rossberg]

Well, I wouldn't want to end up in a future where we have both. If we go for the segment approach, then we should design it such that the new segment kind is extensible to other reference kinds later.

[lukewagner]

@rossberg Your "reference section" idea is exactly what I was imagining.

Regarding binary-encoding vs. validation: I think it'd be kindof a pain to have to forward-declare references in the text format, but if we have them implicitly declared/unified (as we do with function types), then we won't be able to write negative tests. If the distinction is not present in the text format, I'd be inclined to keep it in the binary format as well, but no strong opinion here.

Regarding the elem section idea: scanning the elem section encoding just now, it's already a bit hairy, so I'd be reticent to add another case. Plus, it raises some questions without very satisfying answers:

• Is every index used by the elem section implicitly declared referenceable or is it just the indices in the new "declared" segments? If the former, we conflate "stored in table" with "first-class referenceable" (the former can be better-optimized in some cases); if the latter then we still have the special-case exception for indices in the non-"declared" elem segments.
• Are the referenceable indices stored as constant expressions or just indices? If the former, then the size is ~3x and it also doesn't really feel like a declaration. If the latter, then it's not really symmetric with the other 3 cases of elem segments.

[rossberg]

@lukewagner, wasn't the text format meant to be a faithful representation of the AST, plus some sugar? That implies that everything expressible in the AST should be expressible in the text format. Making ref decls implicit would violate that (unlike implicit function types, which are just a shorthand).

Analogous to export, I'd imagine a declaration

(ref (func $f))

plus the shorthand

(func $f (ref) (param i32) ...)

Regarding elem sections, I think question (1) may have to be answered either way: does the ref section have to repeat indices already mentioned in a segment? If no then the segment approach would actually make that more uniform. For (2), I was imagining something like externkind vec(idx), closer to the existing format of active segments.

I don't understand the distinction you are making between "stored in table" and "first-class". Under this proposal, you can always emulate ref.func via elem section + table.get, so how could an engine treat these cases differently? (Without analysing all the code, because then you wouldn't need the forward declaration in the first place.)

But in general, I agree that a proper reference section would be cleaner.