[mono][interp] Improve tiering performance (#70649)

* [mono][interp] Don't allocate some vars as execution stack

It is not really true and it serves no purpose here.

* [mono][interp] Print code before any optimizations take place

Fix bitrotten mono_interp_print_td_code. Don't print IL_SEQ_POINT opcodes since they are too noisy.

* [mono][interp] Use td->optimized directly in more places

* [mono][interp] Add dummy MINT_LDNULL instruction

We were pushing local that wasn't defined by any instruction, potentially confusing the var offset allocator.

* [mono][interp] Add fast offset allocator, to be used by unoptimized code

This is the old offset allocation scheme that we were using originally, before the var offset allocator was added. Vars have the same offset as they would have in IL code, based on their position on the execution stack at the moment when they are pushed. Whenever we push/pop on the execution stack we keep track of the used stack size. Every var pushed on the execution stack will therefore have this stack_offset remembered. Once the entire IL code is traversed and we have all the global locals allocated, the real offset of the execution stack locals can be determined. It is computed as the originally determined stack_offset added with the offset of the execution stack start (size of the global locals space).

With this offset allocator, calls no longer need to store all the call args sregs and the return_offset is always the same as call_args_offset. This is because all vars are directly placed in the right position and no optimizations can move them around. The offset of the return value will therefore be also the offset where all the args are placed.

The limitation with this way of allocating offsets is that we run into the same problems with opcodes that don't have typical stack usage (use values, pop them, store result). This happens with newobj opcodes. The opcode receives the params, and then it needs to call a ctor with these same params and a newly allocated this object. Since we can't use a var offset allocation pass to compute the offset ideally, the newobj opcodes in the case of unoptimized code must move these params around on the stack, in order to make room for `this`.

* [mono][interp] Add dreg to all calls in unoptimized code

All calls need to have a dreg (a dummy one if it is void call), in order for unoptimized offset allocator to determine the offset of the call. In unoptimized code, the offset of the first argument is always the same as the offset of the return, if any.

* [mono][interp] Fix issue with passing of exvars

Unoptimized code can't use a global local directly (like the exvar), it must first be pushed to a new var on the execution stack. Add a mov instruction when we start executing the basic block for a handler.

Author: BrzVlad

src/mono/mono/mini/interp/interp.c

@@ -5374,6 +5374,19 @@ MINT_IN_CASE(MINT_BRTRUE_I8_SP) ZEROP_SP(gint64, !=); MINT_IN_BREAK;
 			ip += 4;
 			goto call;
 		}
+		MINT_IN_CASE(MINT_NEWOBJ_STRING_UNOPT) {
+			// Same as MINT_NEWOBJ_STRING but copy params into right place on stack
+			cmethod = (InterpMethod*)frame->imethod->data_items [ip [2]];
+			return_offset = ip [1];
+			call_args_offset = ip [1];
+
+			int param_size = ip [3];
+                        if (param_size)
+                                memmove (locals + call_args_offset + MINT_STACK_SLOT_SIZE, locals + call_args_offset, param_size);
+			LOCAL_VAR (call_args_offset, gpointer) = NULL;
+			ip += 4;
+			goto call;
+		}
 		MINT_IN_CASE(MINT_NEWOBJ) {
 			MonoVTable *vtable = (MonoVTable*) frame->imethod->data_items [ip [4]];
 			INIT_VTABLE (vtable);
@@ -5474,6 +5487,49 @@ MINT_IN_CASE(MINT_BRTRUE_I8_SP) ZEROP_SP(gint64, !=); MINT_IN_BREAK;
 			ip += 4;
 			goto call;
 		}
+		MINT_IN_CASE(MINT_NEWOBJ_SLOW_UNOPT) {
+			call_args_offset = ip [1];
+			guint16 param_size = ip [3];
+			guint16 ret_size = ip [4];
+			gpointer this_ptr;
+
+			// Should only be called in unoptimized code. This opcode moves the params around
+			// to compensate for the lack of use of a proper offset allocator in unoptimized code.
+			gboolean is_vt = ret_size != 0;
+			if (!is_vt)
+				ret_size = MINT_STACK_SLOT_SIZE;
+
+			cmethod = (InterpMethod*)frame->imethod->data_items [ip [2]];
+
+			MonoClass *newobj_class = cmethod->method->klass;
+
+			// We allocate space on the stack for return value and for this pointer, that is passed to ctor
+			if (param_size)
+				memmove (locals + call_args_offset + ret_size + MINT_STACK_SLOT_SIZE, locals + call_args_offset, param_size);
+
+			if (is_vt) {
+				this_ptr = locals + call_args_offset;
+				memset (this_ptr, 0, ret_size);
+				call_args_offset += ret_size;
+			} else {
+				// FIXME push/pop LMF
+				MonoVTable *vtable = mono_class_vtable_checked (newobj_class, error);
+				if (!is_ok (error) || !mono_runtime_class_init_full (vtable, error)) {
+					MonoException *exc = interp_error_convert_to_exception (frame, error, ip);
+					g_assert (exc);
+					THROW_EX (exc, ip);
+				}
+				error_init_reuse (error);
+				this_ptr = mono_object_new_checked (newobj_class, error);
+				mono_interp_error_cleanup (error); // FIXME: do not swallow the error
+				LOCAL_VAR (call_args_offset, gpointer) = this_ptr; // return value
+				call_args_offset += MINT_STACK_SLOT_SIZE;
+			}
+			LOCAL_VAR (call_args_offset, gpointer) = this_ptr;
+			return_offset = call_args_offset; // unused, prevent warning
+			ip += 5;
+			goto call;
+		}
 		MINT_IN_CASE(MINT_INTRINS_SPAN_CTOR) {
 			gpointer ptr = LOCAL_VAR (ip [2], gpointer);
 			int len = LOCAL_VAR (ip [3], gint32);

src/mono/mono/mini/interp/mintops.def

@@ -338,6 +338,8 @@ OPDEF(MINT_JMP, "jmp", 2, 0, 0, MintOpMethodToken)
 
 OPDEF(MINT_ENDFILTER, "endfilter", 2, 0, 1, MintOpNoArgs)
 
+OPDEF(MINT_NEWOBJ_SLOW_UNOPT, "newobj_slow_unopt", 5, 1, 0, MintOpMethodToken)
+OPDEF(MINT_NEWOBJ_STRING_UNOPT, "newobj_string_unopt", 4, 1, 0, MintOpMethodToken)
 OPDEF(MINT_NEWOBJ_SLOW, "newobj_slow", 4, 1, 1, MintOpMethodToken)
 OPDEF(MINT_NEWOBJ_ARRAY, "newobj_array", 5, 1, 1, MintOpMethodToken)
 OPDEF(MINT_NEWOBJ_STRING, "newobj_string", 4, 1, 1, MintOpMethodToken)