Expose marker data

[paulsonnentag]

This pull request changes two things:

• Adds paperNumber property to markers
• Paper Programs detects the markers by size and not by the fact that they are not part of a shape. When the user calibrates the colors of the markers, Paper Programs stores the size of the markers as well. Dots which are significantly larger than the paper dots are recognized as markers. It's no longer necessary to filter the markers for size in the paper program to get the real markers.

I haven't implemented the local position conversion, because I realized that it's possible that a paper has multiple canvases with different sizes. I think it's better to have a utility function to convert global coordinates to local coordinates. I want to wait until #44 is merged which restructures paper.js to allow imports.

[shaunlebron]

I realized that it's possible that a paper has multiple canvases with different sizes. I think it's better to have a utility function to convert global coordinates to local coordinates

I was wanting something like this too. Might make sense to attach a projection matrix to each canvas returned by paper.get('canvas', ...)

[shaunlebron]

we can use Math.sign

[paulsonnentag]

Ah thanks, didn't know about that

[janpaul123]

Minor comment, looks good overall. Thanks for this! Let me know when I should merge this.

[janpaul123]

if(sideB.y < 0 && sideA.y > 0)?

[paulsonnentag]

I need some help with transforming the point to local coordinates. I don't really understand how the matrix calculations work. I thought I could do it like this:

const projectionMatrix = forwardProjectionMatrixForPoints(program.points)
const relativePosition = projectPoint(marker.position, projectionMatrix)

Both marker.position and program.points are the original coordinates from detectProgram (value range between 0 and 1). I expected to get a transformed position relative to the top left of the paper (value range between 0 and 1) but this doesn't work.

[shaunlebron]

I learned about it mainly from the figure below in this paper, and experimented with it in this notebook:

• case 1 is a forward projection
• case 2 is an inverse projection
• case 3 is an arbitrary projection created by multiplying matrix 1 and 2 (neat!)

The forward projection (case 1 above) maps unit-square coords to those inside the given quad corners. For example, it maps 1,1 to corner 2.

In context, if you take a local coord inside the paper's canvas, divide it by the canvas size, then project it with the result of forwardProjectionMatrixForPoints you will get its coordinates on the supporter canvas.

This marker problem requires the inverse (case 2 above). Luckily, the paper describes that the inverse of a projection mapping is itself a projection mapping, and it can be created by taking the adjugate:

const forward = forwardProjectionMatrixForPoints(program.points);
const inverse = forward.adjugate();

let relativePosition = projectPoint(marker.position, inverse);
relativePosition = mult(relativePosition, {x: canvas.width, y: canvas.height}); // rescale

And actually, you can avoid that rescaling part by multiplying it with a "canvas size" matrix, though I can never remember the right order (sorta like case 3 above). Example is here though:

paperprograms/client/projector/Program.js

Lines 179 to 181 in c4eeb5a

	forwardProjectionMatrixForPoints(
	program.points.map(point => mult(point, { x: this.props.width, y: this.props.height }))
	).multiply(getCanvasSizeMatrix(width, height))

[janpaul123]

@paulsonnentag would you like me to merge this PR and then follow up with local coordinates in a second PR (after you figure out the matrix stuff) or want to do it all at once?

[paulsonnentag]

I would like to finish the local coordinates first and also update the documentation

[paulsonnentag]

@janpaul123 ready to merge

[janpaul123]

So cool. Can't wait to try this out soon!