It appears to be possible to embed real-time data into a livestream via embedded 608/708 closed-caption data:
This opens a few interesting possibilities:
- Clipfarm, when running in server-side clipping mode, could get match start/end data synchronized to the stream, avoiding the need for image recognition/OCR to determine clipping timestamps
- Data could be integrated into a a purpose-built FTC livestream viewer, allowing for additional, interactive features.
- For example, you could have it so hovering over/clicking a team number on the scoring overlay pops up information about that team, such as team name, location, win/loss/tie record, OPR, etc.
- Another example is having explainers when hovering over/clicking on various parts of the scoring overlay, explaining what they mean and how they affect gameplay
There are probably also other interesting and exciting possibilities this could unlock.
According to https://web.archive.org/web/20230305210310/https://webcaptioner.com/help/integrations/obs, it should be possible to send caption data to OBS via the OBS WebSockets connection we are already using. From one of the images in that webpage, it appears the format is quite simple:
The source code of Webcaptioner is also available here, as a reference: https://github.com/curtgrimes/webcaptioner
Note: It seems like SendStreamCaption is rather limited, when compared to the capabilities of raw 608/708 data. For example, it does not seem to be possible to control when on the screen captions are placed. It seems like OBS probably uses https://github.com/szatmary/libcaption internally, which looks to have more capabilities than what the very limited SendStreamCaption exposes, but also still looks to be limited compared to what the full CEA-708 specification is capable of.
Also, should we just forward messages as-is from the FTC scoring overlay websocket, or should we interpret the data and send our own format?
- Sending the raw data as we receive it means we can add new features into Clipfarm without having to update MatchBox
- Sending the data in raw form also means we technically spoil the result of the "which alliance won" animation, if people are manually looking at subtitles.
- Perhaps we intentionally delay/omit/modify match result screen messages, while leaving the other messages untouched?
See also:
It appears to be possible to embed real-time data into a livestream via embedded 608/708 closed-caption data:
This opens a few interesting possibilities:
There are probably also other interesting and exciting possibilities this could unlock.
According to https://web.archive.org/web/20230305210310/https://webcaptioner.com/help/integrations/obs, it should be possible to send caption data to OBS via the OBS WebSockets connection we are already using. From one of the images in that webpage, it appears the format is quite simple:
The source code of Webcaptioner is also available here, as a reference: https://github.com/curtgrimes/webcaptioner
Note: It seems like
SendStreamCaptionis rather limited, when compared to the capabilities of raw 608/708 data. For example, it does not seem to be possible to control when on the screen captions are placed. It seems like OBS probably uses https://github.com/szatmary/libcaption internally, which looks to have more capabilities than what the very limitedSendStreamCaptionexposes, but also still looks to be limited compared to what the full CEA-708 specification is capable of.Also, should we just forward messages as-is from the FTC scoring overlay websocket, or should we interpret the data and send our own format?
See also: