Dinov3

[reza-shahriari]

Search before asking

• I have searched the RF-DETR issues and found no similar feature requests.

Description

could be nice if you add DinoV3 as backbone as well

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[S-Mahoney]

I've got a branch with it set up, need to check with the powers that be that including it in the Rfdetr repo doesn't violate their licenses. Will also require others to test training dinov2 and dinvo3 versions on machines more powerful than mine.

[S-Mahoney]

See: #324

[isaacrob-roboflow]

If / when we merge a dinov3 pr, it'll be after we've had a chance to do our pretraining which adds some +10 mAP, and after we've done latency benchmarking of the dinov3 implementation. Our dinov2 implementation is modified from the original to be significantly faster in TRT, and the exact modifications don't translate directly to dinov3

I think it would be challenging for a third party to either do the pretrain or speed up the backbone because we STILL haven't released the paper which will contain details as to how those work

Really appreciate the interest :) will share more as we have updates

[reza-shahriari]

Really appreciate the transparency about your process and timeline. The pretraining and optimization work sounds exciting - will definitely keep an eye out for updates!

[liopeer]

Using LightlyTrain you can already do knowledge distillation of DINOv3 teachers into your vision backbones (including RF-DETR): https://github.com/lightly-ai/lightly-train

[isaacrob-roboflow]

I would not recommend doing that as that will remove the objects365 pretraining from the backbone