Broken link detection in development

[YOU54F]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Prerequisites

• I'm using the latest version of Docusaurus.
• I have tried the npm run clear or yarn clear command.
• I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
• I have tried creating a repro with https://new.docusaurus.io.
• I have read the console error message carefully (if applicable).

Description

Hello,

I am currently working through our https://docs.pact.io site and we had a number of broken links.

We have now set onBrokenLinks to true, which causes the build the fail. 👍🏾

however as noted in the docs

The broken links detection is only available for a production build (docusaurus build).

I would love to give my contributors earlier feedback, during development as per to onDuplicateRoutes

By default, it displays a warning after you run yarn start or yarn build.

when they run yarn start.

I haven't looked at the codebase, to see the rational for this only being applicable for production builds at the moment.

Thanks in advance and also for the great project!

Reproducible demo

No response

Steps to reproduce

1. Have a broken link in your docusaurus site
2. set onBrokenLinks to error
3. run yarn start and user is not notified there are broken links
4. run yarn build and build fails with user notified of broken links

Expected behavior

1. Have a broken link in your docusaurus site
2. set onBrokenLinks to error
3. run yarn start and build fails with user notified of broken links
4. run yarn build and build fails with user notified of broken links

Actual behavior

1. Have a broken link in your docusaurus site
2. set onBrokenLinks to error
3. run yarn start and user is not notified there are broken links

Your environment

• Public source code: fix: correct broken links and fail build on further broken links pact-foundation/docs.pact.io#122 (review)

Self-service

• I'd be willing to fix this bug myself.

[Josh-Cena]

I think it's possible, but there'll be false-positives since we can't know if a link is linking to a static asset (we detect that by making FS calls). Still it's mostly fine IMO

[YOU54F]

Thanks @Josh-Cena

🤔 Good point about the static asset linking. False positives are less than ideal, maybe we might set it at warn level in our configuration so at least the user gets feedback, and can make a decision either way.

We may want additional information in an error message to state something to that effect, and the user can run the build check to determine if it is a false positive or not. I don't know if that is just an extra inconvenience.

Would you think we would we better applying this as a boolean with a new property such as onBrokenLinksDevelopment and it honours the level set by onBrokenLinks

[Josh-Cena]

Yes, that's exactly how I would design it: give some feedback, but also remind users about false-positives. @slorber Do you think this sounds reasonable?

[YOU54F]

It seems like this was requested on the canny feature request board as well, my apologies for not checking!

https://docusaurus.io/feature-requests/p/onbrokenlinks-in-development.

Yes this is a technical limitation.
The broken link checked actually render the pages to collect the rendered links and compare against the routes.
This is not ideal because it fails too late in the build process and the feedback loop is slow. However it's also quite reliable because it works for any links, including paths that are created using JS (ie not statically analyzable).
We should find a way to provide a simpler broken link detection algo that would be less reliable but that could be faster

[yzeng25]

Hi @YOU54F , I've had the similar broken link issues. Instead of completely relying on Docusaurus to solve them, there are third-party tools designed for broken link checks. I tried all of them when I was maintaining a document website, and the third tool is more handy than the other two.

https://validator.w3.org/checklink
https://www.brokenlinkcheck.com/
https://www.drlinkcheck.com/

[YOU54F]

Hi @YOU54F , I've had the similar broken link issues. Instead of completely relying on Docusaurus to solve them, there are third-party tools designed for broken link checks. I tried all of them when I was maintaining a document website, and the third tool is more handy than the other two.

https://validator.w3.org/checklink https://www.brokenlinkcheck.com/ https://www.drlinkcheck.com/

Awesome dude, checking these out, as especially external links may die over time and not necessarily at fault to the creator of a PR. Running an check via another tool, as a cron job with a report, and an issue raised in the respective repo to resolve, might be enough of a call to action to fix 👍🏾