feat(v2): Added Lighthouse CI to PR checks by sarthakkundra · Pull Request #3761 · facebook/docusaurus

sarthakkundra · 2020-11-16T19:51:46Z

Motivation

Added Lighthouse CI to integrate finally and create a complete Build bot to analyse stats changed with every new pull request.

Have you read the Contributing Guidelines on pull requests?

Yes.

Test Plan

A new comment action is added.

Related PRs

#3744

netlify · 2020-11-16T19:58:12Z

Deploy preview for docusaurus-2 ready!

Built without sensitive environment variables with commit b407ab8

https://deploy-preview-3761--docusaurus-2.netlify.app

slorber · 2020-11-17T10:34:31Z

    return isActive ? false : item.collapsed;
  });

+  const [enableCSSStyles, setEnableCSSStyles] = useState(true);


Hi

You probably messed up with your Git workflow, because it's not supposed to be there 😅

I know it was 2 am IST. I will fix it today, that is why I didn't message for a review 😅

slorber · 2020-11-17T10:37:13Z

    // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
-    codeBlockTitle = metastring
-      .match(codeBlockTitleRegex)![1];
+    codeBlockTitle = metastring.match(codeBlockTitleRegex)![1];


we already have this code formatting fixed on master

I just need to rebase, will do that when I make the other required changes. Sorry for the confusion

slorber · 2020-11-17T10:37:42Z

Thanks

Do you know how I can validate this PR works as intended?

Will the comment only work after a merge?

lex111 · 2020-11-17T10:43:59Z

We definitely need to create our own config for Lighthouse (lighthouserc.json). At least to provide a path for PWA checks: https://v2.docusaurus.io/?offlineMode=true
An example of such a file can be found here.

sarthakkundra · 2020-11-17T11:55:06Z

@slorber The stats will be provided on each PR. To check them for now you can go inside 'details' in the Github action and check for the Audit tab

sarthakkundra · 2020-11-17T11:56:26Z

@slorber Since you gave the go ahead to upload the Lighthouse stats on a public server, I can add a comment that adds the URL to the uploaded stats just like the Netlify bot. I think that'll be good?

sarthakkundra · 2020-11-17T12:01:24Z

@lex111 Do the score check only need to be applied to the PWA or both online and offline versions?

lex111 · 2020-11-17T13:13:26Z

I think checking one URL (with ?offlineMode=true) will be enough.

removed wait for Netlify action corrected Netlify deploy urls corrected sitename corrected sitename

sarthakkundra · 2020-11-17T21:51:30Z

@lex111 Netlify does not generate an offline preview mode so if I just add this PWA then it will check the live version of it and not the version that is generated with the new changes. So I don't think we should add it. If you had anything else in mind, do let me know.

As far as the config file is needed, it comes out of the box with the action and a budget.json file can be added, more on that here budget. If that's required, do let me know the exact configs of it, although the report generated by the action is pretty extensive with all the runtimes stated.

sarthakkundra · 2020-11-17T21:56:32Z

@slorber The report generated by Lighthouse can be accessed here. To access it manually head over to the details of the Lighthouse CI action and you can find the link here.

Hope this clears the testing plan. Let me know if any changes are required.

lex111

UPD: haven't seen the previous comment, if none of above can be done then LGTM 👍 from my part.

Timed out waiting for start_url (https://deploy-preview-3761--docusaurus-2.netlify.app/classic/index.html) to respond

As I wrote earlier, the URL must be used with offline mode enabled, then this error will not occur.

Also it is better for us to use the config file for Lighthouse, then we can exclude some checks that only distract attention:

Page is blocked from indexing
robots.txt is not valid
Document does not have a valid

These checks will never pass on Netlify deploys, so we can turn them off. As a result, we don't see real SEO metrics.

sarthakkundra · 2020-11-18T08:52:57Z

@lex111 Added a lighthousesrc.json file and removed the following failing SEO audits.

robots-txt
document does not have tap targets
canonical

Unfortunately couldn't find an audit for page is blocked from indexing in the lighthouse config documentation. If you can find it then let me know, I'll add it.
Apart from that I think this PR is ready to be merged and will close this issue. Do let me know if anything else is required.

lex111 · 2020-11-18T15:58:19Z

Unfortunately couldn't find an audit for page is blocked from indexing in the lighthouse config documentation. If you can find it then let me know, I'll add it.

Maybe is-crawlable will help?

sarthakkundra · 2020-11-24T19:04:10Z

@slorber The comment is ready I think it is just failing because you'll have to approve the action / PR. Here's a screenshot of the generated comment. I think this is what you wanted. Let me know if any changes are required.

lex111 · 2020-11-24T19:22:25Z

@sarthakkundra nice work, but can you please add offline_start_url to the skip audits list so that we get 100% score for PWA metric?

sarthakkundra · 2020-11-24T19:39:31Z

@lex111 Done! :)

lex111 · 2020-11-24T19:42:26Z

@sarthakkundra thanks, but it doesn't seem to work :( And if you try to add just start-url to skip audits list?

sarthakkundra · 2020-11-24T19:56:30Z

@lex111 Doesn't seem to work :/

lex111 · 2020-11-24T19:58:59Z

So, I understand, please add works-offline and offline-start-url, then everything will work for sure (I guess) and we get the long-awaited 100% for PWA.

sarthakkundra · 2020-11-24T20:10:27Z

@lex111 Have a look

lex111 · 2020-11-24T20:12:57Z

Excellent! :)

slorber · 2020-11-26T16:41:29Z

Thanks, LGTM, but we'll see after merge if it works fine or not ;)

sarthakkundra · 2020-11-26T16:47:10Z

Hahaha. Sure. Let me know if any changes are required :)

slorber · 2020-11-30T18:43:28Z

@sarthakkundra it seems to work fine, FYI had to change this setting so that it works in forked repos too (#3846)

Was wondering if we can try to make the Lighthouse score more reliable. I think Netlify needs some warm up, wonder if it could get a more. predictable score by running Lighthouse twice on it (or more?), and uploading only the latest results, and uploading only the last result once the CDN caches are hotter?

Are there other settings we can use to make this score more reliable? (I don't know, maybe can we disable TTFB scores and everything that relates to network speed etc)
Can we run Lighthouse multiple times and have an average or something?

sarthakkundra · 2020-11-30T19:16:23Z

@slorber We can definitely make Lighthouse run N number of times. Not sure about whether it uploads only the last score or the average of all.

Maybe you can open an issue and I'll look into the possible ways to make the scores better and make a PR? I'll change the number of time it runs and probably disable tests that are giving a false output / not applicable on Netlify deploys.

* added lighthouse CI removed wait for Netlify action corrected Netlify deploy urls corrected sitename corrected sitename * removed bootstrap and blog-only urls * upload results to public storage * added lighthousesrc.json * corrected configPath * corrected robots-txt assertion * corrected syntax for lighthousesrc * tried different audit * corrected all audits for lighthouse * added is-crawlable audit to skip * added lighthouse stats commit to action * Fixed Lighthouse yml syntax * Fixed lighthouse yml syntax(2) * trying comment * trying yaml syntax * test logging * trying to see a log * logging.... * corrected audit step id * Corrected token * changed comment number from issue to PR * trying different message * corrected message * added offline-start-url to skip audits * added start-url to skip fields * added offline-starts-url

sarthakkundra requested review from lex111 and slorber as code owners November 16, 2020 19:51

facebook-github-bot added the CLA Signed Signed Facebook CLA label Nov 16, 2020

sarthakkundra changed the title ~~Lighthouse~~ Added Lighthouse CI Nov 16, 2020

slorber reviewed Nov 17, 2020

View reviewed changes

Comment thread .github/workflows/lighthouseCI.yml

slorber reviewed Nov 17, 2020

View reviewed changes

added lighthouse CI

3873153

removed wait for Netlify action corrected Netlify deploy urls corrected sitename corrected sitename

sarthakkundra force-pushed the lighthouse branch from b5937ac to 3873153 Compare November 17, 2020 18:51

sarthakkundra added 2 commits November 18, 2020 00:53

removed bootstrap and blog-only urls

d7ba998

upload results to public storage

3f72815

lex111 suggested changes Nov 17, 2020

View reviewed changes

lex111 added this to the v2.0.0-alpha.67 milestone Nov 17, 2020

sarthakkundra added 6 commits November 18, 2020 12:50

added lighthousesrc.json

1cbc5e4

corrected configPath

c4c314e

corrected robots-txt assertion

c4f8490

corrected syntax for lighthousesrc

82a583c

tried different audit

1d90b1a

corrected all audits for lighthouse

5c9ddbb

sarthakkundra added 6 commits November 24, 2020 17:06

logging....

e02e269

corrected audit step id

ee4fcee

Corrected token

607efd4

changed comment number from issue to PR

0f966e0

trying different message

bd70339

corrected message

85f6507

added offline-start-url to skip audits

0ea0d94

added start-url to skip fields

789f845

added offline-starts-url

b407ab8

lex111 requested a review from slorber November 25, 2020 14:57

slorber changed the title ~~Added Lighthouse CI~~ feat(v2): Added Lighthouse CI to PR checks Nov 26, 2020

slorber added the pr: new feature This PR adds a new API or behavior. label Nov 26, 2020

slorber merged commit bd62be9 into facebook:master Nov 26, 2020

HarshCasper mentioned this pull request Oct 2, 2021

Add LightHouse CI for the Community Website moja-global/community-website#68

Closed

Qdigital mentioned this pull request Sep 28, 2023

[Snyk] Security upgrade sharp from 0.29.3 to 0.32.6 Qdigital/docusaurus#92

Open

NikolayS mentioned this pull request Sep 29, 2023

[Snyk] Security upgrade sharp from 0.25.4 to 0.32.6 NikolayS/docusaurus#92

Open

Qdigital mentioned this pull request Sep 29, 2023

[Snyk] Security upgrade sharp from 0.29.3 to 0.32.6 Qdigital/docusaurus#93

Open

Uh oh!

Uh oh!

Conversation

sarthakkundra commented Nov 16, 2020

Motivation

Have you read the Contributing Guidelines on pull requests?

Test Plan

Related PRs

Uh oh!

netlify Bot commented Nov 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

slorber Nov 17, 2020

Choose a reason for hiding this comment

Uh oh!

sarthakkundra Nov 17, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

slorber Nov 17, 2020

Choose a reason for hiding this comment

Uh oh!

sarthakkundra Nov 17, 2020

Choose a reason for hiding this comment

Uh oh!

slorber commented Nov 17, 2020

Uh oh!

lex111 commented Nov 17, 2020

Uh oh!

sarthakkundra commented Nov 17, 2020

Uh oh!

sarthakkundra commented Nov 17, 2020

Uh oh!

sarthakkundra commented Nov 17, 2020

Uh oh!

lex111 commented Nov 17, 2020

Uh oh!

sarthakkundra commented Nov 17, 2020

Uh oh!

sarthakkundra commented Nov 17, 2020

Uh oh!

lex111 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sarthakkundra commented Nov 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lex111 commented Nov 18, 2020

Uh oh!

sarthakkundra commented Nov 24, 2020

Uh oh!

lex111 commented Nov 24, 2020

Uh oh!

sarthakkundra commented Nov 24, 2020

Uh oh!

lex111 commented Nov 24, 2020

Uh oh!

sarthakkundra commented Nov 24, 2020

Uh oh!

lex111 commented Nov 24, 2020

Uh oh!

sarthakkundra commented Nov 24, 2020

Uh oh!

lex111 commented Nov 24, 2020

Uh oh!

slorber commented Nov 26, 2020

Uh oh!

sarthakkundra commented Nov 26, 2020

Uh oh!

slorber commented Nov 30, 2020

Uh oh!

sarthakkundra commented Nov 30, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

netlify Bot commented Nov 16, 2020 •

edited

Loading

lex111 left a comment •

edited

Loading

sarthakkundra commented Nov 18, 2020 •

edited

Loading