feat: Add app import utility and perform the initial setup on CI by SatishGandham · Pull Request #10050 · appsmithorg/appsmith

SatishGandham · 2021-12-29T10:41:24Z

Description

Improvements to perf infra scripts

Run all files in the tests folder in sequence
Do the initial setup
Better error handling and saving of screenshots
Refactor code.
Improve the summary generator, add standard deviation column
Add utility method to import an app
Add a basic performance test on imported app

Fixes #10081

Type of change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Manually tested

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

Test coverage results 🧪

🟢 Total coverage has increased

// Code coverage diff between base branch:release and head branch: ui-perf-add-app-import

Status	File	% Stmts	% Branch	% Funcs	% Lines
🟢	total	55.12 (0.01)	36.61 (0)	34.54 (0)	55.64 (0)
🟢	app/client/src/utils/WorkerUtil.ts	89.76 (0.78)	72.55 (1.96)	100 (0)	93.33 (0.95)

- Better error handling and saving of screenshots - Organize and refactor code. WIP - Improve the summary generator - Add utility method to import an app - Add a basic performance test on imported app

github-actions · 2021-12-30T16:44:41Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1638083133.
Commit: 83ea63b.
Results:

Click to view performance test results

	Run #1	Run #2	Run #3	Run #4	Run #5	Avg
Click on table row
scripting	2575.9	2121.29	2147.28	2039.37	2138.78	2204.52
painting	20.24	31.91	31.56	48.11	17.39	29.84
rendering	423.9	341.58	322.05	337.79	344.46	353.96
Edit input
scripting	278.46	258.33	308.49	310.56	327.55	296.68
painting	17.94	5.23	10.12	11.37	10.18	10.97
rendering	22.58	12.64	17.66	11.48	21.76	17.22
Clear input
scripting	738.37	728.32	768.39	725.64	718.49	735.84
painting	18.34	16.16	12.3	16.01	10.8	14.72
rendering	217.1	215.64	201.23	226.55	203.58	212.82
Edit input again
scripting	267.02	269.11	286.99	268.99	269.24	272.27
painting	6.8	5.26	5.98	15.09	13.26	9.28
rendering	17.4	19.82	16.76	17.84	26.54	19.67

SatishGandham · 2021-12-30T17:21:36Z

/ok-to-test sha=aee1594

github-actions · 2021-12-30T17:27:43Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1638186228.
Commit: 9afd9e2.
Results:

Click to view performance test results

	Run #1	Run #2	Run #3	Run #4	Run #5	Avg
Click on table row
scripting	3317.10	2608.72	2121.06	2468.00	2439.82	NaN
painting	49.86	50.22	17.68	61.48	20.72	NaN
rendering	754.75	421.50	440.84	437.12	451.98	NaN
Edit input
scripting	337.42	413.08	472.40	320.40	368.04	NaN
painting	24.36	21.39	4.29	15.08	13.48	NaN
rendering	22.83	19.23	11.72	14.82	18.75	NaN
Clear input
scripting	876.93	983.91	871.85	815.89	870.72	NaN
painting	12.01	16.80	14.22	14.94	41.09	NaN
rendering	284.80	309.88	297.60	256.19	294.30	NaN
Edit input again
scripting	459.04	394.78	393.48	411.07	357.45	NaN
painting	41.03	6.62	21.70	6.80	7.67	NaN
rendering	30.96	24.77	24.39	17.99	23.47	NaN

github-actions · 2021-12-30T17:49:03Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1638239714.
Commit: aee1594.
Results:

Click to view performance test results

	Run #1	Run #2	Run #3	Run #4	Run #5	Avg
Click on table row
scripting	2544.8	2372.42	2350.47	2668.01	2608.67	2508.87
painting	41.2	26.34	32.03	28.69	30.26	31.70
rendering	396.13	356.86	349.07	402.43	414.22	383.74
Edit input
scripting	335.84	391.41	350.09	374.87	346.57	359.76
painting	8.3	27.02	10.06	25.27	13.94	16.92
rendering	15.74	22.69	27.46	17.83	15.43	19.83
Clear input
scripting	892.05	1012.04	1047.15	969.07	876.84	959.43
painting	8.11	27.66	25.07	17.97	23.56	20.47
rendering	226.07	286.33	316.12	276.33	244.49	269.87
Edit input again
scripting	384.74	409.76	436.06	391.2	375.05	399.36
painting	8.63	17.93	20.11	7.72	10.19	12.92
rendering	28.26	26.58	27.73	23.63	28.29	26.90

github-actions · 2021-12-30T18:25:08Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1638328495.
Commit: aee1594.
Results:

Click to view performance test results

	Run #1	Run #2	Run #3	Run #4	Run #5	Mean	SD.Sample	SD.Population
Click on table row
scripting	2742.42	2158.05	2148.41	2178.82	2190.14	2283.57	11.26	10.07
painting	55.17	23.43	32.16	37.16	30.82	35.75	33.34	29.82
rendering	534.88	398.5	396.17	384.98	348.69	412.64	17.25	15.43
Edit input
scripting	336.34	310.9	311.05	381.19	307.6	329.42	9.46	8.46
painting	6.88	5.62	34.52	8.64	16.38	14.41	83.28	74.46
rendering	21.19	16	15.87	19.07	15.57	17.54	14.20	12.66
Clear input
scripting	779.18	720.39	793.96	754.87	767.42	763.16	3.66	3.27
painting	14.84	21.34	25.48	10.4	10.31	16.47	40.98	36.67
rendering	225.47	209.72	218.08	220.03	216.7	218	2.61	2.34
Edit input again
scripting	303.78	224.57	274.49	282.79	297.26	276.58	11.31	10.12
painting	6.18	4.04	5.55	22.23	9.88	9.58	77.14	69.00
rendering	22.89	17.72	19.53	27.44	20.99	21.71	17.13	15.34

github-actions · 2021-12-30T18:38:00Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1638368435.
Commit: 7a28b43.
Results:

Click to view performance test results

	Run #1	Run #2	Run #3	Run #4	Run #5	Mean	SD.Sample	SD.Population
Click on table row
scripting	2135.97	2611.51	2127.44	2091.7	2116.99	2216.72	9.98	8.93
painting	17.8	40.62	26.12	36.31	19.24	28.02	36.26	32.41
rendering	320.01	494.56	342.81	319.73	313.76	358.17	21.51	19.24
Edit input
scripting	286.54	265.74	263.89	303.67	286.6	281.29	5.90	5.28
painting	8.32	7.91	13.36	20.55	12.09	12.45	40.96	36.71
rendering	11.59	12.82	14.67	14.88	19.39	14.67	20.25	18.06
Clear input
scripting	755.48	709.26	758.61	820.37	711.56	751.06	6.02	5.39
painting	13.22	6.11	10.12	10.5	6.27	9.24	32.79	29.33
rendering	210.34	200.36	211.5	224.52	198.37	209.02	5.00	4.47
Edit input again
scripting	284.82	254.47	284.01	302.38	305.4	286.22	7.08	6.33
painting	9.52	11.76	9.44	9.12	13.06	10.58	16.45	14.74
rendering	20.32	18.63	18.9	20.65	21.53	20.01	6.10	5.45

github-actions · 2021-12-31T04:56:50Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1639724056.
Commit: 7a28b43.
Results:

Click to view performance test results

	Run #1	Run #2	Run #3	Run #4	Run #5	Mean	SD.Sample	SD.Population
Click on table row
scripting	2736.4	2208.28	2159.17	2155.46	2139.18	2279.7	11.26	10.07
painting	34.46	29.09	29.89	28.72	16.19	27.67	24.65	22.05
rendering	524.05	333.06	315.21	326.77	341.84	368.19	23.81	21.30
Edit input
scripting	306.7	320.14	325.2	268.46	253.17	294.73	10.91	9.76
painting	5.13	6.79	11.29	12.08	6.48	8.35	37.37	33.41
rendering	16.94	15.83	18.89	14.39	14.74	16.16	11.26	10.09
Clear input
scripting	760.72	797.3	780.23	737.99	760.37	767.32	2.93	2.62
painting	17.62	11.65	11.43	18.43	17.48	15.32	22.65	20.30
rendering	209.7	217.27	221.02	214.89	215.67	215.71	1.91	1.70
Edit input again
scripting	318.07	288.22	331.66	307.58	268.41	302.79	8.23	7.36
painting	8.29	4.41	5.67	6.06	4.06	5.7	29.30	26.32
rendering	20.87	19.42	19.84	22.06	17.35	19.91	8.84	7.89

github-actions · 2021-12-31T05:31:04Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/1639839067.
Workflow: Appsmith External Integration Test Workflow.
Commit: 9694f51.
PR: 10050.

github-actions · 2021-12-31T08:07:09Z

Unable to find test scripts. Please add necessary tests to the PR.

github-actions · 2021-12-31T08:07:33Z

Unable to find test scripts. Please add necessary tests to the PR.

SatishGandham · 2021-12-31T08:12:03Z

/ok-to-test sha=af9028e

github-actions · 2021-12-31T08:12:36Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/1640164268.
Workflow: Appsmith External Integration Test Workflow.
Commit: af9028e.
PR: 10050.

github-actions · 2021-12-31T08:38:15Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1640164268.
Commit: af9028e.
Results:

Click to view performance test results

	Run 1	Run 2	Run 3	Run 4	Run 5	Mean	SD.Sample	SD.Population
Edit input
scripting	395.72	391.24	355.92	439.13	366.45	389.69	8.28	7.41
painting	14.93	6.49	32.87	37.24	9.56	20.22	69.04	61.77
rendering	15.8	20.56	19.18	43.52	19.43	23.7	47.34	42.36
Clear input
scripting	933.1	917.2	1041.65	942.39	1066.71	980.21	7.01	6.27
painting	13.82	22.72	12.36	23.53	16.83	17.85	28.46	25.49
rendering	354.07	287.42	327.71	313.2	316.82	319.84	7.56	6.76
Edit input again
scripting	414.53	388.27	383.81	416.63	381.22	396.89	4.35	3.89
painting	16.77	6.77	30.94	6.71	8.86	14.01	73.73	65.95
rendering	27.49	23.47	31.27	28.27	25.37	27.17	10.89	9.75

SatishGandham · 2021-12-31T09:23:18Z

/ok-to-test sha=af9028e

github-actions · 2021-12-31T09:23:50Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/1640326152.
Workflow: Appsmith External Integration Test Workflow.
Commit: af9028e.
PR: 10050.

github-actions · 2021-12-31T09:48:05Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1640326152.
Commit: af9028e.
Results:

Click to view performance test results

	Run 1	Run 2	Run 3	Run 4	Run 5	Mean	SD.Sample	SD.Population
Click on table row
scripting	2080.74	2174.43	2924.38	2097.42	2123.69	2280.13	15.87	14.20
painting	15.45	24.33	99.68	24.88	21.2	37.11	94.80	84.77
rendering	339.07	349.34	466.23	347.09	320.74	364.49	15.90	14.23
Edit input
scripting	271.92	299.88	290.93	334.12	284.27	296.22	7.94	7.10
painting	3.95	16.1	4.13	8.35	13.26	9.16	59.28	53.06
rendering	12.04	17.2	12.93	22.08	12.69	15.39	27.68	24.76
Clear input
scripting	787.97	759.78	741.58	781.55	754.93	765.16	2.51	2.25
painting	29.24	21.64	24.49	36.08	20.28	26.35	24.40	21.82
rendering	213.23	220.79	206.93	213.64	219.94	214.91	2.63	2.35
Edit input again
scripting	308.09	280.62	271.44	290.59	293.56	288.86	4.79	4.29
painting	6.92	14.36	10.65	5.44	5.05	8.48	46.70	41.75
rendering	21.23	18.21	18.87	23.9	19.34	20.31	11.32	10.14

SatishGandham · 2022-01-04T05:46:50Z

/ok-to-test sha=af9028e

github-actions · 2022-01-04T05:47:26Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/1652304217.
Workflow: Appsmith External Integration Test Workflow.
Commit: af9028e.
PR: 10050.

github-actions · 2022-01-04T06:15:38Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1652304217.
Commit: af9028e.
Results:

Click to view performance test results

	Run 1	Run 2	Run 3	Run 4	Run 5	Mean	SD.Sample	SD.Population
Click on table row
scripting	2281.53	2134.3	2164.98	2145.13	2216.26	2188.44	2.78	2.49
painting	36.54	24.97	28.05	64.81	24.85	35.84	47.10	42.13
rendering	392.83	311.89	325.92	332.64	325.37	337.73	9.39	8.40
Edit input
scripting	305.73	267.71	296.11	283.7	301.78	291.01	5.31	4.75
painting	2.99	7.81	10.65	17.38	19.71	11.71	58.67	52.43
rendering	16.05	14.85	27.84	20.95	15.62	19.06	28.65	25.60
Clear input
scripting	777.44	760.15	737.41	737.07	742.07	750.83	2.35	2.10
painting	19.19	16.74	13.81	14.74	15.11	15.92	13.25	11.87
rendering	241.87	211.52	201.75	211.98	216.19	216.66	6.95	6.21
Edit input again
scripting	295.96	282.33	281.71	268.19	322.29	290.1	7.07	6.32
painting	7.05	9.35	10.97	7.08	12.25	9.34	24.84	22.16
rendering	25.58	19.37	22.01	29.8	20.7	23.49	17.97	16.05

github-actions · 2022-01-04T07:00:54Z

Unable to find test scripts. Please add necessary tests to the PR.

SatishGandham · 2022-01-04T07:08:24Z

/ok-to-test sha=af9028e

github-actions · 2022-01-04T07:17:01Z

Unable to find test scripts. Please add necessary tests to the PR.

github-actions · 2022-01-04T07:17:20Z

Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/1652533665.
Workflow: Appsmith External Integration Test Workflow.
Commit: af9028e.
PR: 10050.

github-actions · 2022-01-04T07:41:54Z

UI Performance test run logs and artifacts: https://github.com/appsmithorg/appsmith/actions/runs/1652533665.
Commit: af9028e.
Results:

Click to view performance test results

	Run 1	Run 2	Run 3	Run 4	Run 5	Mean	SD.Sample	SD.Population
Click on table row
scripting	2251.61	2113.44	2166.18	2139.78	2129.36	2160.07	2.53	2.26
painting	66.85	50.96	58.43	36.86	39.21	50.46	25.15	22.49
rendering	416.17	355.24	301.15	319.69	319.27	342.3	13.35	11.95
Edit input
scripting	273.33	274.07	385.51	276.35	267	295.25	17.13	15.32
painting	5.98	5.07	14.08	30.61	5.97	12.34	87.93	78.61
rendering	14.29	13.26	21.78	13.66	13.53	15.3	23.79	21.31
Clear input
scripting	746.35	759.59	715.72	763.15	758.38	748.64	2.60	2.32
painting	11.64	19.45	10.38	11.78	10.22	12.69	30.26	27.11
rendering	214.57	216.82	216.05	225.34	218.16	218.19	1.92	1.72
Edit input again
scripting	258.32	294.81	276.4	344.36	277.1	290.2	11.34	10.14
painting	22.67	15.26	10.36	17.77	5.59	14.33	46.06	41.24
rendering	19.51	23.37	27.02	20.09	16.69	21.34	18.60	16.64

) - Run all files in the tests folder in sequence - Better error handling and saving of screenshots - Organise and refactor code. WIP - Improve the summary generator - Add utility method to import an app - Add a basic performance test on imported app Co-authored-by: Satish Gandham <satish@appsmith.com>

- Run all files in the tests folder in sequence

2f9cbf2

- Better error handling and saving of screenshots - Organize and refactor code. WIP - Improve the summary generator - Add utility method to import an app - Add a basic performance test on imported app

vercel bot deployed to Preview December 29, 2021 10:47 View deployment

- Fix issue with invalid file name when saving screenshots

db689f2

vercel bot deployed to Preview December 29, 2021 15:51 View deployment

- Add a script for intial setup. create super user

20269eb

vercel bot deployed to Preview December 30, 2021 14:21 View deployment

- Remove reference to local chrome executable

216a93b

vercel bot deployed to Preview December 30, 2021 14:47 View deployment

- Ignore ssl cert error

063194b

vercel bot deployed to Preview December 30, 2021 15:13 View deployment

- Hard code user name and password for testing

83ea63b

vercel bot deployed to Preview December 30, 2021 16:16 View deployment

- Round off measurements to two decimals

9afd9e2

vercel bot deployed to Preview December 30, 2021 17:00 View deployment

- Fix a small issue with rounding numbers

aee1594

vercel bot deployed to Preview December 30, 2021 17:21 View deployment

appsmithorg deleted a comment from github-actions bot Dec 30, 2021

- Add standard deviation to the results table

7a28b43

vercel bot deployed to Preview December 30, 2021 18:12 View deployment

- Format summary table

9694f51

vercel bot deployed to Preview December 31, 2021 05:30 View deployment

SatishGandham changed the title ~~DONT MERGE - Run all files in the tests folder in sequence~~ feat - add app import utility & do the initial setup on CI Dec 31, 2021

SatishGandham assigned marks0351 Dec 31, 2021

- Remove stray package from package.json

af9028e

vercel bot deployed to Preview December 31, 2021 08:11 View deployment

Nikhil-Nandagopal assigned appsmith-bot and unassigned appsmith-bot Jan 3, 2022

marks0351 approved these changes Jan 4, 2022

View reviewed changes

SatishGandham changed the title ~~feat - add app import utility & do the initial setup on CI~~ feat: Add app import utility & do the initial setup on CI Jan 4, 2022

github-actions bot added the Enhancement New feature or request label Jan 4, 2022

appsmithorg deleted a comment from github-actions bot Jan 4, 2022

riodeuno changed the title ~~feat: Add app import utility & do the initial setup on CI~~ feat: Add app import utility and perform the initial setup on CI Jan 4, 2022

SatishGandham merged commit 92cdeac into release Jan 4, 2022

SatishGandham deleted the ui-perf-add-app-import branch January 4, 2022 08:28

Conversation

SatishGandham commented Dec 29, 2021 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

How Has This Been Tested?

Checklist:

Test coverage results 🧪

Uh oh!

github-actions bot commented Dec 30, 2021 • edited by SatishGandham Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SatishGandham commented Dec 30, 2021

Uh oh!

github-actions bot commented Dec 30, 2021

Uh oh!

github-actions bot commented Dec 30, 2021

Uh oh!

github-actions bot commented Dec 30, 2021

Uh oh!

github-actions bot commented Dec 30, 2021

Uh oh!

github-actions bot commented Dec 31, 2021

Uh oh!

github-actions bot commented Dec 31, 2021

Uh oh!

github-actions bot commented Dec 31, 2021

Uh oh!

github-actions bot commented Dec 31, 2021

Uh oh!

SatishGandham commented Dec 31, 2021

Uh oh!

github-actions bot commented Dec 31, 2021

Uh oh!

github-actions bot commented Dec 31, 2021

Uh oh!

SatishGandham commented Dec 31, 2021

Uh oh!

github-actions bot commented Dec 31, 2021

Uh oh!

github-actions bot commented Dec 31, 2021

Uh oh!

SatishGandham commented Jan 4, 2022

Uh oh!

github-actions bot commented Jan 4, 2022

Uh oh!

github-actions bot commented Jan 4, 2022

Uh oh!

github-actions bot commented Jan 4, 2022

Uh oh!

SatishGandham commented Jan 4, 2022

Uh oh!

github-actions bot commented Jan 4, 2022

Uh oh!

github-actions bot commented Jan 4, 2022

Uh oh!

github-actions bot commented Jan 4, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SatishGandham commented Dec 29, 2021 •

edited by github-actions bot

Loading

github-actions bot commented Dec 30, 2021 •

edited by SatishGandham

Loading