"Many APIs" OwlBot pull request is trying to delete java-gsuite-addons's docs package #8547
Description
#8495 (with manual owlbot-java invocation) tries to delete files under java-gsuite-addons/proto-google-apps-script-type-protos/src/main/java/com/google/apps/script/type/docs.
The pull request is hitting compilation error:
Error: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project proto-google-cloud-gsuite-addons-v1: Compilation failure: Compilation failure:
Error: /home/runner/work/google-cloud-java/google-cloud-java/java-gsuite-addons/proto-google-cloud-gsuite-addons-v1/src/main/java/com/google/cloud/gsuiteaddons/v1/AddOns.java:[262,43] package com.google.apps.script.type.docs does not exist
Error: /home/runner/work/google-cloud-java/google-cloud-java/java-gsuite-addons/proto-google-cloud-gsuite-addons-v1/src/main/java/com/google/cloud/gsuiteaddons/v1/AddOns.java:[290,42] package com.google.apps.script.type.docs does not exist
Error: /home/runner/work/google-cloud-java/google-cloud-java/java-gsuite-addons/proto-google-cloud-gsuite-addons-v1/src/main/java/com/google/cloud/gsuiteaddons/v1/AddOns.java:[305,42] package com.google.apps.script.type.docs does not exist
Error: /home/runner/work/google-cloud-java/google-cloud-java/java-gsuite-addons/proto-google-cloud-gsuite-addons-v1/src/main/java/com/google/cloud/gsuiteaddons/v1/AddOnsOrBuilder.java:[189,35] package com.google.apps.script.type.docs does not exist
Error: /home/runner/work/google-cloud-java/google-cloud-java/java-gsuite-addons/proto-google-cloud-gsuite-addons-v1/src/main/java/com/google/cloud/gsuiteaddons/v1/AddOnsOrBuilder.java:[199,35] package com.google.apps.script.type.docs does not exist
Error: /home/runner/work/google-cloud-java/google-cloud-java/java-gsuite-addons/proto-google-cloud-gsuite-addons-v1/src/main/java/com/google/cloud/gsuiteaddons/v1/AddOns.java:[1776,45] package com.google.apps.script.type.docs does not exist
Somehow java-gsuite-addons/proto-google-apps-script-type-protos/src/main/java/com/google/apps/script/type/docs directory was not generated and thus being deleted in the pull request.
suztomo@suztomo:/tmp/google-cloud-java$ git diff --name-status origin/main |grep "^D"
...
D java-gsuite-addons/proto-google-apps-script-type-protos/src/main/java/com/google/apps/script/type/docs/DocsAddOnManifest.java
D java-gsuite-addons/proto-google-apps-script-type-protos/src/main/java/com/google/apps/script/type/docs/DocsAddOnManifestOrBuilder.java
D java-gsuite-addons/proto-google-apps-script-type-protos/src/main/java/com/google/apps/script/type/docs/DocsAddOnManifestProto.java
D java-gsuite-addons/proto-google-apps-script-type-protos/src/main/java/com/google/apps/script/type/docs/DocsExtensionPoint.java
D java-gsuite-addons/proto-google-apps-script-type-protos/src/main/java/com/google/apps/script/type/docs/DocsExtensionPointOrBuilder.java
D java-gsuite-addons/proto-google-apps-script-type-protos/src/main/proto/google/apps/script/type/docs/docs_addon_manifest.proto
...
How is this supposed to work?
The files in googleapis-gen look fine
googleapis/googleapis-gen repository has the file https://github.com/googleapis/googleapis-gen/blob/master/google/apps/script/type/docs/google-apps-script-type-docs-java/proto-google-apps-script-type-docs-java/src/main/java/com/google/apps/script/type/docs/DocsAddOnManifest.java . You can confirm it via find command too:
~/googleapis-gen $ find . -name DocsAddOnManifest.java
./google/apps/script/type/docs/google-apps-script-type-docs-java/proto-google-apps-script-type-docs-java/src/main/java/com/google/apps/script/type/docs/DocsAddOnManifest.java
So it seems Bazel Bot is working fine.
The owl-bot-staging (created by OwlBot service) does not contain the files
How about OwlBot service? The OwlBot configuration file for java-gsuite-addons module in this monorepo (https://github.com/googleapis/google-cloud-java/blob/main/java-gsuite-addons/.OwlBot.yaml) defines "deep-copy-regex":
deep-copy-regex:
...
- source: "/google/apps/script/type/docs/google-apps-script-type-docs-java/proto-google-apps-script-type-docs-java/src"
dest: "/owl-bot-staging/java-gsuite-addons/v1/proto-google-apps-script-type-protos/src"
...
With these lines, OwlBot service is supposed to copy the content of google/apps/script/type/docs/google-apps-script-type-docs-java/proto-google-apps-script-type-docs-java/src (which is main/java/com/google/apps/script/type/docs/DocsAddOnManifest.java and other files) to /owl-bot-staging/java-gsuite-addons/v1/proto-google-apps-script-type-protos/src. However, the main/java/com/google/apps/script/type/docs directory does not exist in /owl-bot-staging after OwlBot service creates the pull request:
suztomo@suztomo:/tmp/google-cloud-java$ git log -1 |head
commit da4601f671856a74607b05733a10a33cb313ebc1
Author: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Date: Thu Oct 6 22:28:42 2022 +0000
chore(bazel): update protobuf to v3.21.7
PiperOrigin-RevId: 477955264
Source-Link: https://github.com/googleapis/googleapis/commit/a724450af76d0001f23602684c49cd6a4b3a5654
suztomo@suztomo:/tmp/google-cloud-java$ find owl-bot-staging/ -name DocsAddOnManifest.java
suztomo@suztomo:/tmp/google-cloud-java$
How to see owl-bot-staging directory?
In #8495,
da4601f is the commit made by OwlBot service. It has owl-bot-staging directory and owlbot-java post processor moves the files to the appropriate locations. However it seems that the owl-bot-staging directory already does not have the files:
How to reproduce this observation
After cloning this repository, run git checkout -b issue_8547 da4601f671856a74607b05733a10a33cb313ebc1. Try to find DocsAddOnManifest.java in owl-bot-staging:
suztomo@suztomo:/tmp/google-cloud-java$ find owl-bot-staging -name DocsAddOnManifest.java
suztomo@suztomo:/tmp/google-cloud-java$
What about the same change in split repo?
There was no problem in the corresponding pull request in the split repo https://github.com/googleapis/java-gsuite-addons/pull/274/files. The split repo has a very similar OwlBot.yaml as the module in the monorepo.
(The files in the "docs" directory were not removed)
Isn't this owlbot-java postprocessor problem?
No, the problem is before owlbot-java postprocessor runs. The owlbot-java postprocessor extracts files from "owl-bot-staging" directory, which OwlBot service provides based on the configuration file. If the files are not in the staging directory, then it cannot do anything about it.
How about running owlbot-cli copy-code locally?
Local owlbot-cli copy-code operation successfully generates the files in the "docs" package.
I ran:
suztomo@suztomo:/tmp$ git clone https://github.com/googleapis/google-cloud-java
suztomo@suztomo:/tmp$ git clone https://github.com/googleapis/googleapis-gen
suztomo@suztomo:/tmp$ cd google-cloud-java
suztomo@suztomo:/tmp/google-cloud-java$ module=java-gsuite-addons
suztomo@suztomo:/tmp/google-cloud-java$ GOOGLEAPIS_DIR=/tmp/googleapis-gen
suztomo@suztomo:/tmp/google-cloud-java$ docker run --rm --user "$(id -u):$(id -g)" \
-v "$(pwd):/repo" \
-v "${GOOGLEAPIS_DIR}:/googleapis-gen" \
-w /repo \
--env HOME=/tmp \
gcr.io/cloud-devrel-public-resources/owlbot-cli:latest copy-code \
--source-repo=/googleapis-gen \
--config-file=${module}/.OwlBot.yaml
Then the file is there:
suztomo@suztomo:/tmp/google-cloud-java$ find owl-bot-staging -name DocsAddOnManifest.java
owl-bot-staging/java-gsuite-addons/v1/proto-google-apps-script-type-protos/src/main/java/com/google/apps/script/type/docs/DocsAddOnManifest.java