Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,727 changes: 1,727 additions & 0 deletions .github/workflows/smoke-openclaw.lock.yml

Large diffs are not rendered by default.

88 changes: 88 additions & 0 deletions .github/workflows/smoke-openclaw.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
description: Smoke test workflow that validates OpenClaw engine functionality
on:
schedule: daily
workflow_dispatch:
pull_request:
types: [labeled]
names: ["smoke"]
reaction: "rocket"
status-comment: true
permissions:
contents: read
issues: read
pull-requests: read
discussions: read
actions: read

name: Smoke OpenClaw
engine:
id: openclaw
strict: true
features:
experimental-engines: true
imports:
- shared/gh.md
- shared/github-queries-safe-input.md
sandbox:
mcp:
container: "ghcr.io/github/gh-aw-mcpg"
network:
allowed:
- defaults
- github
tools:
github:
toolsets: [repos, pull_requests]
bash:
- "*"
safe-outputs:
add-comment:
hide-older-comments: true
max: 2
create-issue:
expires: 2h
group: true
close-older-issues: true
add-labels:
allowed: [smoke-openclaw]
messages:
footer: "> 🦀 *[Mission Complete] — Powered by [{workflow_name}]({run_url})*"
run-started: "🦀 **DEPLOY!** [{workflow_name}]({run_url}) activating for this {event_type}! *[Claw engaged...]*"
run-success: "🎯 **TARGET ACQUIRED** — [{workflow_name}]({run_url}) **ALL CLEAR!** Systems nominal! ✨"
run-failure: "⚠️ **RETREAT...** [{workflow_name}]({run_url}) {status}! The claw missed its target..."
timeout-minutes: 15
---

# Smoke Test: OpenClaw Engine Validation

**IMPORTANT: Keep all outputs extremely short and concise. Use single-line responses where possible. No verbose explanations.**

## Test Requirements

1. **GitHub MCP Testing**: Review the last 2 merged pull requests in ${{ github.repository }}
2. **Safe Inputs GH CLI Testing**: Use the `safeinputs-gh` tool to query 2 pull requests from ${{ github.repository }} (use args: "pr list --repo ${{ github.repository }} --limit 2 --json number,title,author")
3. **File Writing Testing**: Create a test file `/tmp/gh-aw/agent/smoke-test-openclaw-${{ github.run_id }}.txt` with content "Smoke test passed for OpenClaw at $(date)" (create the directory if it doesn't exist)
4. **Bash Tool Testing**: Execute bash commands to verify file creation was successful (use `cat` to read the file back)
5. **Discussion Interaction Testing**:
- Use the `github-discussion-query` safe-input tool with params: `limit=1, jq=".[0]"` to get the latest discussion from ${{ github.repository }}
- Extract the discussion number from the result (e.g., if the result is `{"number": 123, "title": "...", ...}`, extract 123)
- Use the `add_comment` tool with `discussion_number: <extracted_number>` to add a brief comment stating that the OpenClaw smoke test agent was here

## Output

1. **Create an issue** with a summary of the smoke test run:
- Title: "Smoke Test: OpenClaw - ${{ github.run_id }}"
- Body should include:
- Test results (✅ or ❌ for each test)
- Overall status: PASS or FAIL
- Run URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
- Timestamp

2. Add a **very brief** comment (max 5-10 lines) to the current pull request with:
- ✅ or ❌ for each test result
- Overall status: PASS or FAIL

3. Use the `add_comment` tool to add a brief comment to the latest discussion (using the `discussion_number` you extracted in step 5)

If all tests pass, add the label `smoke-openclaw` to the pull request.
243 changes: 243 additions & 0 deletions actions/setup/js/parse_openclaw_log.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,243 @@
// @ts-check
/// <reference types="@actions/github-script" />

const { createEngineLogParser, truncateString, estimateTokens, formatToolCallAsDetails } = require("./log_parser_shared.cjs");

const main = createEngineLogParser({
parserName: "OpenClaw",
parseFunction: parseOpenClawLog,
supportsDirectories: false,
});

/**
* Parse OpenClaw log content and format as markdown
* OpenClaw's --json flag produces structured JSON output (one JSON object per line)
* @param {string} logContent - The raw log content to parse
* @returns {{markdown: string, logEntries: Array, mcpFailures: Array<string>, maxTurnsHit: boolean}} Parsed log data
*/
function parseOpenClawLog(logContent) {
if (!logContent) {
return {
markdown: "## Agent Log Summary\n\nNo log content provided.\n\n",
logEntries: [],
mcpFailures: [],
maxTurnsHit: false,
};
}

const lines = logContent.split("\n");
let markdown = "";
const logEntries = [];
const mcpFailures = [];
let toolCallCount = 0;
let totalTokens = 0;

markdown += "## 🤖 Reasoning\n\n";

// Parse each line as potential JSON
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed) continue;

// Try to parse as JSON
let entry;
try {
entry = JSON.parse(trimmed);
} catch {
// Not JSON - could be plain text output
// Skip metadata/debug lines
if (trimmed.startsWith("{") || trimmed.startsWith("[")) {
continue; // Malformed JSON, skip
}
// Include non-JSON text in reasoning if substantive
if (trimmed.length > 20 && !trimmed.match(/^\d{4}-\d{2}-\d{2}T/)) {
markdown += `${trimmed}\n\n`;
}
continue;
}

// Process structured JSON entries
if (!entry || typeof entry !== "object") continue;

const entryType = entry.type || entry.event || "";

switch (entryType) {
case "message":
case "thinking":
case "reasoning": {
const content = entry.content || entry.text || entry.message || "";
if (content && content.length > 10) {
markdown += `${truncateString(content, 500)}\n\n`;
logEntries.push({
type: "assistant",
message: {
content: [{ type: "text", text: content }],
},
});
}
break;
}

case "tool_call":
case "tool_use": {
toolCallCount++;
const toolName = entry.name || entry.tool || entry.function || "unknown";
const params = entry.input || entry.arguments || entry.params || {};
const paramsStr = typeof params === "string" ? params : JSON.stringify(params, null, 2);

markdown += formatOpenClawToolCall(toolName, paramsStr, "", "⏳");

const toolUseId = `tool_${logEntries.length}`;
logEntries.push({
type: "assistant",
message: {
content: [
{
type: "tool_use",
id: toolUseId,
name: toolName,
input: typeof params === "string" ? { params } : params,
},
],
},
});
break;
}

case "tool_result": {
const result = entry.result || entry.output || entry.content || "";
const resultStr = typeof result === "string" ? result : JSON.stringify(result, null, 2);
const isError = entry.is_error || entry.error || false;
const toolUseId = `tool_result_${logEntries.length}`;

logEntries.push({
type: "user",
message: {
content: [
{
type: "tool_result",
tool_use_id: toolUseId,
content: resultStr,
is_error: isError,
},
],
},
});
Comment on lines +107 to +125

Copilot AI Feb 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tool_result entries are emitted with a fresh tool_use_id (tool_result_${logEntries.length}), which will never match the id generated for the preceding tool_use entry. This breaks log_parser_shared.cjs pairing logic (it joins tool results to tool calls by matching tool_use_id to the tool_use id), so tool results won’t render under their corresponding tool call.

Track the most recent (or a queue/stack of) generated tool_use IDs and reuse that ID for the subsequent tool_result entry (or use an ID from the OpenClaw JSON if one exists). Also coerce is_error to a boolean (e.g., Boolean(entry.is_error || entry.error)), since entry.error can be a string.

Copilot uses AI. Check for mistakes.
break;
}

case "error": {
const errorMsg = entry.message || entry.error || JSON.stringify(entry);
markdown += `❌ **Error:** ${truncateString(errorMsg, 200)}\n\n`;
break;
}

case "mcp_init":
case "mcp_connected": {
const serverName = entry.server || entry.name || "";
if (serverName) {
markdown += `✅ MCP Server connected: **${serverName}**\n\n`;
}
break;
}

case "mcp_error":
case "mcp_failed": {
const serverName = entry.server || entry.name || "";
const error = entry.error || entry.message || "";
if (serverName) {
mcpFailures.push(serverName);
markdown += `❌ MCP Server failed: **${serverName}** - ${error}\n\n`;
}
break;
}

case "usage":
case "token_usage": {
const tokens = entry.total_tokens || entry.tokens || 0;
if (tokens > totalTokens) {
totalTokens = tokens;
}
break;
}

default:
// Skip unknown entry types
break;
}
}

// Add commands and tools section
markdown += "## 🤖 Commands and Tools\n\n";
if (toolCallCount > 0) {
markdown += `**Tool Calls:** ${toolCallCount}\n\n`;
} else {
markdown += "No tool calls detected.\n\n";
}

// Add information section
markdown += "## 📊 Information\n\n";
if (totalTokens > 0) {
markdown += `**Total Tokens Used:** ${totalTokens.toLocaleString()}\n\n`;
}

return {
markdown,
logEntries,
mcpFailures,
maxTurnsHit: false, // OpenClaw uses timeout, not max-turns
};
}

/**
* Format an OpenClaw tool call with HTML details
* @param {string} toolName - The tool name
* @param {string} params - The parameters as JSON string
* @param {string} response - The response as JSON string
* @param {string} statusIcon - The status icon
* @returns {string} Formatted HTML details string
*/
function formatOpenClawToolCall(toolName, params, response, statusIcon) {
const totalTokens = estimateTokens(params) + estimateTokens(response);

let metadata = "";
if (totalTokens > 0) {
metadata = `<code>~${totalTokens}t</code>`;
}

const summary = `<code>${toolName}</code>`;

const sections = [];

if (params && params.trim()) {
sections.push({
label: "Parameters",
content: params,
language: "json",
});
}

if (response && response.trim()) {
sections.push({
label: "Response",
content: response,
language: "json",
});
}

return formatToolCallAsDetails({
summary,
statusIcon,
metadata,
sections,
});
}

// Export for testing
if (typeof module !== "undefined" && module.exports) {
module.exports = {
main,
parseOpenClawLog,
formatOpenClawToolCall,
};
}
Loading
Loading