Skip to content

channel support #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
newbe36524 merged 7 commits into from
Feb 15, 2026
Merged

channel support #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ba23ad4
release channel support
newbe36524 Feb 15, 2026
8678f19
8
newbe36524 Feb 15, 2026
0424ae8
1
newbe36524 Feb 15, 2026
f55ef44
1
newbe36524 Feb 15, 2026
bb3af96
1
newbe36524 Feb 15, 2026
715c9a8
2
newbe36524 Feb 15, 2026
57375ed
1
newbe36524 Feb 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/sync-azure-storage.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ name: Sync Release to Azure Storage

# This workflow is called by build.yml after a release is created.
on:
push:
branches:
- syncToAzure
workflow_call:
inputs:
release_tag:
Expand Down
4 changes: 4 additions & 0 deletions .nuke/build.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,10 @@
"description": "Azure upload retries",
"format": "int32"
},
"ChannelMapping": {
"type": "string",
"description": "Custom channel mapping (JSON format)"
},
"FeishuWebhookUrl": {
"type": "string",
"description": "Feishu webhook URL for notifications",
Expand Down
229 changes: 215 additions & 14 deletions nukeBuild/Adapters/AzureBlobAdapter.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
using Azure.Storage.Blobs.Models;
using AzureStorage;
using System.Text.Json;
using System.Security.Cryptography;
using Utils;

namespace Adapters;

Expand All @@ -12,10 +14,34 @@ namespace Adapters;
public class AzureBlobAdapter : IAzureBlobAdapter
{
private readonly AbsolutePath _rootDirectory;
private Dictionary<string, string> _customChannelMapping;

public AzureBlobAdapter(AbsolutePath rootDirectory)
public AzureBlobAdapter(AbsolutePath rootDirectory, string channelMappingJson = "")
{
_rootDirectory = rootDirectory;
_customChannelMapping = ParseChannelMapping(channelMappingJson);
}

/// <summary>
/// Parses custom channel mapping from JSON string
/// </summary>
/// <param name="channelMappingJson">JSON string mapping version patterns to channels</param>
/// <returns>Dictionary of version patterns to channel names</returns>
private static Dictionary<string, string> ParseChannelMapping(string channelMappingJson)
{
if (string.IsNullOrWhiteSpace(channelMappingJson))
return new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);

try
{
var mapping = JsonSerializer.Deserialize<Dictionary<string, string>>(channelMappingJson);
return mapping ?? new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
}
catch (JsonException)
{
Log.Warning("Invalid channel mapping JSON, using default rules");
return new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
}
}

public async Task<bool> ValidateSasUrlAsync(string sasUrl)
Expand Down Expand Up @@ -75,13 +101,41 @@ public async Task<AzureBlobPublishResult> UploadArtifactsAsync(List<string> file
: $"{versionPrefix}{fileName}";

var blobClient = containerClient.GetBlobClient(blobName);
Log.Information("Uploading: {File} -> {Container}/{Blob}", fileName, options.ContainerName, blobName);

await using var stream = File.OpenRead(filePath);
await blobClient.UploadAsync(stream, overwrite: true);
var blobUrl = blobClient.Uri.ToString();
result.UploadedBlobs.Add(blobUrl);
Log.Information("Upload successful: {Url}", blobUrl);
// Check if blob exists and compare hashes to skip unnecessary uploads
bool shouldUpload = true;
if (await blobClient.ExistsAsync())
{
var properties = await blobClient.GetPropertiesAsync();
var remoteHash = properties.Value.ContentHash;

// Calculate local file hash
byte[] localHash;
await using (var stream = File.OpenRead(filePath))
using (var md5 = MD5.Create())
{
localHash = await md5.ComputeHashAsync(stream);
}

// Compare hashes
if (remoteHash != null && localHash.SequenceEqual(remoteHash))
{
Log.Information("Skipping {File} (unchanged, hash: {Hash})", fileName, Convert.ToHexString(localHash)[..8]);
shouldUpload = false;
result.SkippedBlobs.Add(blobClient.Uri.ToString());
}
}

if (shouldUpload)
{
Log.Information("Uploading: {File} -> {Container}/{Blob}", fileName, options.ContainerName, blobName);

await using var stream = File.OpenRead(filePath);
await blobClient.UploadAsync(stream, overwrite: true);
var blobUrl = blobClient.Uri.ToString();
result.UploadedBlobs.Add(blobUrl);
Log.Information("Upload successful: {Url}", blobUrl);
}
Comment on lines +105 to +138
Copy link

@coderabbitai coderabbitai bot Feb 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Azure SDK UploadAsync ContentHash Content-MD5 automatic computation persistence

💡 Result:

  • Azure Storage does not “magically” persist an MD5 for every upload. What happens depends on which REST operation the SDK ends up using:

    • If the upload is a single Put Blob request and you omit Content-MD5, the service generates an MD5 and it shows up later in blob properties / listings as Content-MD5. [2]
    • If the upload is a block upload (Put Block / Put Block List), the service generally does not compute/persist a whole-blob MD5 for you; you must compute it client-side and set it explicitly (via the blob property header x-ms-blob-content-md5 / SDK “content hash” header). [1]

  • UploadAsync in Azure SDK can switch between those paths. The .NET SDK’s Upload/UploadAsync may use PutBlob for small uploads and PutBlock/PutBlockList for larger uploads, so you can see “sometimes MD5 exists, sometimes it’s null” behavior unless you always set it yourself. [3]

  • To persist an MD5 using the SDK, you must supply it (the SDK won’t reliably compute-and-store it for you across all upload strategies). For ADLS Gen2 / DataLake, this is done by setting HttpHeaders.ContentHash in upload options. [5]

Sources: Microsoft Learn REST Put Blob (service MD5 behavior) [2]; Microsoft Q&A explanation of PutBlob vs PutBlock/PutBlockList MD5 persistence [1]; Azure SDK upload method behavior (PutBlob vs blocks) [3]; DataLake UploadAsync + ContentHash example [5].

🌐 Web query:

Azure.Storage.Blobs BlobUploadOptions BlobHttpHeaders ContentHash documentation

💡 Result:

In Azure.Storage.Blobs (.NET):

  • BlobUploadOptions.HttpHeaders lets you set standard blob HTTP headers (including content headers) as part of an upload. [1]
  • BlobHttpHeaders.ContentHash is the blob’s MD5 (as a byte[]). If you specify it on upload, the service validates the transmitted content; if the MD5 doesn’t match, the request fails with HTTP 400 (Bad Request). [2]
  • When reading blob properties, BlobProperties.ContentHash is returned (as byte[]) when the blob has an MD5 and the response is for the full blob, so clients can validate integrity. [3]

Example (upload with MD5 + content type):

using Azure.Storage.Blobs.Models;
using System.Security.Cryptography;

byte[] md5 = MD5.HashData(dataBytes); // 16 bytes

var options = new BlobUploadOptions
{
    HttpHeaders = new BlobHttpHeaders
    {
        ContentType = "application/octet-stream",
        ContentHash = md5
    }
};

await blobClient.UploadAsync(new BinaryData(dataBytes), options);

Sources: [1] [2] [3]

🌐 Web query:

Azure Blob Storage UploadAsync automatically compute Content-MD5 hash

💡 Result:

UploadAsync in the Azure .NET v12 Blob SDK does not reliably/always compute and set the blob Content-MD5 for you.

  • Upload/UploadAsync may upload as a single Put Blob for small payloads, but switches to Put Block / Put Block List for larger uploads. (azure.github.io)
  • For block uploads, the service won’t compute/populate Content-MD5 automatically; you’re expected to compute it client-side and set it yourself. (learn.microsoft.com)

What to do (compute it yourself and set it on upload)

In Azure.Storage.Blobs, set BlobHttpHeaders.ContentHash (MD5 bytes) via BlobUploadOptions:

using System.Security.Cryptography;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;

await using var stream = File.OpenRead(path);
byte[] md5 = MD5.HashData(stream);
stream.Position = 0;

await blobClient.UploadAsync(stream, new BlobUploadOptions
{
    HttpHeaders = new BlobHttpHeaders { ContentHash = md5 }
});

(Equivalent tooling example: AzCopy/Storage Explorer “Put MD5” computes MD5 client-side and stores it to Content-MD5.) (learn.microsoft.com)

Citations:

Set ContentHash on upload to ensure hash persistence across all file sizes.

Azure's UploadAsync uses different strategies based on file size: single Put Blob for small uploads (auto-computes MD5) and Put Block / Put Block List for larger uploads (does not auto-persist MD5). Since the current code omits ContentHash, the hash comparison skip logic will only work for small files and fail inconsistently for large files.

Move the hash calculation before the existence check and pass it via BlobUploadOptions.HttpHeaders.ContentHash to ensure it's always persisted:

🔧 Suggested fix 
-                bool shouldUpload = true;
-                if (await blobClient.ExistsAsync())
-                {
-                    var properties = await blobClient.GetPropertiesAsync();
-                    var remoteHash = properties.Value.ContentHash;
-
-                    // Calculate local file hash
-                    byte[] localHash;
-                    await using (var stream = File.OpenRead(filePath))
-                    using (var md5 = MD5.Create())
-                    {
-                        localHash = await md5.ComputeHashAsync(stream);
-                    }
+                // Calculate local file hash (used for compare + upload headers)
+                byte[] localHash;
+                await using (var hashStream = File.OpenRead(filePath))
+                using (var md5 = MD5.Create())
+                {
+                    localHash = await md5.ComputeHashAsync(hashStream);
+                }
+
+                bool shouldUpload = true;
+                if (await blobClient.ExistsAsync())
+                {
+                    var properties = await blobClient.GetPropertiesAsync();
+                    var remoteHash = properties.Value.ContentHash;

                     // Compare hashes
                     if (remoteHash != null && localHash.SequenceEqual(remoteHash))
                     {
                         Log.Information("Skipping {File} (unchanged, hash: {Hash})", fileName, Convert.ToHexString(localHash)[..8]);
                         shouldUpload = false;
                         result.SkippedBlobs.Add(blobClient.Uri.ToString());
                     }
                 }

                 if (shouldUpload)
                 {
                     Log.Information("Uploading: {File} -> {Container}/{Blob}", fileName, options.ContainerName, blobName);
-
-                    await using var stream = File.OpenRead(filePath);
-                    await blobClient.UploadAsync(stream, overwrite: true);
+                    await using var stream = File.OpenRead(filePath);
+                    var uploadOptions = new BlobUploadOptions
+                    {
+                        HttpHeaders = new BlobHttpHeaders { ContentHash = localHash }
+                    };
+                    await blobClient.UploadAsync(stream, uploadOptions);
                     var blobUrl = blobClient.Uri.ToString();
                     result.UploadedBlobs.Add(blobUrl);
                     Log.Information("Upload successful: {Url}", blobUrl);
                 }
🤖 Prompt for AI Agents 
In `@nukeBuild/Adapters/AzureBlobAdapter.cs` around lines 105 - 138, Compute the
local MD5 hash (using MD5.Create() and md5.ComputeHashAsync on the file stream)
before checking blobClient.ExistsAsync(), then use that localHash for the
pre-upload comparison against properties.Value.ContentHash (remoteHash) and,
when calling blobClient.UploadAsync, pass a BlobUploadOptions with
HttpHeaders.ContentHash set to localHash so the hash is persisted for both small
and large uploads; update the existing logic around localHash, remoteHash,
shouldUpload, and the UploadAsync call (references: blobClient,
GetPropertiesAsync, remoteHash, localHash, UploadAsync,
BlobUploadOptions.HttpHeaders.ContentHash) so skip/upload behavior remains
correct and consistent for all file sizes.

}

result.Success = true;
Expand Down Expand Up @@ -125,7 +179,8 @@ public async Task<string> GenerateIndexOnlyAsync(AzureBlobPublishOptions options

var jsonOptions = new JsonSerializerOptions
{
WriteIndented = !minify
WriteIndented = !minify,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};

var jsonContent = JsonSerializer.Serialize(indexData, jsonOptions);
Expand Down Expand Up @@ -321,35 +376,43 @@ public async Task<string> GenerateIndexFromBlobsAsync(AzureBlobPublishOptions op
})
.ToList();

return new
return new VersionGroup
{
version = kv.Key,
files = versionFiles
Version = kv.Key,
Files = versionFiles.Cast<object>().ToList()
};
})
.ToList();

// Build channels object
var channelsData = BuildChannelsObject(versionList);

var indexData = new
{
updatedAt = DateTime.UtcNow.ToString("o"),
versions = versionList
versions = versionList,
channels = channelsData
};

var jsonOptions = new JsonSerializerOptions
{
WriteIndented = !minify
WriteIndented = !minify,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};

var jsonContent = JsonSerializer.Serialize(indexData, jsonOptions);

await File.WriteAllTextAsync(outputPath, jsonContent);

var versionCount = versionList.Count;
var totalFiles = versionList.Sum(v => v.files.Count);
var totalFiles = versionList.Sum(v => v.Files.Count);

Log.Information("✅ Index.json generated at: {Path}", outputPath);
Log.Information(" Versions: {Count}", versionCount);
Log.Information(" Total files: {Count}", totalFiles);
Log.Information("=== index.json Content ===");
Log.Information(jsonContent);
Log.Information("=== End of index.json ===");

return jsonContent;
}
Expand All @@ -359,6 +422,126 @@ public async Task<string> GenerateIndexFromBlobsAsync(AzureBlobPublishOptions op
return string.Empty;
}
}

/// <summary>
/// Extracts channel name from version string
/// - No dash (-) = stable (正式版), e.g., "1.0.0", "2.3.1"
/// - With dash (-) = check prerelease identifier for channel
/// </summary>
/// <param name="version">Version string (e.g., "1.0.0", "0.1.0-beta.11")</param>
/// <returns>Channel name as string</returns>
private string ExtractChannelFromVersion(string version)
{
if (string.IsNullOrWhiteSpace(version))
return "beta";

// Remove 'v' prefix if present
version = version.TrimStart('v', 'V');

// Check custom channel mapping first
foreach (var (pattern, channel) in _customChannelMapping)
{
if (version.Contains(pattern, StringComparison.OrdinalIgnoreCase))
return channel;
}

// Check for dash (-) to determine if it's a stable release
var dashIndex = version.IndexOf('-');
if (dashIndex <= 0)
{
// No prerelease identifier = stable (正式版)
return "stable";
}

// Has prerelease identifier - determine channel
var prerelease = version.Substring(dashIndex + 1).ToLowerInvariant();

if (prerelease.StartsWith("beta.") || prerelease.StartsWith("beta"))
return "beta";
if (prerelease.StartsWith("canary.") || prerelease.StartsWith("canary"))
return "canary";
if (prerelease.StartsWith("alpha.") || prerelease.StartsWith("alpha"))
return "alpha";
if (prerelease.StartsWith("dev.") || prerelease.StartsWith("dev"))
return "dev";
if (prerelease.StartsWith("preview.") || prerelease.StartsWith("preview"))
return "preview";
if (prerelease.StartsWith("rc.") || prerelease.StartsWith("rc"))
return "preview";

// Other prerelease types default to preview
return "preview";
}

/// <summary>
/// Groups versions by their channel
/// </summary>
/// <param name="versions">List of version groups</param>
/// <returns>Dictionary mapping channel names to list of version strings</returns>
private Dictionary<string, List<string>> GroupVersionsByChannel(List<VersionGroup> versions)
{
var channelGroups = new Dictionary<string, List<string>>(StringComparer.OrdinalIgnoreCase);

foreach (var version in versions)
{
var channel = ExtractChannelFromVersion(version.Version);

if (!channelGroups.ContainsKey(channel))
{
channelGroups[channel] = new List<string>();
}

channelGroups[channel].Add(version.Version);
}

return channelGroups;
}

/// <summary>
/// Builds the channels object for index.json
/// Contains latest version and versions array for each channel
/// </summary>
/// <param name="versions">List of version groups</param>
/// <returns>Dictionary mapping channel names to channel information</returns>
private Dictionary<string, object> BuildChannelsObject(List<VersionGroup> versions)
{
var channelGroups = GroupVersionsByChannel(versions);
var channelsData = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);

foreach (var (channelName, versionStrings) in channelGroups)
{
// Parse versions and find the latest using Semver precedence comparison
Semver.SemVersion latestVersion = null;
string latestVersionString = null;

foreach (var versionStr in versionStrings)
{
if (SemverExtensions.TryParseVersion(versionStr, out var semver))
{
if (latestVersion == null ||
Semver.SemVersion.ComparePrecedence(semver, latestVersion) > 0)
{
latestVersion = semver;
latestVersionString = versionStr;
}
}
}

// Fallback to first version string if no valid Semver found
if (latestVersionString == null && versionStrings.Count > 0)
{
latestVersionString = versionStrings[0];
}

channelsData[channelName] = new ChannelInfo
{
Latest = latestVersionString ?? "",
Versions = versionStrings
};
}

return channelsData;
}
}

/// <summary>
Expand All @@ -370,3 +553,21 @@ public class AzureBlobInfo
public long Size { get; init; }
public DateTime LastModified { get; init; }
}

/// <summary>
/// Version group for index.json generation
/// </summary>
public class VersionGroup
{
public required string Version { get; init; }
public List<object> Files { get; init; } = new();
}

/// <summary>
/// Channel information for index.json
/// </summary>
public class ChannelInfo
{
public required string Latest { get; init; }
public List<string> Versions { get; init; } = new();
}
1 change: 1 addition & 0 deletions nukeBuild/AzureStorageConfiguration.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ public class AzureBlobPublishResult
{
public bool Success { get; set; }
public List<string> UploadedBlobs { get; set; } = new();
public List<string> SkippedBlobs { get; set; } = new();
public string ErrorMessage { get; set; } = string.Empty;
public List<string> Warnings { get; set; } = new();
}
Expand Down
8 changes: 6 additions & 2 deletions nukeBuild/Build.AzureStorage.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@
{
Log.Information("=== 步骤 2: 上传到 Azure Blob ===");

var adapter = new AzureBlobAdapter(RootDirectory);
var adapter = new AzureBlobAdapter(RootDirectory, ChannelMapping);

if (!await adapter.ValidateSasUrlAsync(AzureBlobSasUrl))
{
Expand All @@ -164,6 +164,10 @@

Log.Information("✅ 构建产物已上传");
Log.Information(" 上传文件数: {Count}", result.UploadedBlobs.Count);
if (result.SkippedBlobs.Count > 0)
{
Log.Information(" 跳过文件数: {Count} (已存在且哈希一致)", result.SkippedBlobs.Count);
}
}
else
{
Expand All @@ -174,7 +178,7 @@
{
Log.Information("=== 步骤 3: 生成并上传 index.json ===");

var adapter = new AzureBlobAdapter(RootDirectory);
var adapter = new AzureBlobAdapter(RootDirectory, ChannelMapping);

if (!await adapter.ValidateSasUrlAsync(AzureBlobSasUrl))
{
Expand Down Expand Up @@ -225,7 +229,7 @@
/// <summary>
/// 使用 gh CLI 获取最新 release tag
/// </summary>
private async Task<string?> GetLatestReleaseTagUsingGhAsync()

Check warning on line 232 in nukeBuild/Build.AzureStorage.cs

View workflow job for this annotation

GitHub Actions / Sync Release Assets to Azure Storage 

The annotation for nullable reference types should only be used in code within a '#nullable' annotations context.
{
try
{
Expand Down
3 changes: 3 additions & 0 deletions nukeBuild/Build.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@
[Parameter("Azure Blob SAS URL for authentication and upload")]
[Secret] readonly string AzureBlobSasUrl = "";

[Parameter("Skip Azure Blob publish")] readonly bool SkipAzureBlobPublish = false;

Check warning on line 30 in nukeBuild/Build.cs

View workflow job for this annotation

GitHub Actions / Sync Release Assets to Azure Storage 

The field 'Build.SkipAzureBlobPublish' is assigned but its value is never used

[Parameter("Generate Azure index.json")] readonly bool AzureGenerateIndex = true;

Check warning on line 32 in nukeBuild/Build.cs

View workflow job for this annotation

GitHub Actions / Sync Release Assets to Azure Storage 

The field 'Build.AzureGenerateIndex' is assigned but its value is never used

[Parameter("Azure upload retries")] readonly int AzureUploadRetries = 3;

Expand All @@ -56,19 +56,22 @@
readonly string ReleaseTag = "";

[Parameter("Release channel (stable, beta, dev)")]
readonly string ReleaseChannel = "beta";

Check warning on line 59 in nukeBuild/Build.cs

View workflow job for this annotation

GitHub Actions / Sync Release Assets to Azure Storage 

The field 'Build.ReleaseChannel' is assigned but its value is never used

[Parameter("Custom channel mapping (JSON format)")]
readonly string ChannelMapping = "";

[Parameter("Feishu webhook URL for notifications")]
[Secret] readonly string FeishuWebhookUrl = "";

Check warning on line 65 in nukeBuild/Build.cs

View workflow job for this annotation

GitHub Actions / Sync Release Assets to Azure Storage 

The field 'Build.FeishuWebhookUrl' is assigned but its value is never used

[Parameter("GitHub Actions run URL")]
readonly string GitHubRunUrl = "";

[Parameter("GitHub SHA")]
readonly string GitHubSha = "";

Check warning on line 71 in nukeBuild/Build.cs

View workflow job for this annotation

GitHub Actions / Sync Release Assets to Azure Storage 

The field 'Build.GitHubSha' is assigned but its value is never used

[Parameter("GitHub actor")]
readonly string GitHubActor = "";

Check warning on line 74 in nukeBuild/Build.cs

View workflow job for this annotation

GitHub Actions / Sync Release Assets to Azure Storage 

The field 'Build.GitHubActor' is assigned but its value is never used

#endregion

Expand Down
Loading
Loading