Development

.github/instructions/html-lang.instructions.md

@@ -0,0 +1,23 @@
+---
+applyTo: '**/*.html'
+---
+
+# HTML Language Guide
+
+- Use 4 spaces per indentation level. No tabs.
+
+- Use double quotes for all HTML attributes. Ex: `<div class="my-class">`
+
+- Self-closing tags should include the trailing slash. Ex: `<img src="image.png" />`
+
+- Use semantic HTML5 elements where appropriate. Ex: `<header>`, `<nav>`, `<main>`, `<footer>`
+
+- Include ARIA roles and attributes to enhance accessibility. Ex: `role="button"`, `aria-label="Close"`
+
+- Keep inline styles to a minimum; prefer CSS classes for styling.
+
+- Use comments to separate major sections of the HTML document. Ex: `<!-- Header Section -->`
+
+- Use Jinja templating syntax consistently for dynamic content, settings, and configuration. Ex: `{{ variable }}`, `{% if condition %}`
+
+- Use bootstrap classes for layout, styling consistency, and data presentation. Ex: `class="container"`, `class="row"`, `class="col-md-6"`, `data-bs-toggle="modal"`

.github/instructions/javascript-lang.instructions.md

@@ -18,4 +18,6 @@ applyTo: '**/*.js'
 
 - Do not use display:none. Instead add and remove the d-none class when hiding or showing elements.
 
-- Prefer inline html notifications or toast messages using Bootstrap alert classes over browser alert() calls.
+- Prefer inline html notifications or toast messages using Bootstrap alert classes over browser alert() calls.
+
+- Use bootstrap modals for confirmation dialogs instead of native browser confirm() calls.

.github/instructions/python-lang.instructions.md

@@ -4,12 +4,37 @@ applyTo: '**/*.py'
 
 # Python Language Guide
 
-- Files should start with a comment of the file name. Ex: `# functions_personal_agents.py`
+- All files MUST start with a comment of the file name. Ex: `# functions_personal_agents.py`
 
-- Imports should be grouped at the top of the document after the module docstring, unless otherwise indicated by the user or for performance reasons in which case the import should be as close as possible to the usage with a documented note as to why the import is not at the top of the file.
+## Rule: Follow Standard Python Conventions
 
+- Standard python conventions, such as the use of snake case, must be followed.
+
+## Rule: Imports Must Be Organized and at the Top of the File !IMPORTANT
+
+- IMPORTANT: `from` and `import` statements MUST be grouped at the top of the document after the module docstring, unless otherwise indicated by the code writer or for performance reasons in which case the import should be as close as possible to the usage with a comment explaining why the import is not at the top of the file. CodeQL hammers us on this in the findings. If you find imports that are not at the top of the file, move them to the top and add a comment if there is a reason they cannot be moved. This also helps prevent multuple imports of the same module in different places which can lead to confusion and maintenance issues.
+
+## Rule: Indentation, Logging, and Decorators
 - Use 4 spaces per indentation level. No tabs.
 
 - Code and definitions should occur after the imports block.
 
-- Prefer log_event from functions_appinsights.py for logging activites.
+- All logging should used tag based prefixes. Ex: [GPTClient] or [SKLoader] to identify the source of the log message and make it easier to trace. Tags should be enclosed in square brackets. Tags should be generalized to the operation that is occurring. Any existins logging that is missing tags should have tags added.
+
+- Always import `log_event` from `functions_appinsights.py` for any logging activities.
+
+- All files MUST use the `log_event` function from `functions_appinsights.py` for production-type logging activities. Ensure that all log events include relevant contextual information as properties to facilitate effective monitoring and troubleshooting. Messages returned to the client should not contain sensitive information, but should be informative enough to understand the context of the event.
+
+- Prefer using `log_event` from functions_appinsights.py for production-type logging activites.
+
+- Use `log_event` from functions_appinsights.py with debug_only=True for debug logging purposes. All method, calls, warnings and errors should be debug_logged. 
+
+- Files with routes MUST import `from swagger_wrapper import swagger_route, get_auth_security` and use the `@swagger_route(security=get_auth_security())` decorator for all route functions.
+
+- When editing group workspace content, always use the `assert_group_role(user_id, group_id, allowed_roles=("Owner", "Admin", "DocumentManager", "User"))` function to verify the user's current membership and role in the group before allowing access to group-scoped resources or operations. This ensures that users cannot access or modify group content based solely on a potentially stale or tampered `activeGroupOid` reference. Roles should vary based on the level of access required for the operation, but should always include "Owner" and "Admin" as allowed roles.
+
+- Unless otherwise indicated, all routes MUST include `@login_required` decorator. 
+
+- Always use f-strings for string interpolation. Ex: `f"User ID: {user_id}"` instead of `"User ID: {}".format(user_id)"`
+
+- Never use `except:` without specifying the exception type. Always catch specific exceptions or use `except Exception as ex:` to capture the exception details. This also avoids accidentally catching system-exiting exceptions like `KeyboardInterrupt` or `SystemExit`.

.github/instructions/ui_tests.instructions.md

@@ -0,0 +1,158 @@
+---
+applyTo: '**/*.html, **/*.js, **/*.css'
+---
+
+# UI Tests
+
+- Always create or update Azure Playwright UI tests when changing HTML, CSS, or JavaScript that affects rendering, user interaction, layout, client-side state, or browser workflows. Leverage the `azure-mgmt-playwright` Python library.
+
+## Location
+
+- Store all UI tests in the `ui_tests/` folder at the root of the project.
+- Keep reusable helpers, fixtures, authentication utilities, and page models inside `ui_tests/` so UI automation stays separate from `functional_tests/`.
+- Place screenshots, traces, and other test artifacts under a dedicated subfolder such as `ui_tests/artifacts/`.
+
+## When UI Tests Are Required
+
+### Always create or update UI tests for:
+- New pages, views, or templates.
+- HTML changes that affect structure, accessibility, forms, navigation, or conditional rendering.
+- CSS changes that affect layout, spacing, responsive behavior, visibility, theming, or visual regressions.
+- JavaScript changes that affect event handlers, DOM updates, modals, tabs, filtering, validation, async loading, or error handling.
+- Bug fixes involving display logic, user flows, or browser-side regressions.
+- Features that require proof that the browser experience works end to end.
+
+### Focus areas for coverage:
+- Rendering: page content, empty states, conditional sections, and accessibility attributes.
+- Interaction: clicks, typing, keyboard navigation, dialogs, dropdowns, and form submission.
+- Styling and layout: visible states, responsive breakpoints, overflow issues, and Bootstrap-driven behavior.
+- JavaScript behavior: dynamic updates, loading indicators, validation messages, and error states.
+
+## Required Technology
+
+- Use Azure Playwright for UI automation.
+- Use the `azure-mgmt-playwright` Python package when provisioning or connecting to Azure Playwright resources.
+- Use `DefaultAzureCredential` for authentication. Never hardcode credentials, API keys, or secrets in UI tests.
+- Write UI tests in Python unless the user explicitly requests another language.
+
+## Naming Conventions
+
+### File naming:
+- Use `test_{page_or_feature}_{scenario}.py` for executable UI tests.
+- Use descriptive names tied to the user-visible workflow being validated.
+
+### Examples:
+- `test_chat_sidebar_navigation.py`
+- `test_workspace_document_filters.py`
+- `test_group_workspace_modal_validation.py`
+- `test_public_workspace_responsive_layout.py`
+
+## Suggested Directory Structure
+
+```text
+ui_tests/
+	test_chat_sidebar_navigation.py
+	test_workspace_document_filters.py
+	conftest.py
+	auth_helpers.py
+	page_models/
+	fixtures/
+	artifacts/
+```
+
+## Test Design Requirements
+
+- Keep each test independent and runnable on its own.
+- Cover the complete browser workflow: navigation, setup, action, validation, and cleanup when needed.
+- Validate both expected success paths and visible failure states.
+- Prefer stable selectors such as `id`, `name`, `data-testid`, ARIA roles, labels, and text that is intentionally user-facing.
+- Avoid brittle selectors based on deeply nested CSS paths or Bootstrap implementation details.
+- If a UI change is responsive, validate at least desktop and mobile viewport behavior.
+- If a change is accessibility-sensitive, validate visible labels, roles, focus movement, and keyboard behavior where practical.
+
+## Python UI Test Template
+
+```python
+# test_chat_sidebar_navigation.py
+"""
+UI test for chat sidebar navigation.
+
+Version: [current version from config.py when applicable]
+Implemented in: [version when fix or feature was added]
+
+This test ensures that the sidebar navigation renders correctly,
+supports keyboard and mouse interaction, and updates the visible
+chat panel without browser errors.
+"""
+
+from pathlib import Path
+
+import pytest
+from playwright.sync_api import expect
+
+
+@pytest.mark.ui
+def test_chat_sidebar_navigation(page):
+	"""Validate that users can navigate between sidebar destinations."""
+	page.goto("http://127.0.0.1:5000")
+	page.get_by_role("button", name="Open navigation").click()
+	page.get_by_role("link", name="Workspaces").click()
+
+	expect(page.get_by_role("heading", name="Workspaces")).to_be_visible()
+	expect(page.get_by_text("Recent documents")).to_be_visible()
+```
+
+## Authentication and Environment Rules
+
+- Authenticate Azure dependencies with `DefaultAzureCredential`.
+- Read environment-specific values such as base URLs, tenant-specific settings, or test accounts from environment variables or secure configuration.
+- Never commit secrets, session tokens, storage keys, or passwords.
+- If a test requires signed-in state, create a reusable login helper in `ui_tests/` rather than duplicating login steps across files.
+
+## What to Validate in UI Tests
+
+### HTML changes:
+- Key headings, landmarks, labels, buttons, links, and form elements render correctly.
+- Conditional content appears or stays hidden at the right time.
+- Accessibility-related attributes remain intact.
+
+### CSS changes:
+- Elements are visible when expected and hidden only when intended.
+- Layout behaves correctly at the supported viewport sizes.
+- Important visual states such as error, loading, selected, disabled, and hover-adjacent states are testable.
+
+### JavaScript changes:
+- Event handlers trigger the correct visible behavior.
+- DOM updates complete after async actions.
+- Validation, toast messages, modals, drawers, and loading indicators behave correctly.
+- Browser console errors should be treated as failures when they are caused by the changed workflow.
+
+## Reuse and Maintainability
+
+- Extract repeated UI actions into helper functions or page model classes.
+- Keep assertions close to the action they validate so failures are easy to diagnose.
+- Use concise setup utilities for viewport, authentication, seeded test data, and navigation.
+- Prefer a small number of focused tests over one large brittle end-to-end script.
+
+## Execution Patterns
+
+### Typical workflow:
+- Start the local app or target environment.
+- Run only the relevant UI test file while iterating.
+- Run the broader affected UI suite before completing the change.
+
+### Example commands:
+```bash
+cd ui_tests
+pytest test_chat_sidebar_navigation.py
+pytest -m ui
+```
+
+## Best Practices
+
+- Write UI tests as part of the same change that updates the HTML, CSS, or JavaScript.
+- Keep test names and docstrings explicit about the user behavior being validated.
+- Capture screenshots or traces for failures when the workflow is difficult to diagnose.
+- Avoid sleeping for fixed durations; wait for meaningful UI conditions instead.
+- Verify user-visible outcomes, not just implementation details.
+- If a bug fix changes browser behavior, add a regression UI test that fails without the fix.

.github/workflows/release-notes-check.yml

@@ -22,7 +22,7 @@ jobs:
 
     - name: Get changed files
       id: changed-files
-      uses: tj-actions/changed-files@v44
+      uses: tj-actions/changed-files@v46.0.1
       with:
         files_yaml: |
           code:

.github/workflows/swagger-route-check.yml

@@ -0,0 +1,50 @@
+name: Swagger Route Check
+
+on:
+  pull_request:
+    branches:
+      - main
+      - Development
+      - staging
+    paths:
+      - 'application/single_app/**/*.py'
+      - 'scripts/check_swagger_routes.py'
+      - '.github/workflows/swagger-route-check.yml'
+
+jobs:
+  swagger-route-check:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Get changed Python files
+        id: changed-files
+        uses: tj-actions/changed-files@v46.0.1
+        with:
+          files_yaml: |
+            route_python:
+              - 'application/single_app/**/*.py'
+              - 'scripts/check_swagger_routes.py'
+
+      - name: Run swagger route validation
+        env:
+          CHANGED_ROUTE_FILES: ${{ steps.changed-files.outputs.route_python_all_changed_files }}
+        run: |
+          if [[ -z "$CHANGED_ROUTE_FILES" ]]; then
+            echo "No changed application Python files detected."
+            exit 0
+          fi
+
+          echo "Changed Python files:"
+          printf '%s\n' "$CHANGED_ROUTE_FILES" | tr ' ' '\n'
+
+          python scripts/check_swagger_routes.py $CHANGED_ROUTE_FILES

.gitignore

@@ -45,3 +45,8 @@ flask_session
 tmp**cwd
 /tmp_images
 nul
+/.github/plans
+*.xlsx
+/artifacts/tests
+scripts/agent.json
+scripts/me.json

CLAUDE.md

@@ -58,6 +58,7 @@ return render_template('page.html', settings=public_settings)
 
 ## Version Management
 
+- Its important to update the version at the end of every plan
 - Version is stored in `config.py`: `VERSION = "X.XXX.XXX"`
 - When incrementing, only change the third segment (e.g., `0.238.024` -> `0.238.025`)
 - Include the current version in functional test file headers and documentation files
@@ -83,7 +84,7 @@ return render_template('page.html', settings=public_settings)
 
 ## Release Notes
 
-After completing code changes, offer to update `docs/explanation/release_notes.md`.
+After completing plans and code changes, offer to update `docs/explanation/release_notes.md`.
 
 - Add entries under the current version from `config.py`
 - If the version was bumped, create a new section at the top: `### **(vX.XXX.XXX)**`