Add colour to logs

[tim-x-y-z]

Description

it would be nice to have the log box colour the logs based on the log attributes.
specifically using the colour schemes attached to the logs display them accordingly.
this would be great to more easily see the different log levels.

perhaps another path is to have airflow use its own default set of colours and do it based on the log level it receives.

Use case/motivation

coloured logs based on their level

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[rawwar]

From what I remember, Unless you use a chrome extension which can convert and show ansi color's, you are going to see something like this

Summary
-------------�[0m
�[0;32mSucceeded: 24�[0m (�[0;32mchanged=3�[0m)
�[0;36mFailed:     0�[0m
�[0;36m-------------
Total states run:     24�[0m

above log taken from here
Its been a while I saw this. But, I remember using structlog package for this. But, that was from a while ago.

[tim-x-y-z]

how about then a piece on a side of the logs ?
in grafana it displays something like:

i think sentry does something similar as well

[rawwar]

how about then a piece on a side of the logs ? in grafana it displays something like:

i think sentry does something similar as well

I mentioned about the way you would see the logs in a web browser. In command line, I think you can see colors.

Sorry, I don't mean to say that its not possible. I think, the current log tab is just a window showing text and I was talking about it.

[potiuk]

Sorry, I don't mean to say that its not possible. I think, the current log tab is just a window showing text and I was talking about it.

Well it's a javascript-controlled window. It can display anything, it's just a matter of formatting the text you put there. And yes that woudl be a nice option to be able to see the logs colored - we already show them colored in terminal, so showing them colored in the UI would be nice. I think both options, coloring with ANSI colors similarly to what we do in terminal or the grafana-like approach is good. Someone would need to design, propose get positive feedback and implement it though

[tirkarthi]

Changes were made in #35280 to log tab to allow links which also allows html instead of a code block. I tried a rough attempt patch like below which applies green to info and red to error. Note files in the logs don't have ANSI color codes stored so this has to be done in frontend based on the log level for the statement. I can go ahead with a PR if okay,

diff --git a/airflow/www/static/js/dag/details/taskInstance/Logs/utils.ts b/airflow/www/static/js/dag/details/taskInstance/Logs/utils.ts
index c3c4084044..3e805dab45 100644
--- a/airflow/www/static/js/dag/details/taskInstance/Logs/utils.ts
+++ b/airflow/www/static/js/dag/details/taskInstance/Logs/utils.ts
@@ -101,9 +101,10 @@ export const parseLogs = (
     ) {
       // sanitize the lines to remove any tags that may cause HTML injection
       const sanitizedLine = sanitizeHtml(parsedLine, {
-        allowedTags: ["a"],
+          allowedTags: ["a", "span"],
         allowedAttributes: {
-          a: ["href", "target", "style"],
+            a: ["href", "target", "style"],
+	    span: ["style"],
         },
         transformTags: {
           a: (tagName, attribs) => {
@@ -113,6 +114,13 @@ export const parseLogs = (
               attribs,
             };
           },
+	    span: (tagName, attribs) => {
+            attribs.style = "color: red; text-decoration: none;";
+            return {
+              tagName: "span",
+              attribs,
+            };
+          }
         },
       });
 
@@ -122,7 +130,14 @@ export const parseLogs = (
         '<a href="$1" target="_blank" style="color: blue; text-decoration: underline;">$1</a>'
       );
 
-      parsedLines.push(lineWithHyperlinks);
+	let lineWithColor = sanitizedLine;
+	if (sanitizedLine.includes("INFO")) {
+	    lineWithColor = `<span style="color: green; text-decoration: none;"> ${sanitizedLine} </span>`
+	} else if (sanitizedLine.includes("ERROR")) {
+	    lineWithColor = `<span style="color: red; text-decoration: none;"> ${sanitizedLine} </span>`
+	}
+
+      parsedLines.push(lineWithColor);
     }
   });

[eladkal]

@tirkarthi feel free to raise PR. We need to make sure we don't end up in color clash after #37985

[romanzdk]

This would be super useful! I would keep info logs black though. Yellow warning and red error :)

[tirkarthi]

I just realized since ansi codes are now converted to html this is already possible with a custom log handler. Below is a custom handler to print lines with error or exception in bold red. Maybe we can add a default color handler to airflow so that users can switch to this if needed from airflow.cfg along with add it as a recipe in logs documentation to customize color, style etc. as per needs.

Color logs :

# airflow.cfg changes in logging section

logging_config_class = log_config.LOGGING_CONFIG
task_log_reader = color_handler

# ~/airflow/config/log_config.py

from copy import deepcopy
import os

from airflow.config_templates.airflow_local_settings import DEFAULT_LOGGING_CONFIG

LOGGING_CONFIG = deepcopy(DEFAULT_LOGGING_CONFIG)


LOGGING_CONFIG["handlers"]["color_handler"] = {
    'class': "airflow.utils.log.file_task_handler.ColorFileHandler", # replace this with correct path
    'formatter': 'airflow',
    'base_log_folder': '/home/karthikeyan/airflow/logs',
    'filters': ['mask_secrets']
}


LOGGING_CONFIG["loggers"]["airflow.task"]["handlers"] = ["task", "color_handler"]

class ColorFileHandler(FileTaskHandler):

    ERROR_KEYWORDS = ["error", "exception"]

    def _read(self,
              ti: TaskInstance,
              try_number: int,
              metadata: dict[str, Any] | None = None):

        import colorama

        log, metadata = super()._read(ti, try_number, metadata)
        processed_lines = []

        for line in log.splitlines():
            if any(keyword in line.lower() for keyword in self.ERROR_KEYWORDS):
                if line.startswith("["):
                    timestamp, level, msg = line.split(maxsplit=2)
                    processed_lines.append(f'{timestamp} {level} {colorama.Style.BRIGHT}{colorama.Fore.RED}{msg}{colorama.Fore.RESET}{colorama.Style.RESET_ALL}')
                else:
                    processed_lines.append(f'{colorama.Style.BRIGHT}{colorama.Fore.RED}{line}{colorama.Fore.RESET}{colorama.Style.RESET_ALL}')
            else:
                processed_lines.append(line)

        return "\n".join(processed_lines), metadata