Persister in json

[Hirogen]

This PR fixes

• Can't save filter v1.20.0 #456
• Settings deserialization fails if FilterParams.CurrentColumnizer can't be deserialized #470

And changes the persistence files to json files, for better readability, but also for future, older persistence files will no longer be loaded

[Copilot]

Pull Request Overview

This PR modernizes the persistence system by transitioning from XML to JSON serialization while maintaining backward compatibility with existing XML files. The changes include code style improvements with the use of var declarations and introduction of a custom JSON converter for columnizer serialization.

• Implements dual persistence (XML and JSON) with custom JSON converter for columnizers
• Replaces explicit type declarations with var throughout the codebase for improved readability
• Adds proper serialization attributes to ensure consistent JSON persistence

Reviewed Changes

Copilot reviewed 78 out of 78 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/LogExpert.Core/Classes/Persister/Persister.cs	Main persistence changes - adds JSON serialization alongside XML, implements dual save functionality
src/LogExpert.Core/Classes/Persister/ColumnizerJsonConverter.cs	New custom JSON converter for ILogLineColumnizer serialization using reflection and attributes
src/LogExpert.Core/Classes/Persister/JsonColumnizerPropertyAttribute.cs	New attribute to mark properties for JSON serialization
Multiple entity classes	Added [Serializable] attributes to support JSON serialization
src/LogExpert.Core/Classes/Highlight/HighlightEntry.cs	Renamed IsRegEx to IsRegex and updated related regex field naming
Multiple source files	Replaced explicit type declarations with var for improved code consistency

[Copilot]

Pull Request Overview

Copilot reviewed 78 out of 78 changed files in this pull request and generated 5 comments.

[Copilot]

Pull Request Overview

Copilot reviewed 78 out of 78 changed files in this pull request and generated 1 comment.

[AndisGrossteins]

I'm not sure if backwards compatibility with XML persistence is maintained.

I see two exceptions in NLog file when loading a log file with existing persistence file running the debug build from latest action run. After this the UI is borked and log view lines are blank and log tab title in empty. Manually setting columnizer fixes the log lines display.

First exception:

LogExpert.UI.Controls.LogWindow.LogWindow|Error loading persistence data: Unexpected character encountered while parsing value: <. Path '', line 0, position 0. Newtonsoft.Json.JsonReaderException Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
   at Newtonsoft.Json.JsonTextReader.ParseValue()
   at Newtonsoft.Json.JsonTextReader.Read()
   at Newtonsoft.Json.JsonReader.ReadAndMoveToContent()
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at LogExpert.Core.Classes.Persister.Persister.LoadInternal(String fileName) in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Persister\Persister.cs:line 270
   at LogExpert.Core.Classes.Persister.Persister.LoadPersistenceDataOptionsOnly(String logFileName, Preferences preferences) in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Persister\Persister.cs:line 88
   at LogExpert.UI.Controls.LogWindow.LogWindow.LoadPersistenceOptions() in D:\a\LogExpert\LogExpert\src\LogExpert.UI\Controls\LogWindow\LogWindow.cs:line 2260

Second exception trace:

LogExpert.Core.Classes.Log.LogfileReader|Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.Unexpected character encountered while parsing value: <. Path '', line 0, position 0. Newtonsoft.Json.JsonReaderException Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
   at Newtonsoft.Json.JsonTextReader.ParseValue()
   at Newtonsoft.Json.JsonTextReader.Read()
   at Newtonsoft.Json.JsonReader.ReadAndMoveToContent()
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at LogExpert.Core.Classes.Persister.Persister.LoadInternal(String fileName) in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Persister\Persister.cs:line 270
   at LogExpert.Core.Classes.Persister.Persister.LoadPersistenceData(String logFileName, Preferences preferences) in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Persister\Persister.cs:line 75
   at LogExpert.UI.Controls.LogWindow.LogWindow.LoadPersistenceData() in D:\a\LogExpert\LogExpert\src\LogExpert.UI\Controls\LogWindow\LogWindow.cs:line 2357
   at System.Windows.Forms.Control.InvokeMarshaledCallbackHelper(Object obj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Windows.Forms.Control.InvokeMarshaledCallbacks()
--- End of stack trace from previous location ---
   at System.Windows.Forms.Control.MarshaledInvoke(Control caller, Delegate method, Object[] args, Boolean synchronous)
   at System.Windows.Forms.Control.Invoke(Delegate method, Object[] args)
   at LogExpert.UI.Controls.LogWindow.LogWindow.OnLogFileReaderFinishedLoading(Object sender, EventArgs e) in D:\a\LogExpert\LogExpert\src\LogExpert.UI\Controls\LogWindow\LogWindow.cs:line 704
   at LogExpert.Core.Classes.Log.LogfileReader.OnLoadingFinished() in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Log\LogfileReader.cs:line 1807
   at LogExpert.Core.Classes.Log.LogfileReader.MonitorThreadProc() in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Log\LogfileReader.cs:line 1446

It looks like you're trying to parse XML as JSON without checking first. Maybe checking first bytes of the persistence file for XML opening tag "<logexpert>" would be an option.

[AndisGrossteins]

After persistence file is saved as JSON, loading it causes issues. One exception is thrown because Newtonsoft.Json can't deserialize LogExpert.Core.Entities.Bookmark from JSON path 'BookmarkList.0.LineNum':

Error loading persistence data: Unable to find a constructor to use for type LogExpert.Core.Entities.Bookmark. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'BookmarkList.0.LineNum', line 4, position 16. Newtonsoft.Json.JsonSerializationException Newtonsoft.Json.JsonSerializationException: Unable to find a constructor to use for type LogExpert.Core.Entities.Bookmark. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'BookmarkList.0.LineNum', line 4, position 16.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateDictionary(IDictionary dictionary, JsonReader reader, JsonDictionaryContract contract, JsonProperty containerProperty, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at LogExpert.Core.Classes.Persister.Persister.LoadInternal(String fileName) in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Persister\Persister.cs:line 270
   at LogExpert.Core.Classes.Persister.Persister.LoadPersistenceDataOptionsOnly(String logFileName, Preferences preferences) in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Persister\Persister.cs:line 88
   at LogExpert.UI.Controls.LogWindow.LogWindow.LoadPersistenceOptions() in D:\a\LogExpert\LogExpert\src\LogExpert.UI\Controls\LogWindow\LogWindow.cs:line 2260

Next load attempt same persistence file (BookmarkList element is empty now), it's struggling to deserialize Encoding:

LogExpert.UI.Controls.LogWindow.LogWindow|Error loading persistence data: Could not create an instance of type System.Text.Encoding. Type is an interface or abstract class and cannot be instantiated. Path 'Encoding.BodyName', line 8, position 15. Newtonsoft.Json.JsonSerializationException Newtonsoft.Json.JsonSerializationException: Could not create an instance of type System.Text.Encoding. Type is an interface or abstract class and cannot be instantiated. Path 'Encoding.BodyName', line 8, position 15.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at LogExpert.Core.Classes.Persister.Persister.LoadInternal(String fileName) in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Persister\Persister.cs:line 270
   at LogExpert.Core.Classes.Persister.Persister.LoadPersistenceDataOptionsOnly(String logFileName, Preferences preferences) in D:\a\LogExpert\LogExpert\src\LogExpert.Core\Classes\Persister\Persister.cs:line 88
   at LogExpert.UI.Controls.LogWindow.LogWindow.LoadPersistenceOptions() in D:\a\LogExpert\LogExpert\src\LogExpert.UI\Controls\LogWindow\LogWindow.cs:line 2260

[Hirogen]

interesting, this did not happen while I was testing with Bookmarks and Encoding

I'll have a look tonight