Skip to content

DTensor from text file#54

Merged
alphaville merged 7 commits intomainfrom
feature/53-serialisation-simple
Nov 26, 2024
Merged

DTensor from text file#54
alphaville merged 7 commits intomainfrom
feature/53-serialisation-simple

Conversation

@alphaville
Copy link
Copy Markdown
Member

@alphaville alphaville commented Nov 26, 2024

Main Changes

  • Simple text-based serialisation of tensor data
  • Example in main.cu where we read data from file

Example

Suppose we have the file my.dtensor with the following content:

3
2
1
5.6
2
-1.90
7.6
20.5
-100.123

To parse it, do:

auto z = DTensor<double>::parseFromTextFile("my.dtensor");

This will create the tensor:

Tensor [3 x 2 x 1]:
>> layer: 0
       5.6,        7.6, 
         2,       20.5, 
      -1.9,   -100.123, 

TODOs

  • Update README file
  • Write unit tests
  • Update CHANGELOG

Associated Issues

- Simple text-based serialisation of tensor data
- example in main.cu where we read data from file
Copy link
Copy Markdown
Collaborator

@ruairimoran ruairimoran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Should we add row/column major option, or only accept one? I think python will store arrays in text files in row major

@ruairimoran
Copy link
Copy Markdown
Collaborator

python saves as row major

image

@alphaville alphaville marked this pull request as ready for review November 26, 2024 22:39
@alphaville alphaville merged commit 4d88cd8 into main Nov 26, 2024
@alphaville alphaville deleted the feature/53-serialisation-simple branch December 3, 2024 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Serialisation and deserialisation

2 participants