Skip to content

Implementations for methods for machine_views and associated modules #1429

Merged
lockshaw merged 9 commits intoflexflow:repo-refactorfrom
Marsella8:machine-related
Jul 19, 2024
Merged

Implementations for methods for machine_views and associated modules #1429
lockshaw merged 9 commits intoflexflow:repo-refactorfrom
Marsella8:machine-related

Conversation

@Marsella8
Copy link
Contributor

@Marsella8 Marsella8 commented Jul 2, 2024

Description of changes:

Implemented NOT_IMPLEMENTED() functions within machine_view.cc, strided_rectangle.cc, strided_rectangle_sides.cc, device_id.cc.

Related Issues:

Linked Issues:

  • Issue #

Issues closed by this PR:

  • Closes #

This change is Reviewable

@codecov
Copy link

codecov bot commented Jul 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 40.05%. Comparing base (d30c8f0) to head (61ae1c6).
Report is 6 commits behind head on repo-refactor.

Additional details and impacted files
@@                Coverage Diff                @@
##           repo-refactor    #1429      +/-   ##
=================================================
+ Coverage          38.78%   40.05%   +1.26%     
=================================================
  Files                256      258       +2     
  Lines               8973     9096     +123     
  Branches             331      334       +3     
=================================================
+ Hits                3480     3643     +163     
+ Misses              5493     5453      -40     
Flag Coverage Δ
unittests 40.05% <100.00%> (+1.26%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
lib/pcg/src/pcg/machine_view.cc 92.85% <100.00%> (+92.85%) ⬆️
lib/pcg/src/pcg/strided_rectangle_side.cc 100.00% <100.00%> (+100.00%) ⬆️
lib/pcg/src/strided_rectangle.cc 100.00% <100.00%> (+100.00%) ⬆️
lib/pcg/test/src/test_machine_view.cc 100.00% <100.00%> (ø)
lib/pcg/test/src/test_strided_rectangle.cc 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

Pietro Max Marsella added 2 commits July 2, 2024 14:51
Copy link
Collaborator

@lockshaw lockshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 7 of 8 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 7 unresolved discussions (waiting on @Marsella8)


lib/pcg/src/strided_rectangle.cc line 24 at r2 (raw file):

num_points_t get_num_points(StridedRectangle const &rect) {
  int num = 1;
  for (StridedRectangleSide const &side : rect.sides) {

Replace with product from utils/containers.h?


lib/pcg/src/pcg/machine_view.cc line 79 at r2 (raw file):

  }
  assert(get_device_type(start) == DeviceType::GPU);
  return make_1d_machine_view(unwrap_gpu(start), num_points, stride);

Suggestion:

  } else {
    assert(get_device_type(start) == DeviceType::GPU);
    return make_1d_machine_view(unwrap_gpu(start), num_points, stride);
  }

lib/pcg/src/pcg/machine_view.cc line 108 at r2 (raw file):

  }
  assert(get_device_type(start) == DeviceType::GPU);
  return make_1d_machine_view(unwrap_gpu(start), interval_size, stride);

Suggestion:

  } else {
    assert(get_device_type(start) == DeviceType::GPU);
    return make_1d_machine_view(unwrap_gpu(start), interval_size, stride);
}

lib/pcg/test/src/test_machine_view.cc line 22 at r2 (raw file):

    device_id_t start_cpu{cpu_id_t{3}};

    MachineView gpu_mv{start_gpu, rect};

Split gpu and cpu processing into subcases--the "declare a whole bunch of stuff, then run a whole bunch of tests on it" makes testcases rather hard to read. It's a lot more readable if the declarations are closer to and associated with their uses


lib/pcg/test/src/test_machine_view.cc line 25 at r2 (raw file):

    MachineView cpu_mv{start_cpu, rect};

    CHECK(make_1d_machine_view(

Can you wrap these in subcases with names that clarify the differences between all these different checks? (if it's testing overloads, for example, it could be SUBCASE("<the function signature>"))


lib/pcg/test/src/test_strided_rectangle.cc line 12 at r2 (raw file):

    StridedRectangleSide result =
        strided_side_from_size_and_stride(side_size_t{7 * 5}, 5);
    CHECK(result == side);

Separate test case (or at least a separate subcase)

Code quote:

    StridedRectangleSide result =
        strided_side_from_size_and_stride(side_size_t{7 * 5}, 5);
    CHECK(result == side);

lib/pcg/test/src/test_strided_rectangle.cc line 24 at r2 (raw file):

    CHECK(get_num_dims(rect) == 3);
    CHECK(get_side_at_idx(rect, ff_dim_t{0}) == s0);

At the very least subcases for different functions

Copy link
Contributor Author

@Marsella8 Marsella8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 7 unresolved discussions (waiting on @lockshaw)


lib/pcg/src/strided_rectangle.cc line 24 at r2 (raw file):

Previously, lockshaw (Colin Unger) wrote…

Replace with product from utils/containers.h?

Changed (rect.sides is of type FFOrdered so direct application of transform is not possible, let me know if this is preferrable).


lib/pcg/src/pcg/machine_view.cc line 79 at r2 (raw file):

  }
  assert(get_device_type(start) == DeviceType::GPU);
  return make_1d_machine_view(unwrap_gpu(start), num_points, stride);

Done.


lib/pcg/src/pcg/machine_view.cc line 108 at r2 (raw file):

  }
  assert(get_device_type(start) == DeviceType::GPU);
  return make_1d_machine_view(unwrap_gpu(start), interval_size, stride);

Done.


lib/pcg/test/src/test_machine_view.cc line 22 at r2 (raw file):

Previously, lockshaw (Colin Unger) wrote…

Split gpu and cpu processing into subcases--the "declare a whole bunch of stuff, then run a whole bunch of tests on it" makes testcases rather hard to read. It's a lot more readable if the declarations are closer to and associated with their uses

Done.


lib/pcg/test/src/test_machine_view.cc line 25 at r2 (raw file):

Previously, lockshaw (Colin Unger) wrote…

Can you wrap these in subcases with names that clarify the differences between all these different checks? (if it's testing overloads, for example, it could be SUBCASE("<the function signature>"))

Done.


lib/pcg/test/src/test_strided_rectangle.cc line 12 at r2 (raw file):

Previously, lockshaw (Colin Unger) wrote…

Separate test case (or at least a separate subcase)

Done.


lib/pcg/test/src/test_strided_rectangle.cc line 24 at r2 (raw file):

Previously, lockshaw (Colin Unger) wrote…

At the very least subcases for different functions

Done.

Copy link
Collaborator

@lockshaw lockshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r3, all commit messages.
Reviewable status: all files reviewed, 9 unresolved discussions (waiting on @Marsella8)


lib/pcg/src/strided_rectangle.cc line 24 at r2 (raw file):

Previously, Marsella8 wrote…

Changed (rect.sides is of type FFOrdered so direct application of transform is not possible, let me know if this is preferrable).

It's defined here: https://github.com/flexflow/FlexFlow/blob/repo-refactor/lib/op-attrs/include/op-attrs/dim_ordered/transform.h


lib/pcg/test/src/test_machine_view.cc line 12 at r3 (raw file):

    gpu_id_t start(1);
    MachineView mv{device_id_t{start}, rect};
    CHECK(num_dims(mv) == 2);

Subcases?


lib/pcg/test/src/test_machine_view.cc line 24 at r3 (raw file):

    SUBCASE("make_1d_machine_view(gpu_id_t start, gpu_id_t stop, int stride)") {
      CHECK(make_1d_machine_view(
                start_gpu, device_id_t{gpu_id_t(1 + 7 * 5)}, 5) == gpu_mv);

Follow this pattern where possible for clarity, apply to the subcases below

Suggestion:

      MachineView result = make_1d_machine_view(start_gpu, device_id_t{gpu_id_t(1 + 7 * 5)}, 5);
      MachineView correct = gpu_mv;
      CHECK(result == correct);

lib/pcg/test/src/test_strided_rectangle.cc line 6 at r3 (raw file):

TEST_SUITE(FF_TEST_SUITE) {
  TEST_CASE("get_side_size(StridedRectangleSide s)") {

Suggestion:

  TEST_CASE("get_side_size(StridedRectangleSide)") {

lib/pcg/test/src/test_strided_rectangle.cc line 11 at r3 (raw file):

    CHECK(get_side_size(side) == side_size_t{7 * 5});
  }
  TEST_CASE("Make StridedRectangleSide from size and stride") {

If you're testing a function, just put the function name

Suggestion:

  TEST_CASE("strided_side_from_size_and_stride") {

lib/pcg/test/src/test_strided_rectangle.cc line 12 at r3 (raw file):

  }
  TEST_CASE("Make StridedRectangleSide from size and stride") {
    StridedRectangleSide side{num_points_t{10}, 3};

Suggestion:

    StridedRectangleSide correct{num_points_t{10}, 3};

lib/pcg/test/src/test_strided_rectangle.cc line 25 at r3 (raw file):

    StridedRectangle rect{{s0, s1, s2}};

    SUBCASE("number of dimensions") {

Suggestion:

    SUBCASE("get_num_dims") {

lib/pcg/test/src/test_strided_rectangle.cc line 28 at r3 (raw file):

      CHECK(get_num_dims(rect) == 3);
    }
    SUBCASE("number of points") {

Suggestion:

   SUBCASE("get_num_points") {

lib/pcg/test/src/test_strided_rectangle.cc line 31 at r3 (raw file):

      CHECK(get_num_points(rect) == num_points_t{7 * 8 * 10});
    }
    SUBCASE("each side is correctly stored") {

Suggestion:

  SUBCASE("get_side_at_idx")

Copy link
Contributor Author

@Marsella8 Marsella8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 5 of 8 files reviewed, 9 unresolved discussions (waiting on @lockshaw)


lib/pcg/src/strided_rectangle.cc line 24 at r2 (raw file):

Previously, lockshaw (Colin Unger) wrote…

It's defined here: https://github.com/flexflow/FlexFlow/blob/repo-refactor/lib/op-attrs/include/op-attrs/dim_ordered/transform.h

Fixed.


lib/pcg/test/src/test_machine_view.cc line 12 at r3 (raw file):

Previously, lockshaw (Colin Unger) wrote…

Subcases?

Done


lib/pcg/test/src/test_machine_view.cc line 24 at r3 (raw file):

Previously, lockshaw (Colin Unger) wrote…

Follow this pattern where possible for clarity, apply to the subcases below

Done.


lib/pcg/test/src/test_strided_rectangle.cc line 6 at r3 (raw file):

TEST_SUITE(FF_TEST_SUITE) {
  TEST_CASE("get_side_size(StridedRectangleSide s)") {

Done.


lib/pcg/test/src/test_strided_rectangle.cc line 11 at r3 (raw file):

Previously, lockshaw (Colin Unger) wrote…

If you're testing a function, just put the function name

Done.


lib/pcg/test/src/test_strided_rectangle.cc line 12 at r3 (raw file):

  }
  TEST_CASE("Make StridedRectangleSide from size and stride") {
    StridedRectangleSide side{num_points_t{10}, 3};

Done.


lib/pcg/test/src/test_strided_rectangle.cc line 25 at r3 (raw file):

    StridedRectangle rect{{s0, s1, s2}};

    SUBCASE("number of dimensions") {

Done.


lib/pcg/test/src/test_strided_rectangle.cc line 28 at r3 (raw file):

      CHECK(get_num_dims(rect) == 3);
    }
    SUBCASE("number of points") {

Done.


lib/pcg/test/src/test_strided_rectangle.cc line 31 at r3 (raw file):

      CHECK(get_num_points(rect) == num_points_t{7 * 8 * 10});
    }
    SUBCASE("each side is correctly stored") {

Done.

Copy link
Collaborator

@lockshaw lockshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r4, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @Marsella8)

@lockshaw lockshaw merged commit 989972a into flexflow:repo-refactor Jul 19, 2024
dylanllim pushed a commit to dylanllim/FlexFlow that referenced this pull request Jul 31, 2024
…lexflow#1429)

* initial commit for machine view adjacent modules

* Formatting

* Tests for new machine_view.cc functions

* formatting

* Minor Test correction

* formatting

* PR fixes

* PR Fixes

---------

Co-authored-by: Pietro Max Marsella <marsella@stanford.edu>
dylanllim pushed a commit that referenced this pull request Jan 28, 2025
…1429)

* initial commit for machine view adjacent modules

* Formatting

* Tests for new machine_view.cc functions

* formatting

* Minor Test correction

* formatting

* PR fixes

* PR Fixes

---------

Co-authored-by: Pietro Max Marsella <marsella@stanford.edu>
dylanllim pushed a commit to dylanllim/FlexFlow that referenced this pull request Jan 28, 2025
…lexflow#1429)

* initial commit for machine view adjacent modules

* Formatting

* Tests for new machine_view.cc functions

* formatting

* Minor Test correction

* formatting

* PR fixes

* PR Fixes

---------

Co-authored-by: Pietro Max Marsella <marsella@stanford.edu>
@victorli2002 victorli2002 mentioned this pull request Mar 12, 2025
@victorli2002 victorli2002 mentioned this pull request Apr 21, 2025
lockshaw added a commit that referenced this pull request May 2, 2025
* test_utils refactor, local_cpu_allocator

* test utils modification, cast, reverse, and replicate cpu kernels

* combine kernel

* combine kernels .h file

* Implementations for methods for machine_views and associated modules  (#1429)

* initial commit for machine view adjacent modules

* Formatting

* Tests for new machine_view.cc functions

* formatting

* Minor Test correction

* formatting

* PR fixes

* PR Fixes

---------

Co-authored-by: Pietro Max Marsella <marsella@stanford.edu>

* test utils logic cleanup, reverse cpu_kernel pedagogical implmentation, other minor fixes

* cpu_kernel's refactor, generic tensor accessor indexing

* accessor.h formatting

* mk_runtime_error formatting

* reverse_kernels include

* test_utils refactor and clarity

* formatting

* comment removal reverse_kernels

* Issue #1435, tests for managed stream and handle

* #1435 formatting

* #1409 issue, change datatype for linear kernels away from void *

* R & W accessor changes, minimize code bloat

* code formatting and refactor

* issue #1502 & issue #1540

* format check

* branch merge and test fixes

* build issues

* Add AWS linux AMI to runs-on for testing (#1589)

* Pin runs-on images (#1590)

* GPU CI Fix (Pin runs-on GPU image) (#1588)

* Debug

* Change to base DL AMI

* Print disk usage

* Run nvidia-smi

* Remove excess cuda installs in base ami

* Re-enable freeing space in GPU CI

* Try updating nix-develop version

* Check what happens if you just enter the non-nixGL environment

* Try switching AMIs

* Try to remove the module stuff

* Move to lockshaw/develop-action

* Try pointing at a fixed commit

* Update nix-develop action

* Update nix-develop action to use BASH_FUNC filtering

* Remove all the /usr/local/cuda entries

* Switch back to gpu-ci env

* Update the cuda arch

* Try out the new runs-on gpu image

* Move over to pinned runs-on image

* Remove a bunch more unnecessary stuff in image to get back disk space

* Try using an emphemeral store

* Try mounting

* Fix bug

* Try sudo

* Move nix into _work

* Rollback all unnecessary changes

* Re-enable waiting on cpu-ci

* Merge substitution-builder (#1575)

* Start on pcg builder

* Add tests and some implementation for pcg builder

* Add pcg tests, make dtgen constructors explicit to fix bug

* Add remainder of PCG tests

* Fix build issues in local-execution

* Format

* Address Reyna comments, add topological_order function for PCG

* Pre multidigraph refactor

* Removing visitable from sp code

* Add open dataflow graph, start to replace pcg dataflow graph

* Start refactoring substitutions

* Add utility functions to support pattern matching

* Pre-refactor inputs

* Fix proj url

* Get back to substitutions, now with unordered graph inputs

* Get substitutions building

* substitutions-tests now builds

* Fix bug in filter, pass some initial substitution tests

* Add tests for fmt::to_string, fix some substitutions bugs

* Pass initial unit tests for find_pattern_matches

* Start on unit tests for pcg pattern

* Pass initial test for find_pattern_matches

* Fix small build issue in tests

* Format

* Sync tests in CI with tests in proj

* Fix minor build errors in kernels and local-execution

* Format

* Remove outdated code

* More outdated code removal

* More cleanup, add test for sp decomposition

* Pull apart containers.h

* More sp testing and fixes

* Break up graph algorithms.h

* Pre- full SP algo commit

* Add initial implementation and tests for cbc decomposition and inverse line graph

* Pass test for get_inverse_line_graph

* Add new multidigraph

* Fix get_inverse_line_graph to return a MultiDiGraph instead of a DiGraph

* Add tests for parallel and series reduction finding

* Add really rough implementation of valdez sp decomposition

* Fix local-execution build

* Add implementations and tests for applying series/parallel reductions

* Format

* Clean up sp decomposition interface and tests

* Format

* Add comments for top-level substitutions functions, add proj doxygen support

* Start sketching out substitutions code

* Fix build errors

* Add ability to permute node ids

* Cleanup and start to test new substitutions code

* Add test case for evaluate_substitution_output

* Add naive isomorphism detection code

* Add graph inputs to open dataflow graph isomorphism

* Add input permutation to evaluate_substitution_output

* Fix permute_node_ids

* Add test for permute_input_ids

* Migrate over to mutable implementation of apply_substitution

* Add fast isomorphism checking and an initial implementation of full substitution logic

* Pass initial full substitutions test

* Cleanup old isomorphism checking code

* Fix post-merge bugs

* Fix broken pcg builder test

* Format

* Reorganize code and remove some outdated code pre-code-review

* Format

* Restarting work on this after working on export-model-arch

* Adding in some a simple function to get the currently available substritutions

* nonnegative_int additions, code cleanup, etc.

* A bunch more moving over to nonnegative_int

* Even more nonnegative_int updating

* Fix build

* Fix failing tests

* Format

* Format

---------

Co-authored-by: Colin Unger <lockshaw@lockshaw.net>
Co-authored-by: Victor Li <vli42@sapling2.stanford.edu>

* test_utils refactor, local_cpu_allocator

* test utils modification, cast, reverse, and replicate cpu kernels

* combine kernel

* test utils logic cleanup, reverse cpu_kernel pedagogical implmentation, other minor fixes

* cpu_kernel's refactor, generic tensor accessor indexing

* test_utils refactor and clarity

* R & W accessor changes, minimize code bloat

* issue #1502 & issue #1540

* branch merge and test fixes

* merge

* build after merge

* kernel issues

* managed stream / handle test case fix

* test_utils update, kernel/ops refactor

* Review fixes

* Update doctest includes in kernels

* More PR review

* Try using rhel package-based nixgl

* Format

* Update proj with test command fixes

* Attempt to fix gpu CI

* Use custom AMI in GPU CI

* Fix proj bug in cpu-ci

* Try including run id

* Temporarily allow gpu ci to run regardless for testing purposes

* Try using official ubuntu ami in gpu ci

* Try out new ami

* Change to use new flexflow-gpu-ci AMI

* Fix bugs in GPU tests and restore GPU CI gating

* Format

* Fix bug in accessor formatting test cases

* Bugfixes and updated proj

* Fix all cpu tests

* Format

* Add improved test failure output for replicate cpu vs gpu tests

* Continue debugging replicate cuda testcases

* Format

* Fix incorrect tensor size in replicate kernel tests

* Transpose replicate backward cpu kernel

* Try flipping output dimensions in replica cuda kernel test

* Update proj

---------

Co-authored-by: Marsella8 <45826022+Marsella8@users.noreply.github.com>
Co-authored-by: Pietro Max Marsella <marsella@stanford.edu>
Co-authored-by: Colin Unger <lockshaw@lockshaw.net>
Co-authored-by: Victor Li <32348970+victorli2002@users.noreply.github.com>
Co-authored-by: Victor Li <vli42@sapling2.stanford.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants