Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interface for DPPY interoperability #788

Merged
merged 30 commits into from
Feb 9, 2023
Merged

Interface for DPPY interoperability #788

Changes from 2 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
f297ea2
added partition interface to DNDarray
coquelin77 May 17, 2021
9191599
added 'locals' key to partition interface
coquelin77 May 19, 2021
89fda67
renamed locals to lcls to avoid global name
coquelin77 May 19, 2021
8806b7c
corrected format of locals
coquelin77 May 19, 2021
51368b7
renamed dunder class attr of DNDarray to __partitioned__
coquelin77 May 19, 2021
81e47c8
corrected split=0 case, corrected DNDarray property to be 'partitioned'
coquelin77 May 19, 2021
857f585
DNDarray.__partitioned__ -> __partitions_dict__, DNDarray.partitioned…
coquelin77 May 19, 2021
dfb3231
Merge branch 'master' into features/772-partition-interface
coquelin77 Jun 8, 2021
93d1e5e
Merge branch 'master' into features/772-partition-interface
coquelin77 Jun 22, 2021
d69fdd6
added tests for partitioned attribute
coquelin77 Jun 22, 2021
0afc772
minor changes to test cases to check that things after the resplit ar…
coquelin77 Jun 22, 2021
750cc2b
split=None tests
coquelin77 Jun 22, 2021
4307306
Merge branch 'master' into features/772-partition-interface
coquelin77 Jul 1, 2021
992c385
changelog update
coquelin77 Jul 13, 2021
ac3928e
Merge branch 'master' into features/772-partition-interface
coquelin77 Jul 13, 2021
012318d
added 'get' attributed to __partitioned__ to get a tile from a DNDarray
coquelin77 Jul 13, 2021
26203c0
reduced level of abstraction for __partitioned__['get']
coquelin77 Jul 13, 2021
d44109c
Merge branch 'master' into features/772-partition-interface
coquelin77 Aug 23, 2021
0d22fbd
Merge branch 'master' into features/772-partition-interface
coquelin77 Aug 26, 2021
e434b7c
Merge branch 'master' into features/772-partition-interface
coquelin77 Aug 26, 2021
29d0385
adding from_partitioned; aligning __partitioned__ with current spec
fschlimb Aug 26, 2021
6f0a0ac
Merge branch 'master' into features/772-partition-interface
coquelin77 Sep 17, 2021
4983751
Merge pull request #860 from fschlimb/features/772-from_partition
coquelin77 Sep 17, 2021
a0c8ab4
updating from_partitioned function
coquelin77 Sep 23, 2021
7c70eae
added nonzero split support to from partition dictionary, added tests…
coquelin77 Sep 23, 2021
ab71539
Merge branch 'master' into features/772-partition-interface
coquelin77 Jan 4, 2022
c1e6bcf
Merge branch 'master' into features/772-partition-interface
coquelin77 Jan 17, 2022
a505e99
Merge branch 'main' into features/772-partition-interface
ClaudiaComito Jun 1, 2022
9d10b7b
Merge branch 'main' into features/772-partition-interface
ClaudiaComito Feb 8, 2023
fd7ec83
Ensure is None when virtually resplitting to None on 1 process
ClaudiaComito Feb 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion heat/core/dndarray.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -664,7 +664,6 @@ def create_partition_interface(self):
-------
dictionary containing the partition interface as shown above.
"""
# sp =
lshape_map = self.create_lshape_map()
start_idx_map = torch.zeros_like(lshape_map)

Expand Down Expand Up @@ -705,6 +704,12 @@ def create_partition_interface(self):
"partitions": partitions,
"locals": [tuple(lcls)],
}

def _partition_getter(key):
return partition_dict["partitions"][key]["data"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to accept whatever is located in partition_dict["partitions"][key]["data"].
A user would use it like this pdict['get'](pdict['partitions"][key]["data"].

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here, it's really lambda x: x

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah! i was thinking one level of abstraction more than that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now this is what the snippet would look like:

    x = ht.arange(8* 3* 2).reshape((8, 3, 2)).resplit(0)
    print(x.__partitioned__['get'](x.__partitioned__['partitions'][(0,0,0)]['data']))


partition_dict["get"] = _partition_getter

return partition_dict

def __float__(self) -> DNDarray:
Expand Down