Remove psl-monad submodule

Andy Jordan (@andschwa)

Problem

Having product code in the submodule of the main project makes the simplest fix a commit dance.
The simple workflow is highly desirable to speed-up on-boarding.

Having psl-monad submodule makes sd<->git integration easier.
Meanwhile, it's possible to have a reasonable level of separationg and move psl-monad from submodule to a folder in the super-project.

Proposal

Move psl-monad\monad to root level monad folder.
Move psl-monad\wmi to root level wmi folder.

The reason behind it: make git cherry-pick from gitsd admin repo match one-to-one.
This way we keep sd -> git story pretty clear. Question: can we keep simple cherry-pick workflow, but teach it to map root folder into src\monad?

2 branches would be still around in PowerShell/PowerShell: master and source-depot.
It's important to save git history.

Execution plan

We can achieve the desired result with a simple merge of two repos:

From PowerShell

git remote add monad github.com/PowerShell/psl-monad
git merge monad/master
git checkout -b source-depot
git merge monad/source-depot

cc Andy Jordan (@andschwa) Zach Folwick (@zach-folwick) Steve Lee (@SteveL-MSFT) James Truher (@JamesWTruher)

UPD:

Inventarisation

Tree below shows the folder structure.
Need dig into some of the subtrees.
Number in brackets is the number of files.

Legend

Mark	Meaning
❌	no need to get anything from this folder
♻️	need a subset from this folder
✅	need everything from this folder
❓	investigate

Tree

D:\dev\PowerShell\src\monad
├──monad (26803)
│ ├──core_powershell (3) ❌
│ ├──incubation (392) ❌ Jason Shirk (@lzybkr): but we should move a couple xsd files to miscfiles
│ ├──InstallerSignJob (3) ❌
│ ├──keys (9) ❌
│ ├──loc (36) ❓
│ ├──ManagedSignJob (3) ❌
│ ├──miscfiles (124) ♻️
│ ├──nttargets (661) ❌
│ ├──prototypes (2) ❌
│ ├──Providers (83) ❓ Jason Shirk (@lzybkr): maybe not needed
│ ├──ScriptSignJob (13) ❌
│ ├──Setup (972) ❓
│ │
│ ├──src (5403)
│ │ ├──Activities (227) ✅ Jason Shirk (@lzybkr): we need it for workflow
│ │ ├──build (3) ❌
│ │ ├──cimSupport (41) ✅
│ │ ├──cmdletDesigner (440) ❌
│ │ ├──commands (504) ✅
│ │ ├──CoreCLR (5) ✅
│ │ ├──DotNetCode (31) ❓
│ │ ├──DscSupport (1) ✅
│ │ ├──engine (523) ✅
│ │ ├──graphicalhost (210) ❌
│ │ ├──help (429) ♻️
│ │ ├──host (1136) ♻️
│ │ ├──LocalAccounts (30) ❓
│ │ ├──logging (13) ✅
│ │ ├──m3p (168) ❓
│ │ ├──minishell (20) ♻️
│ │ ├──namespaces (38) ❓
│ │ ├──oneget (535) ❓
│ │ ├──OSS (89) ❌
│ │ ├──PSGetAddOn (2) ❓
│ │ ├──PSReadline (37) ❌
│ │ ├──samples (579) ❓
│ │ ├──ScheduledJob (31) ❓
│ │ ├──ScriptAnalyzer (174) ❌
│ │ ├──sdk (12) ❓ :
│ │ ├──security (28) ✅
│ │ ├──singleshell (28) ✅
│ │ ├──tools (1) ❌
│ │ └──utils (66) ✅
│ │
│ ├──tests (15771)
│ │ ├──ci (937) # binplaced
│ │ │ ├──Appx (3) ❌
│ │ │ ├──Lite3UnitTest (19) ❌
│ │ │ ├──Lite_Template (19) ❌
│ │ │ ├──MP (1) ❌
│ │ │ ├──OnlineHelp (17) ❓
│ │ │ ├──PowerShell (470) ♻️
│ │ │ ├──PowerShellMultiMachine (1) ❌
│ │ │ ├──PowerShellPerformance (7) ❓
│ │ │ ├──PowerShellStress (1) ❌
│ │ │ ├──PSGet (39) ❓
│ │ │ ├──PSOne (2) ❓
│ │ │ ├──PSWF (56) ❓
│ │ │ └──PSWS (300) ❓
│ │ │
│ │ ├──citargets (955) ♻️ # compiled
│ │ ├──mae (3674) ♻️
│ │ ├──monad (10052) ♻️
│ │ ├──MultiMachine (148) ♻️
│ │ └──performance (4) ❓
│ │
│ ├──UnmanagedSignJob (3) ❌
│ ├──V2 (263) ❌
│ └──WTR (3056) ❌
└──wmi (105) ✅

Question: can we keep simple cherry-pick workflow, but teach it to map root folder into src\monad?

This is actually where Git subtree shines. I encourage you try it out and read up on subtree merging to see if it's what you all would like.

Essentially, git subtree let's you map one repository into another as a simple folder. You maintain one repo. You can git subtree split -P src/monad to obtain patches in that folder as if it were the root; yet the patches themselves can originally be committed to the "superproject" (in quotes because it is now one repo). You still should create separate commits for changes to the subtree and other trees, but this is the case regardless.

This also merges the history together, which is a lot more difficult, as we need to get an all-clear from legal to do this. As of now, the superproject only has code that the PSL team has written since last summer, and so is not subject to any potential PowerShell "patents." If we merge the history, this will no longer be true.

We can achieve the desired result with a simple merge of two repos:

I wouldn't encourage this, as you lose the meta-information that using the git subtree script gives you. However, the end result is similar; we would just have to perform more complex subtree (not the script) merges later.

Steve Lee (@SteveL-MSFT)

Thank you, I will play around with subtrees, they may solve the problem for us.

Steve Lee (@SteveL-MSFT) for LCA stuff, is Andy's concern about merge a problem?

I've created an example repository with every submodule replaced with subtree imports. If you want to play with it, it is PowerShell/PowerShell-Subtree.

The Pester, GoogleTest, and windows-build were squashed upon import. The OMI and provider submodules haven't been imported (unneeded for testing so far).

Doing this for real would require more planning. The psl-monad code would need to be vetted and squashed upon import. OMI and the provider should probably not be included. I would hope that windows-build can be eliminated.

Thanks for putting it together.
I played a little bit: tried to update src\monad subtree.

What's the good way to put changes from subtree into the original repo (psl-monad)?

The workflow that I have in mind:

We replace submodule by subtree (let's do psl-monad first, the rest is not mandatory).
PS team switch to GitHub
We still want to periodically update psl-monad from source-depot and integrate this changes to subtree (with subtree pull)
Do we even need subtree -> psl-monad way? There is not much difference, when you trying to integrate changes back to sd.

Also rebase after subtree seems like a PITA http://stackoverflow.com/questions/12858199/how-to-rebase-after-git-subtree-add

Nah, it's really not too bad. You probably won't ever want to rebase the add merge commit itself (though you can with git rebase -i -p, I've used it to undo them no problem). We likely won't have any subtree commits that need rebasing (which is the problematic part), since we'll squash the initial import. Commits that are descendents of the subtree merge commit can be rebased no problem; they're regular "superproject" repository commits.

Jason Shirk (@lzybkr)

Jason Shirk (@lzybkr) suggested something better then go thru the inventarisation:

pull code that we use for building into the appropriate assemblies subfolders
have a mapping script to do the integration (i.e. merge new changes from source-depot branch of psl-monad)
pull more code if we need it in future

If we do this, I think we skip the subtree step.

Yes, that's a viable option. I like the immediate move to an organized tree.

Steve Lee (@SteveL-MSFT)

I prepared a preview under

https://github.com/PowerShell/PowerShell-Subtree/tree/new-master
https://github.com/PowerShell/PowerShell-Subtree/tree/new-source-depot
https://github.com/PowerShell/PowerShell-Subtree/tree/new-server2016

Please, take a look
Steve Lee (@SteveL-MSFT) John W. Fawcett (@FawcettJohnW) Andy Jordan (@andschwa) Jason Shirk (@lzybkr)

Update commiting.md : src/monad submodule is gone

Remove psl-monad submodule

Update commiting.md : src/monad submodule is gone

Navigation Menu

Repository navigation

Remove psl-monad submodule #656

Problem

Proposal

Execution plan

Inventarisation

Legend

Tree

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Remove psl-monad submodule #656

Description

Problem

Proposal

Execution plan

Inventarisation

Legend

Tree

Activity

andyleejordan commented on Mar 12, 2016

vors commented on Mar 12, 2016

andyleejordan commented on Mar 16, 2016

vors commented on Mar 17, 2016

vors commented on Mar 17, 2016

andyleejordan commented on Mar 17, 2016

vors commented on Mar 18, 2016

andyleejordan commented on Mar 18, 2016

vors commented on Mar 23, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions