Skip to content

Decouple the bundled modules to seperate Repo's #1979

Description

@kilasuit

Now that we have PowerShell Open Sourced we should really move to decouple all of the modules that are shipped "inbox" into their own repos so that updates to these can be make separately to the core engine and can also be pulled from the PowerShell Gallery

However this brings a number of additional challenges that warrants a detailed discussion and a plan of action as not to break the current build process.

Modules that should be decoupled and pulled in as submodules

  • - PowerShellGet Priority
  • - PackageManagement (this may already be out there in the OneGet Repo)
  • - Microsoft.PowerShell.ODataUtils
  • - PSReadline (already decoupled just needs to become a submodule)
  • - PowerShell.Archive (as above)
  • - Pester (as above but submodule to point to Pester not current submodule)
  • - CimCmdlets
  • - Microsoft.PowerShell.Diagnostics
  • - Microsoft.PowerShell.LocalAccounts
  • - Microsoft.PowerShell.Management
  • - Microsoft.PowerShell.Security
  • - Microsoft.WSMan.Management
  • - PowerShell.Utility
  • - PSDiagnostics
  • - PSScheduledJob
  • - PSWorkflow
  • - PSWorkflowUtility

This is especially more important as in the differing subfolders in https://github.com/PowerShell/PowerShell/tree/master/src/Modules there are a number of code duplications due to the modules residing in multiple folders which is rather unnecessary and adds unneeded complexity for management of this repository going forward.

Activity

  1. lzybkr commented on Aug 20, 2016

    @lzybkr
    Contributor

    We have definite plans to do this for some of the modules like PackageManagement, PowerShellGet, Pester, and PSReadline.

    Utility and Management might not be worth the hassle because of implicit or explicit circular dependencies.

  2. kilasuit commented on Aug 20, 2016

    @kilasuit
    CollaboratorAuthor

    I thought it may have been "on the plans" but couldn't see an issue for it so felt it best to raise one for visibility

  3. Carringguns commented on Aug 20, 2016

    @Carringguns

    Thanks im not free and clear yet. Little cleanup still. But want go to the source and offer you the data. and its running state im keeping in a loop.

  4. Carringguns commented on Aug 20, 2016

    @Carringguns

    Its design to eexplore when mission complete. Love this one. Better than most ive seen

  5. self-assigned this
    on Sep 2, 2016
  6. vors commented on Sep 2, 2016

    @vors
    Collaborator

    The current proposition is the following:
    Make sure that upstream version is suitable as a drop-in replacement.
    If not, massage it to the point, when it is.

    Replace code by a git submodule.

    cc Andy Jordan (@andschwa) Jason Shirk (@lzybkr)
    cc Dave Wyatt (@dlwyatt) (for Pester)

  7. andyleejordan commented on Sep 2, 2016

    @andyleejordan
    Member

    Oh yeah. I can help save the history for PSReadLine (and fix incoming changes on the upstream repo).

  8. vors commented on Sep 2, 2016

    @vors
    Collaborator

    Andy Jordan (@andschwa) sweet, then I will assume you will take care about PSReadLine

  9. andyleejordan commented on Sep 2, 2016

    @andyleejordan
    Member

    Okay, sounds good.

  10. SteveL-MSFT commented on Sep 7, 2016

    @SteveL-MSFT
    Member

    ODataUtils should be split out as well

  11. dragonwolf83 commented on Sep 8, 2016

    @dragonwolf83

    It is probably a bit late to change it, but Is there a better way to organize all the repos under https://github.com/PowerShell to handle splitting the code base more?

    It makes perfect since to break out the modules, but it is a bit messy right now with all of the individual DSC repos + Documentation + RFC. There are 6 pages of repos already. I can imagine a dozen more springing up to clean up the code base and dozens more for new things.

    Something to keep in mind for those of us watching the Repos! A landing page dedicated to source would probably be enough of it is self-maintaining (or just maintained).

  12. modified the milestone: on Sep 8, 2016
  13. RamblingCookieMonster commented on Sep 10, 2016

    @RamblingCookieMonster

    Troy Campbell (@dragonwolf83) - Agreed, probably worth opening an issue for this, although not sure where you would do that, given that it's an org-level issue : )

    Could go as simple as a readme that points to the various repos, with a little organization... or even something like GitHub Pages. Ideally, the community could submit PRs and issues, so IMHO a Microsoft hosted site (unless fed by CI/CD from GitHub) would be less appropriate.

    Cheers!

  14. vors commented on Sep 10, 2016

    @vors
    Collaborator

    A landing page dedicated to source would probably be enough of it is self-maintaining (or just maintained).

    I don't think it's worth the effort to be honest.

    This is a generically applicable statement for any big github organization.
    Take https://github.com/Microsoft or https://github.com/google as examples.

    How people are finding repos in these cases?
    Trying to list all the repos is not practical. There are projects that only a handful of people are interested in. With modern search engines (and also GitHub search) it's easy to find a particular project or explore interesting projects.

    Also https://github.com/dotnet has a github-pages hosted site for the org, but there is no list of all projects there.

  15. 10 remaining items

  16. KirkMunro commented on Oct 7, 2016

    @KirkMunro
    Contributor

    I generally agree with sergei (@vors) about which modules should stay in the PowerShell/PowerShell repo.

    My line in the sand to decide what stays vs what is submoduled would be whether or not module updates are already or may become later available to downlevel versions of PowerShell. If module updates do apply to downlevel versions of PowerShell, then use a submodule so that the module can be maintained independently as one that works for multiple versions of PowerShell, allowing for updates to be leveraged by all supported versions. If not, then keep it in the PowerShell/PowerShell repo because it is bound to that project.

    PowerShellGet already has downlevel version support (although I don't think the way downlevel support is done in that module right now is the way to go, so there's some work to do there). PSScriptAnalyzer has downlevel version support. Microsoft.PowerShell.Archive is outside of PowerShell/PowerShell already, and currently requires version 5+, but it will have downlevel version support shortly once I submit my pull request. Other core modules listed above in sergei (@vors) post should stay in the main project IMHO.

  17. lzybkr commented on Oct 7, 2016

    @lzybkr
    Contributor

    I see some value in breaking out all modules - e.g. the recent improvement to Join-Path in Microsoft.PowerShell.Management and Get-Credential in Microsoft.PowerShell.Security are both useful and shouldn't require updating to a newer version of PowerShell.

    That said, decoupling a couple of the core modules has some risk, so I'd consider that lower priority.

  18. removed their assignment
    on Nov 16, 2016
  19. vors commented on Nov 20, 2016

    @vors
    Collaborator

    PSGet and OneGet are separated in #2711

  20. removed this from the milestone on May 24, 2017
  21. added
    Issue-Code Cleanupthe issue is for cleaning up the code with no impact on functionality
    and removed
    Issue-Discussionthe issue may not have a clear classification yet. The issue may generate an RFC or may be reclassif
    on Jun 1, 2017
  22. kilasuit commented on Mar 5, 2018

    @kilasuit
    CollaboratorAuthor

    Joey Aiello (@joeyaiello) - please have a look over this issue from 2016.

    Is there any chance could we get some update/traction on this to make things easier/better going forward PowerShell 6.1 onwards

  23. iSazonov commented on Mar 6, 2018

    @iSazonov
    Collaborator

    The only thing left to complete is PSReadline.
    I agree with sergei (@vors) and don't see any benefit in moving out the rest of the modules. No one is actively developing these modules now. What prevents? Does someone have a big additions to (ex.) Odatautil in their repositories? Why can not commit here?

    I would have seen more benefits in moving very popular cmdlets to this repository after they became stable.

  24. kilasuit commented on Jun 14, 2018

    @kilasuit
    CollaboratorAuthor

    Bumping this for a review as PSReadline was recently decoupled and there is still a checklist (checked off PSReadline just now) that needs to be confirmed/denied if/when there are plans to do so

  25. iSazonov commented on Jun 14, 2018

    @iSazonov
    Collaborator

    ODataUtils and Pester is in separate repos.

  26. kilasuit commented on Jun 14, 2018

    @kilasuit
    CollaboratorAuthor

    updated the checkboxes - thanks Ilya (@iSazonov)

  27. iSazonov commented on Nov 21, 2018

    @iSazonov
    Collaborator

    I think we can close the issue. All works is done. The remaining modules will still be in this repository as discussed.
    /cc Steve Lee (@SteveL-MSFT)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Issue-Code Cleanupthe issue is for cleaning up the code with no impact on functionalityResolution-FixedThe issue is fixed.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions