Repository navigation
Repository REST API #118
Description
I'd like to be able to have a repository accessible via REST APIs rather than a signed MSIX package.
The API will need a full design. This will also have an impact on client behavior. The MSIX index package can become a backup mechanism if the APIs are offline, or the client is unable to connect.
Notes:
The client needs to be able to get the list of packages to enable "search"
The client needs to be able to get the manifest so it can "install"
This implies that the client will build a local cache of packages after retrieving the list so they can perform a local search.
In the future we may create a web based search API (TBD).
The contents of the Microsoft Community Package manager should be available via this API once it's built.
Activity
svmastersamurai commentedon May 20, 2020
This aspect is super interesting to those of us in orgs that build and deploy internally built software.
As someone who has to manage chocolatey infrastructure: please please please keep this design simple.
As an operator I would love to be able to run a tool over "packages" to generate a static website. Assuming the API is of trivial complexity, the server might be about 20 or 30 lines of code in Go or I could use something like apache or nginx to serve the content. I could more easily leverage a CI/CD system to handle rebuilding the repo metadata when there is a change.
For example we looked into sleet but chocolatey speaks an older dialect of nuget and that ended up being non-viable.
I think other package managers like RPM and apt are great examples to draw inspiration from.
The tools for repo management should ideally be cross-platform so we could build or manage changes to repo metadata on either POSIX-based OSes or Windows.
denelon commentedon May 21, 2020
Dan Sedlacek (@svmastersamurai) That's great feedback. I like the idea of running the tool over the packages. Would you think in terms of UNC style path's to the binaries or would HTTP still be OK?
I'm thinking about a concept where you might have the installers in a structure that looks like the manifests directory in the repository here, and then having the manifests able to co-locate in the same structure. If the installers are MSIX, we can pull nearly all of the meta-data to generate the manifest based on what is found.
svmastersamurai commentedon May 21, 2020
Demitrius Nelon (@denelon) I think to generate a static website that makes perfect sense. This assumes that there will be adequate saturation of MSIX usage of course! There are a lot of legacy installers out there, so would this be able to also handle the usual suspects we know about?
Our team manages multiple operating systems so we tend to prefer HTTP since it is usable cross platform. We use the same webservers to serve installers for Windows that we do for Munki packages on MacOS and RPMs for our fedora fleet.
One thing we do for redundancy is have a few locations to retrieve the installers from. It appears the package spec supports this, so from a tooling perspective if I have 5 mirrors with different hostnames it would be nice to be able to have the tool aware of the mirrors (in a simplistic sense, I can just give it a list in a file for example) and be able to append those sources to the specs if they don't exist. From the
winget-cli's perspective it could cycle through the mirrors until it finds one that is accessible. This would probably lead to templating within the package spec, which I admittedly haven't dived too much into this project to know if that's supported or if there was a plan for that already 😄.majkinetor commentedon Jun 2, 2020
This deserves a separate ticket IMO and has nothing to do with REST interface to repository. Such tool would indeed be great for multiple reasons, one of which is that it allows anybody to keep their public list of packages and host it anywhere (i.e. github pages and friends).
Such tool might not be practical in producing all desired functionality of REST backend such as versioning and dependencies although it could go long way into that direction.
denelon commentedon Mar 2, 2021
We are actively working on the REST API and our initial implementation in the client. The schemas for validating the new YAML syntax have been checked in, but may very well churn a bit. https://github.com/microsoft/winget-cli/tree/master/schemas/JSON/manifests/v1.0.0
The REST API will have it's own object structure, but is expected to be essentially conforming to the same meta-data.
10 remaining items
denelon commentedon May 25, 2021
It's coming soon [TM] with a reference implementation.
ghost1372 commentedon May 25, 2021
Thanks, where can we find the documentation when it was published?
denelon commentedon May 25, 2021
It's another GitHub repository. I'll have copious documentation updates to make as soon as we flip the repository to public, Think in terms of days not weeks. It does include Swagger documentation.
denelon commentedon May 26, 2021
Mahdi Hosseini (@ghost1372) https://github.com/microsoft/winget-cli-restsource is where the reference implementation is.
ghost1372 commentedon May 27, 2021
Thank you
mherrmann commentedon Jun 28, 2021
Hi Demitrius Nelon (@denelon), may I ask out of curiosity what the motivation for the REST API is vs. a UNC source of offline manifest files (#160)? What I don't understand is that it seems to me that a UNC source will always be easier to create and deploy than a REST API. What does the REST API give us in return, in terms of use cases that can be implemented?
Thank you!
Michael
denelon commentedon Jun 28, 2021
Michael Herrmann (@mherrmann) we had more asks for a REST API during early feedback cycles with Microsoft MVPs and ISVs than a UNC implementation. The two primary use cases at the time were hosting repositories for software distribution by ISVs (to the public), and enterprise customers hosting internal repositories for a distributed workforce.
mherrmann commentedon Jun 28, 2021
Demitrius Nelon (@denelon) I see, thank you very much for the prompt response. I am working in a tangential space and would love to get a better feel for the industry opinion. Is there any chance the feedback cycles you mentioned are public or documented somewhere? If not, can you recommend people (eg. Microsoft MVPs) I might reach out to to find out more?
Thank you again!
Best,
Michael
denelon commentedon Jun 28, 2021
Mathias (@mhermann) those early discussions were under NDA before the preview launched. I'd be willing to reach out via Twitter if you have some specific questions to get some feedback.
mherrmann commentedon Jun 28, 2021
Demitrius Nelon (@denelon) I understand, and thank you. I'd like to understand why people asked for REST APIs, when static files would be easier to host. I would love to understand which features people want that require a dynamic approach such as REST and cannot be implemented with static files.
My twitter is @m_herrmann if you want to include it. Thank you 🙏
matsmcp commentedon Oct 11, 2021
I also would like to upvote a more static approach on a Webserver.
The scenario is both for the MSP use case where multiple Customers would use the MSP:s server for MSP created packages and for larger organisations with multiple domains where a fileshare is unpractical. Having a webserver provide a static site with the packages and a contentfile (ie packages.tgz in a Unix world) would be very simple to implement and also be completly plattform independant.
This also needs a Winget client that is usable over SSH/winrm to be usefull
denelon commentedon Oct 11, 2021
Michael Herrmann (@mherrmann) I think I must have missed an e-mail notification from GitHub on this. A couple of other features in the works related to Delivery Optimization are likely to have an impact on the API and subsequently a UNC driven approach. I believe it's still possible to generate a static site with the appropriate values to support the client search, install, upgrade, and list behaviors. We just went down the dynamic path due to the scope/scale of duplicating the Windows Package Manager Community App Repository. We have some active development in the area currently, and we're also looking at possibly refactoring the search interface.
mherrmann commentedon Oct 11, 2021
Demitrius Nelon (@denelon) I see. Thank you for clarifying :-)
Squashed 'src/Valijson/valijson/' changes from 7a52fc88..2dfc7499
Squashed 'src/Valijson/valijson/' changes from 7a52fc88..2dfc7499