Adding caching to Azure Pipelines

Alex Mullans

For a long while, Azure Pipelines users have been asking to improve performance on the hosted build agents by adding caching for common scenarios like package restore. The issue came up in a recent popular Hacker News item, so we wanted to share an update.

Pipeline Caching is starting development now. You can see the design in this PR. Near the end of March, we’ll be releasing restore and save cache tasks that allow you to cache any file or set of files to a cache key of your choice. If you’re building some dependencies over and over or restoring a lot of packages, these tasks can add immediate time savings to your pipelines.

Continuing in Q2, we’ll improve those tasks after we see how you use them in your pipelines. We’ll also be enabling caching by default in the most popular ecosystems (think NuGet, pip, npm) as we learn more about the performance of those tools.

Please share your feedback on the PR; we’ll be reading and evolving the design as we go. Also feel free to get in touch with me and Mitch Denny, the feature owner for Pipeline Caching. We’re eager to hear about your ideas for using caching in your pipelines.

9 comments

Discussion is closed. Login to edit/delete existing comments.

  • Antonio De Almeida 0

    I am absolutely excited for this! Thank you Microsoft and the team for really just listening to the feedback from the community and making us feel valued!

  • sudhir kumar 0

    Hi @Alex, when will be the Pipeline Caching feature available ?

  • Joseph Priolo 0

    Earnestly waiting on caching feature! Any news on ETA? Thanks so much!
    JP

  • Trevor Blanchard 0

    Will this be available for On-Prem setups?

    • Alex MullansMicrosoft employee 0

      We don’t have plans to bring this to Azure DevOps Server at the moment, since on-premises you have persistent agents that can ‘cache’ by virtue of not being brand new machines for each run.

  • Jos Krause 0

    @Alex is there any update or updated roadmap for this feature? 2 minute builds for something that builds locally in 8 secs is a bit…attrocious 🙂

  • Caspar Bang Jespersen 1

    Hi @Alex. Will caching also be available for Docker agents? A lot of time in building containers (significantly on Windows Containers) is time spent downloading and extracting base layers, that could easily be cached.

  • Paul Vrugt (Infoland) 1

    Is there any update on this? The PR is merged, but caching still seems to be in beta?

  • Weston Weems 0

    I think the fact this functionality is starting to make it into azure devops, but in testing early versions of this I was dissappointed that we effectively traded some time up front to do a npm restore, to time at the end of the build to copy everything that was in node_modules to cache.

    I think it’d be smarter to create a cache folder for the build/folder path and symlink it into the build folder at the beginning.. .that way when build is occurring and things are being installed, its just putting it into a folder external to the folder that gets cleared out on build, but would require no copying process at the end of the said build

Feedback usabilla icon