#monorepo

For those using monorepos or that only want Screwdriver jobs to run on changes in certain subdirectories or files, you can use the new source paths feature.

Source paths can be used to specify source code paths that will trigger a job upon modification.

Die Frage Monorepo oder Polyrepo wird meist als Organisations- und Tooling-Frage diskutiert. Aus Sicherheitssicht ist sie mindestens genauso relevant. Wer alle Projekte in einem Repository hält, trifft andere Risikoentscheidungen als jemand, der hundert kleine Repositories betreibt. Beide Modelle haben blinde Flecken, und die meisten Teams kennen ihre eigenen nicht. Zugriffsrechte: alles oder…

Sometimes you yearn for the simplicity of a monorepo. Maybe you just want everything in one place, maybe you just want to more easily share code.

Regardless of your motivation, git makes it easy to split and combine repos via git subtree. While it isn't part of git core, most distributions package git with subtree anyway.

After your monorepo is set up, you can start importing repos into it with git subtree add:

git subtree -P <prefix> add <repository> <remote-ref>

-P says which subdirectory in the monorepo the imported repo should live. This allows you to organize the code that you're importing.

is the thing you want to import, e.g. https://github.com/user/repo

is the branch or tag to import. Typically, this is main or master.

Compared to submodules, subtrees exist independently of their remote. If the remote of the subtree is deleted, the subtree in your monorepo is a copy so it persists. However, if the remote of the submodule is deleted then you can no longer clone it.

Compared to good ol' copy-paste, subtrees preserve git history. Although if you don't want the history you can also --squash.

Release Management for Microservices: Multi vs. Monorepos

Imagine a microservice application consisting of dozens of continuously-deployed autonomous services. Each of the application’s constellation of services has its own repository, with a different versioning scheme and a different team continually shipping new versions. Riddle me this: How can I tell the (whole) application’s version? Being that the change history is scattered among dozens of…

Before getting the answer to this question let's have a look at 3 major different types of software architecture. 

1.  Monolithic  Architecture

According to the geeksforgeeks blog, Monolithic Architecture is like a big container, wherein all the software components of an app are assembled and tightly coupled, i.e., each component fully depends on each other.

In a Monolithic application, every feature is in the same code and deployed on a single server.

It looks like this 👇 

Splitting a large, monolithic codebase into small, encapsulated modules is usually good practice from an architectural perspective. Modularization is useful for everything from microservices to libraries of reusable components. However, it can also be a nightmare from a publishing and source code management perspective.

Discovery goes downhill when there are dozens or hundreds of code repositories. When new repositories are created, everyone has to add their existence to their mental model of where code lives.

Access control becomes tedious and error prone, especially when access is granted to repositories one at a time on a “need to know” basis. New employees tend to go through an endless loop of dependency hell. Each access request simply reveals two more repositories they need to access as well.

Versioning is also complicated by a highly modular architecture. It often makes sense to “snapshot” a collection of related modules by incrementing their version in lockstep (this is what Babel and React do, for example). However, asking humans to remember to do this across a large collection of packages (including packages they haven’t had to touch) is asking for trouble.

Duplication of dependencies between packages can vastly increase the time required to install dependencies. Some ecosystems, npm in particular, are highly modular. This encourages reuse but also means that some packages are likely duplicated in every package you develop.

Monorepos are a popular solution to this problem. Instead of one code repository per module, you put all modules in the same code repository. This “monorepo” is then the only thing a developer needs in order to do development with your application. Discovery, access control, and versioning are all simpler by virtue of everything being in the same place.

Obviously, you have to be okay with granting access to all of the code or none of the code. But if this is acceptable, monorepos can give you all the benefits of modularization with very straightforward source code management.

So how do monorepos work with npm? The two most popular solutions are Yarn workspaces and lerna. Since lerna works with npm, let’s take a look at how it implements monorepos.

First, let’s install lerna globally.

$ npm install -g lerna

Next, we need to create new lerna repository:

$ mkdir monorepo_example $ cd monorepo_example $ lerna init

If you look at the contents of lerna.json, you can see where the version and packages are defined.

$ cat lerna.json {   "packages": [     "packages/*"   ],   "version": "0.0.0" }

As seen above, the version for all of the packages in our monorepo is 0.0.0. However, there aren’t any packages in our monorepo yet. Before we add them, we should login to our registry so lerna sets the publishConfig correctly on each new package.

If you’re publishing somewhere other than the public npm registry (for example, npm Enterprise), you’ll first need to set your registry.

$ npm config set registry https://registry.npm.yourcompany.com

Next, we authenticate to the registry with a user that has publish permissions. If your packages won’t have a scope, you can omit the --scope flag.

$ npm login --scope test

We now add a few scoped packages to our monorepo.

$ lerna create @test/a $ lerna create @test/b $ lerna create @test/c

If you chose the default settings for each package, you’ll now have three packages (a, b, c) in the test scope with version 0.0.0. Before we can increment the version, we need to commit what we’ve got so far and create a remote for git:

$ git add . $ git commit -m “Initial commit” $ git remote add origin [email protected]:username/reponame.git $ git push -u origin master

Now we can bump the version of all packages with one command.

$ lerna version major

This command not only bumps every package to 1.0.0, but also pushes the version update to git for you. The last step is to publish these packages to npm. Lerna lets us do this with a single command.

$ lerna publish from-git

If this command succeeded, you’ve successfully published all packages in your first monorepo to npm. Hooray!

In summary, we:

Created a new monorepo with Lerna.

Added three new scoped packages to our monorepo.

Bumped the version of all packages and committed to git with a single command.

Published all packages to an npm registry with a single command.

For more on Lerna, including their solution to the duplication problem (hoisting), I recommend their excellent docs at https://github.com/lerna/lerna.

We’re also excited to announce that we plan to bring first-class monorepo support to npm@7. If you use monorepos with npm, we need your feedback! Let us know what you like or dislike about your current monorepo solution.

Happy coding,