Displaying DAG(needs) relationships in GitLab CI pipeline overview

Since this is the first time I am putting up a work on this website that relates to GitLab CI, let me start off with explaining in detail what GitLab CI and pipelines really are. Going by the book:

GitLab CI (Continuous Integration) service is a part of GitLab that build and test the software whenever developer pushes code to application.

Breaking it down into simpler words, GitLab CI consists of a set of actions/steps to be performed once a new code or a change is pushed to a repository. User get to define their own steps, which in this context are called Jobs and all the jobs together form a Pipeline.

The problem statement

Conventionally, the jobs in the gitLab CI pipelines were segregated into stages and the execution was set to happen sequentially. The system worked just fine until it was realised that some jobs which were only dependant on a single job in one of the prior stages, had an awful lot of waiting to do(for all jobs in all previous stages to execute and succeed first) until they could execute.

To reduce the wait time and make the pipelines faster, a new keyword called needs was introduced that now allows users to define the dependencies in the .gitlab-ci.yml(a configuration file to define CI pipeline in GitLab) and alter the format of execution. With needs, the jobs in a pipeline don’t have to wait of the prior jobs that it doesn’t directly depend upon.

Now even though the problem on the .gitlab-ci.yml side was fixed, we needed a way to reflect the relationships visually on the pipeline overview graph.

GitLab Issue

The design and conversations around the design could be found under the following issue: https://gitlab.com/gitlab-org/gitlab/-/issues/232946/

The users

The personas that interact with the graph section of the pipelines are mostly software developers and DevOps engineers who use the GitLab CI for their day to day work, or the development teams leads or even managers who want to get a quick idea about the status of the development process.

Considerations

At GitLab, we believe in a boring solutions. No, this does not mean that we only play safe. This implies we avoid breaking changes until necessary, and in the context of user experience, I translate this into ‘causing least disruption to user’s existing mental model’ when it comes to using GitLab CI.

The solution

In the absence of needs keyword in the .gitlab-ci.yml the default view would be ‘Organize by stage’ and the option to toggle would not be available

When needs is used, the default view for the pipeline visualization would be based ‘Organized by needs’ (or DAG). But users would still have the option to toggle back to ‘Organize by Stage’ view.

The connections between the jobs would always persist, in a non-overwhelming way. While hovering over a job, the train of jobs that depends on it would be highlighted along with the connection between them

To get a peek into the conversation that led to the final draft of the design, please look at the issue.

Disclaimer: the actual implementation might differ from the above-mentioned proposal.