While Google started automating its build tests in 2003, the engineering industry took longer to do the same. But automation was sorely needed: Software systems are growing larger and ever more complex… To make matters worse, new versions are pushed to users frequently, sometimes multiple times each day. This is a far cry from the world of shrink-wrapped software that saw updates only once or twice a year. The ability for humans to manually validate every behavior in a system has been unable to keep pace with the explosion of features and platforms in most software. - Software Engineering at Googlehttps://abseil.io/resources/swe-book/html/ch11.html#testing_overview Add author/source Sun Microsystem’s engineer, Kohsuke Kawaguchi, was key to ushering in the next era of testing. In 2004, he created “Hudson”https://community.jenkins.io/t/lets-thank-kohsuke-the-creator-of-jenkins/168 (later renamed to Jenkins in fun Oracle drama). At his day job, Kohsuke “got tired of incurring the wrath of his team every time his code broke the build.” He could have manually triggered tests before each code contribution, but instead, Kohsuke chose the classic engineering solution and created an automated program. The Hudson tool acted as a long-lived test server that could automatically verify each code change as it integrated into the codebase.
The sentiment of the first half of your comment is the cause of the problem you describe in the second half. Why /should/ the CI tool have any “steps” built in? Use a task runner, or script in your repo for any task you expect CI to do. Configure CI to run the same command you would run locally.
It’s not the same thing. Sometimes there are rules/conditions in the pipeline you need to test. Sometimes there are child pipelines. There really should be a way to simulate a pipeline run locally.
It can make the UI much nicer (though I agree there are probably better ways to do it, e.g. native support for letting the program output determine the UI).
Sometimes you need to run different things on different machines/OSes, and it’s easier if you have separate steps there.
The sentiment of the first half of your comment is the cause of the problem you describe in the second half. Why /should/ the CI tool have any “steps” built in? Use a task runner, or script in your repo for any task you expect CI to do. Configure CI to run the same command you would run locally.
It’s not the same thing. Sometimes there are rules/conditions in the pipeline you need to test. Sometimes there are child pipelines. There really should be a way to simulate a pipeline run locally.
I totally agree, with some caveats: