While Google started automating its build tests in 2003, the engineering industry took longer to do the same. But automation was sorely needed: Software systems are growing larger and ever more complex… To make matters worse, new versions are pushed to users frequently, sometimes multiple times each day. This is a far cry from the world of shrink-wrapped software that saw updates only once or twice a year. The ability for humans to manually validate every behavior in a system has been unable to keep pace with the explosion of features and platforms in most software. - Software Engineering at Googlehttps://abseil.io/resources/swe-book/html/ch11.html#testing_overview Add author/source Sun Microsystem’s engineer, Kohsuke Kawaguchi, was key to ushering in the next era of testing. In 2004, he created “Hudson”https://community.jenkins.io/t/lets-thank-kohsuke-the-creator-of-jenkins/168 (later renamed to Jenkins in fun Oracle drama). At his day job, Kohsuke “got tired of incurring the wrath of his team every time his code broke the build.” He could have manually triggered tests before each code contribution, but instead, Kohsuke chose the classic engineering solution and created an automated program. The Hudson tool acted as a long-lived test server that could automatically verify each code change as it integrated into the codebase.
I agree so, so much; and I’ve been saying similar things for years.
But I recognize that it’s probably a hard problem. For one thing, auth is almost never going to work the same way in CI as it does locally.
…still, though, I feel like there could be some much nicer tooling here.
It can’t be that hard to create a test harness, so I can plug my own variables into the runner context.
Even just an 80% solution would speed up development tremendously.
Well, for GitHub Actions, there’s this: https://github.com/nektos/act