We Let AI Build Shopify Code. The First Failed Publish Changed Everything.
Almost two years ago, we began with what seemed like a straightforward idea.
At Shugert, we had accumulated years of experience completing Shopify engineering tasks across different stores, themes, apps, and business models.
Many requests looked repetitive:
Create a custom product section.
Modify a collection template.
Fix a JavaScript interaction.
Add structured data.
Improve performance.
Change a cart behavior.
Implement a tracking event.
Update a theme without breaking the storefront.
We had already solved variations of these problems many times. The knowledge existed in our repositories, internal documentation, completed projects, and engineering processes.
The original assumption was simple:
If we could organize that experience into a sufficiently large library of Shopify tasks, an AI system could identify a request, find a similar implementation, adapt it to another store, and execute the change.
The language models could already write the code.
We thought that would be the difficult part.
We were wrong.
The first version was essentially an intelligent task library
Our earliest approach was built around work we had already performed manually.
Each task contained some combination of:
The original merchant request.
The relevant theme files.
The implementation steps.
The code changes.
The validation process.
The expected result.
The goal was not to generate arbitrary Shopify code from scratch. It was to reuse patterns that had already worked and adapt them to the context of another store.
Then the models improved.
They could inspect Liquid templates, generate theme sections, modify JavaScript, work with JSON templates, and explain what they had changed. The resulting code often looked surprisingly good.
A merchant could describe a change in plain English, and the system could produce something close to a working implementation.
That felt like the breakthrough.
It wasn’t.
Code that looks correct can still be wrong for the store
Shopify themes are not interchangeable.
Two stores may ask for the same visible result while requiring completely different implementations.
One theme may use custom elements and event-driven JavaScript. Another may rely on jQuery. A third may have an app intercepting the cart. A fourth may contain years of undocumented modifications from several developers.
Even seemingly simple changes depend on context:
Which theme is currently published?
Does the merchant have an unpublished staging theme?
Has the production theme changed since the task was created?
Which apps modify the same interface?
Are there theme-specific conventions that must be preserved?
Does the implementation affect subscriptions, analytics, localization, accessibility, SEO, or checkout behavior?
What happens if only part of the change is successfully applied?
The model could generate valid Liquid and JavaScript while still making an invalid engineering decision.
We learned this when early publishing attempts caused themes to fail.
The problem was not necessarily that the generated code was syntactically incorrect. The code had been created or applied with an incomplete understanding of the environment surrounding it.
A reasonable implementation had been placed into the wrong context.
That failure changed the direction of the entire product.
We stopped treating code generation as the product
Generating code is an action.
Engineering is a controlled process that surrounds that action.
A trustworthy autonomous engineering system must determine:
What the merchant is actually requesting.
Whether the request is sufficiently clear.
Which store and theme the request applies to.
What dependencies and risks exist.
Which files may be changed.
What the expected result should be.
How the implementation will be tested.
Where the change will be deployed.
What evidence will be presented for approval.
What happens if deployment fails.
How the system recovers if the store changes during execution.
Who authorized the final publication.
The model is only one component.
The product is everything that surrounds it.
That realization pushed us from an AI code generator toward what became TaskerArmy: an autonomous Shopify engineering platform designed around controlled execution, reviewable output, and merchant approval.
The first rule became: never the live theme first
Once a system can modify a real storefront, permissions become more important than prompts.
A good prompt cannot replace a deployment boundary.
Our most important boundary became simple:
Never your live theme first.
TaskerArmy stages work on a safe theme instead of treating the published storefront as a development environment.
The merchant should be able to inspect the result before deciding whether it belongs in production.
That staging-first approach affects the entire workflow:
Ask → Plan → Build → Test → Preview → Approve
The output is not merely a block of generated code pasted into a conversation. It is a proposed changeset attached to a specific store, theme, request, and approval state.
This distinction matters because chat is temporary, but engineering state is not.
Evidence is more valuable than confidence
Language models are very good at producing confident explanations.
Confidence is not proof.
An engineering agent should not say that a task was completed merely because it generated code and encountered no obvious error.
It should show evidence.
Depending on the task, that evidence may include:
The exact files that changed.
A structured diff.
Automated validation results.
Theme-check results.
Screenshots of the staged storefront.
Browser-based interaction tests.
Performance comparisons.
Accessibility checks.
The preview URL.
Known limitations or unresolved risks.
The merchant should not have to trust that the agent interpreted the request correctly. The system should make the result inspectable.
That also changes the meaning of approval.
Approval is not a casual response in a chat window. It is an authorization tied to a particular version of a changeset. If the code changes afterward, the previous approval should no longer be valid.
The thing that was approved must be the thing that gets deployed.
Rollback is not an undo button
Recovery became another area where the initial mental model was incomplete.
It is tempting to think of rollback as restoring the files that existed before a deployment.
But Shopify stores continue changing.
A merchant may publish another theme. An app may modify a template. A developer may deploy an unrelated fix. Theme settings may change between the original execution and the attempted rollback.
Restoring an old snapshot without reconciling the current state can overwrite legitimate work.
Rollback is therefore another deployment.
It requires:
A known previous state.
A known current state.
Drift detection.
Conflict analysis.
A new changeset.
Validation.
Authorization.
This is less exciting than watching an AI generate a complete section in seconds, but it is far more important if the system is expected to operate on real businesses.
Autonomy should be bounded, not absolute
We no longer think of autonomy as a switch that is either on or off.
Different tasks require different levels of authority.
A low-risk content adjustment may be safe to stage automatically. A change affecting checkout, subscriptions, pricing, tracking, or a heavily customized theme may require additional review.
Some requests are too ambiguous to execute responsibly.
The correct autonomous action may be to stop and ask a question.
That is not a failure of autonomy. It is evidence that the system understands its authority.
A useful engineering agent needs to know:
What it may inspect.
What it may modify.
Where it may deploy.
Which validations are mandatory.
When it must request clarification.
When human approval is required.
Which operations it must never perform autonomously.
The goal is not to remove humans from every decision. The goal is to remove unnecessary manual execution while preserving human control over consequential decisions.
The operating system matters more than the model
Models will continue improving.
They will generate better code, understand larger repositories, call more tools, and complete longer sequences of work.
We designed the surrounding system with the assumption that models are replaceable.
The durable value lives elsewhere:
Store context.
Theme awareness.
Permissions.
Planning.
Changesets.
Validation.
Evidence.
Deployment state.
Recovery.
Project memory.
Accountability.
A stronger model can improve execution, but it does not eliminate the need for those layers.
In some ways, a more capable model makes them more important. The greater the system’s ability to act, the more precisely its authority must be defined.
What we are building now
TaskerArmy is evolving toward a system that can own the workflow from a Shopify request to an approved staging result.
The intended experience is straightforward:
A merchant connects a store.
The merchant describes the required change.
The system inspects the relevant context.
It creates an execution plan.
It builds and validates the implementation.
It deploys the result to staging.
It presents the preview and supporting evidence.
The merchant reviews and approves the result.
The system performs the authorized deployment and records what happened.
Behind that simple experience is the difficult part: safely coordinating state, permissions, tools, models, Shopify resources, validation, and recovery.
AI made code faster.
Code generation alone did not create a trustworthy engineering workflow.
That has been the central lesson from almost two years of building this system.
The question we are still exploring
Most conversations about coding agents focus on how much code they can generate or how many software tasks they can complete.
I think the more important question is different:
What must exist around an AI agent before you can responsibly give it permission to act?
For anyone building with coding agents:
Where do you draw the line between autonomous execution and mandatory human approval?
I published the complete engineering retrospective, with more detail about authority, changesets, staging, evidence, deployment, and recovery, on Medium:
The strongest insight is that code generation was never the real bottleneck. Once an agent can change a live storefront, staging, evidence, approval, and rollback become part of the product itself.