AMA Recap: What We Learned About Tracking AI-Driven Profit on Shopify

On August 25, we joined the Shopify Community for an AMA around tracking AI-driven profit, with a simple question at the center of the discussion:

As AI becomes another way for shoppers to discover and buy products, how should merchants actually measure whether that traffic is valuable?

The conversation moved beyond simply tracking AI traffic. We talked about product discovery, attribution, profitability, and where tools like Shopify Sidekick fit into the reporting process.

Here are the main points that came out of the AMA.

AI product discovery and AI citations are two different things

One of the most useful clarifications from the discussion was that AI product discovery and AI citations should not be treated as the same pipeline.

Community member Tim57 pointed out that Shopify Catalog and Agentic Storefronts are what help AI shopping experiences access product information such as variants, availability, attributes, and other commerce data.

Meanwhile, AI crawlers visiting store pages for search or citations are a different process.

This distinction matters because seeing AI crawler traffic does not necessarily mean your products are appearing in AI shopping results.

It was a useful reminder that merchants should look at how product data is being distributed, not only whether an AI crawler has visited their website.

AI attribution is still messy

Another topic that came up was attribution.

AI-referred visits do not always appear neatly under an “AI” traffic source. Some sessions may still be categorized as Direct or otherwise lose part of their referral context.

That creates a challenge when merchants try to answer a seemingly simple question:

How much revenue did AI actually generate?

The discussion reinforced that merchants should be careful about relying on one attribution number alone.

Instead, AI traffic should currently be treated much like an emerging acquisition channel: useful, promising, but still imperfect from a reporting perspective.

Order volume alone does not tell you whether AI traffic is good

The discussion also came back to one of the core themes of the AMA: more orders do not automatically mean better business results.

When evaluating AI traffic, merchants should look beyond order count and revenue.

Refund behavior, product costs, fulfillment costs, and the actual contribution left from those orders matter just as much.

A channel can generate impressive revenue while creating relatively little profit.

That is why one of the broader takeaways from the session was to evaluate AI like any other acquisition channel: compare the quality and economics of the orders, not only the number of orders generated.

Sidekick is useful for exploration, but the underlying data still matters 

Shopify Sidekick was another part of the conversation.


A useful way of thinking about Sidekick that came out of the discussion was:

Sidekick can help you find where to look, but it should not replace the underlying business data.

For example, a merchant can use Sidekick to identify an interesting channel, product, or change in performance. But when the question becomes whether that channel is truly profitable, the merchant should still verify the actual orders, refunds, and costs behind the result.

As Tim57 put it in the discussion, Sidekick is useful as a map, rather than the final source of truth for profitability.

Some merchants may not have explored Agentic Storefronts yet

The AMA also surfaced a very practical point: before analyzing AI commerce performance, merchants first need to know what AI commerce features are actually enabled for their store.

For example, the discussion noted that some merchants may not yet have explored Agentic Storefronts in their Shopify settings.

That sounds simple, but it is an important first check. There is little value in building an AI reporting workflow before understanding which AI surfaces are actually available or active for the store.

What we took away from the AMA

The biggest takeaway was that AI commerce should not be treated as one single metric.

There are several separate questions involved:

  • Can AI understand and surface your products?
  • Is AI actually sending shoppers to the store?
  • Can those visits be attributed correctly?
  • Are those shoppers converting?
  • Are the resulting orders profitable?
  • Can AI tools help merchants investigate that data without hiding the underlying numbers?

 

The discussion also showed how quickly this area is still evolving. Some parts of AI commerce already fit into familiar ecommerce reporting, while others, particularly attribution and product discovery, still require merchants to look at multiple signals rather than one dashboard number.

Thanks for joining us

Thank you to everyone who joined the AMA, followed the discussion, and shared feedback.

Special thanks to the community members who added practical context around AI product discovery, attribution, and reporting. Those perspectives helped move the conversation from “AI is growing” to the more useful question:

“How do we know whether AI is actually creating value for the business?”

You can revisit the original discussion here:

Tracking AI-driven profit: 8 checks every Shopify merchant can do now

And if you’re already seeing AI-referred traffic or orders in your Shopify store, we’d still love to hear what you’re seeing.

What has been hardest to measure so far: AI discovery, attribution, conversion, or profit?