How We Conduct AI Search Research

Our Research Principles

Methodology Should Be Visible

Research findings are more useful when readers understand how the research was conducted.

Where appropriate, research reports should explain:

• research question
• sample size
• data source
• collection period
• platforms tested
• prompts or queries used
• inclusion criteria
• exclusions
• analysis method
• limitations

We avoid presenting conclusions without enough information for readers to understand the basis behind them.

Evidence Comes Before Conclusions

We do not begin research with a predetermined answer that the data must support.

The research question should come first.

The evidence should then determine what conclusions are reasonable.

If the data does not support a strong conclusion, the report should say so.

Research Questions

Every original study should begin with a clearly defined question.

Examples may include:

• Which sources are most frequently cited in ChatGPT Search?
• How often do local businesses appear in AI-generated recommendations?
• Do websites ranking highly in Google also appear frequently in AI responses?
• What types of pages are cited in Google AI Overviews?
• Which Canadian brands receive the strongest AI-search visibility?
• How frequently do AI search platforms cite primary sources?
• How stable are AI-generated recommendations over time?

A clear research question helps determine the appropriate sample, data and methodology.

Data Collection

The method used to collect data depends on the research question.

Data may come from:

• manually conducted searches
• structured query sets
• AI-platform outputs
• search engine results
• publicly available websites
• platform documentation
• APIs where permitted and available
• third-party SEO datasets
• original surveys
• agency or business submissions
• publicly accessible business information

Research reports should identify the relevant data sources.

AI Search Testing

Research involving AI-search platforms requires special care because generated responses may change.

Depending on the study, testing may document:

• platform used
• account or access context where relevant
• date and time period
• query wording
• geographic context
• device or browser context where relevant
• whether search/browsing functionality was enabled
• source citations returned
• response variation

Where appropriate, the same or similar prompts may be tested multiple times to evaluate consistency.

Prompt and Query Design

The wording of a query can influence AI-generated responses.

For that reason, studies involving prompts should explain how prompts were selected.

Query sets may include:

• informational queries
• commercial queries
• local queries
• comparison queries
• recommendation queries
• branded queries
• non-branded queries

Where possible, prompts should reflect realistic questions users may actually ask rather than being designed solely to produce a desired result.

Sample Selection

Research quality depends heavily on how the sample is selected.

A study may include:

• websites
• businesses
• agencies
• industries
• search queries
• cities
• AI responses
• citations
• search result pages

The report should explain why the sample was selected and what population it is intended to represent.

We should avoid implying that a small sample represents an entire market without appropriate qualification.

Sample Size

A larger sample can sometimes improve confidence, but sample size alone does not determine research quality.

A smaller, carefully controlled study may be more useful than a larger dataset with unclear methodology.

Where applicable, we disclose:

• total number of queries
• total responses analyzed
• total websites or businesses evaluated
• number of cities
• number of industries
• number of citations
• number of repeated tests

Inclusion and Exclusion Criteria

Studies may need rules defining what counts and what does not.

Examples include:

• whether sponsored results are included
• whether duplicate citations count separately
• whether social platforms are treated as sources
• whether the same domain may appear multiple times
• whether inactive businesses are excluded
• whether only Canadian businesses are included
• whether aggregator sites are counted separately

These rules should be defined before drawing conclusions wherever practical.

How We Evaluate AI Citations

A citation study may examine characteristics such as:

• cited domain
• page type
• brand
• source category
• ranking position
• publication type
• local relevance
• authority signals
• whether the source directly supports the generated answer

We may also distinguish between:

• primary sources
• secondary sources
• media
• directories
• government sources
• academic sources
• company websites
• community platforms
• user-generated content

Not every study will use every category.

How We Measure AI Visibility

AI visibility does not have one universally accepted measurement standard.

Depending on the research, we may evaluate:

• frequency of brand mentions
• citation frequency
• recommendation frequency
• presence across query sets
• share of mentions
• ranking or position within generated lists
• platform coverage
• query-category coverage
• changes over time

If we develop proprietary metrics, the methodology should explain how those scores are calculated.

A proprietary score should not be presented as an official industry standard.

Manual Review

Automated data collection may help process larger datasets, but automated systems can make classification errors.

Where appropriate, research may include human review to:

• verify citations
• classify sources
• remove duplicates
• confirm business identity
• detect irrelevant results
• review ambiguous responses
• identify false positives
• validate unusual findings

The balance between automation and manual review should be explained when it materially affects the findings.

Use of AI in Research

AI tools may assist with parts of the research workflow.

Examples include:

• organizing data
• identifying patterns
• categorizing records
• summarizing large datasets
• helping draft explanations
• checking consistency

AI-generated analysis should not automatically be treated as factual evidence.

Important findings should be validated against the underlying data.

AI tools should not invent missing observations, statistics or sources.

Data Cleaning

Raw datasets often require cleaning before analysis.

Depending on the research, this may include:

• removing duplicate records
• normalizing company names
• normalizing domains
• correcting obvious formatting errors
• grouping equivalent source types
• removing invalid responses
• identifying incomplete records
• documenting exclusions

Material cleaning decisions should be described where they could affect the outcome.

Analysis

The analysis method should match the research question.

Depending on the study, analysis may include:

• frequency counts
• percentages
• comparisons
• category analysis
• visibility scores
• citation distribution
• platform comparisons
• time-based comparisons
• geographic comparisons
• industry comparisons

We avoid using complicated statistical methods simply to make research appear more sophisticated.

Correlation Is Not Causation

AI-search research frequently identifies relationships between factors.

For example, a study may find that certain types of websites appear more frequently in AI citations.

That does not automatically prove that the observed characteristic caused the citation.

We should clearly distinguish between association and causation unless the research design genuinely supports a causal conclusion.

Research Limitations

Every study has limitations.

Potential limitations may include:

• limited sample size
• geographic restrictions
• query selection
• personalization
• rapidly changing AI models
• platform updates
• response variation
• incomplete public data
• inaccessible proprietary systems
• time period of collection

Limitations should not be hidden.

Explaining limitations helps readers understand how broadly the findings should be applied.

Reproducibility

Where practical, we want readers to understand enough of the methodology to repeat or independently evaluate the study.

A research report may therefore include:

• prompt examples
• query lists
• scoring definitions
• classification rules
• data collection dates
• sample description
• methodology notes
• downloadable datasets where appropriate

Some information may not be published if doing so would expose confidential, private or licensed data.

Research Updates

AI search can change quickly.

A study represents the conditions observed during its research period.

Results should not automatically be assumed to remain unchanged indefinitely.

Research may be repeated or updated when:

• AI models change significantly
• search interfaces change
• citation behaviour changes
• new platforms emerge
• larger datasets become available
• earlier findings require reassessment

Where a study is updated, the publication should make the relevant dates clear.

Corrections

If a material error is discovered in a published research report, we may:

• correct the data
• update the methodology
• revise a chart
• correct a calculation
• revise the conclusion
• add clarification
• publish a correction notice where appropriate

Minor formatting changes do not necessarily require a formal correction notice.

Research Independence

Research findings should not be changed simply because they are inconvenient for:

• an agency
• an advertiser
• a sponsor
• a software company
• a commercial partner

If research receives external sponsorship, that relationship should be disclosed.

Sponsorship should not guarantee a favourable conclusion.

Sponsored Research

AI SEO Experts Canada may eventually conduct sponsored research.

Sponsored studies should clearly identify the sponsorship.

Where possible, the sponsor should not control:

• the underlying data
• the final findings
• whether unfavourable findings are omitted
• the editorial conclusion

Any material methodological limitations created by the sponsorship arrangement should be disclosed.

Research vs Editorial Content

Research reports and editorial articles serve different purposes.

Research

Research is based on a defined question, methodology and dataset.

Editorial Analysis

Editorial analysis interprets developments, research or industry changes.

Educational Content

Educational content explains concepts and practices.

Readers should be able to understand which type of content they are reading.

What We Do Not Consider Good Research Practice

We avoid:

• inventing data
• selectively hiding inconvenient findings
• presenting opinion as research
• changing methodology after seeing results without disclosure
• using tiny samples to make sweeping claims
• citing AI-generated answers as independent proof of themselves
• presenting correlation as causation
• copying third-party statistics without reviewing the source
• publishing unsupported percentages
• presenting proprietary scores as universal standards

Our Research Checklist

Before publishing original research, we should be able to answer:

• What question did we investigate?
• What data did we collect?
• How was the sample selected?
• When was the data collected?
• How was the data analyzed?
• Can readers understand our methodology?
• What limitations exist?
• Do the conclusions actually follow from the evidence?
• Could commercial interests have influenced the result?
• Can important findings be independently checked?

If those questions cannot be answered adequately, the research is not ready for publication.

Explore Our Research

AI SEO Experts Canada will publish research examining how businesses, websites and sources are being discovered across emerging AI-powered search experiences.