How to choose an AI website personalisation tool
3 min read · updated
Judge an AI personalisation tool on five questions rather than its demo. What exactly can the AI change on your page, and who approves it? How is lift measured, and is there a control group that never sees changes? What happens when the tool is slow or wrong? What visitor data does it store and where? And what will it cost at your real traffic, not the traffic in the sales deck? A tool that answers all five clearly is safer than one with a more impressive demo.
What is the AI allowed to change?
This is the question that separates tools most. Some generate whole new page sections or code. Some draw overlays and popups on top of your site. Some only change text, visibility and links on elements you marked in advance. More freedom can mean bigger wins, but it also means more ways to publish something off brand, wrong or broken.
Ask for the list of operations in writing. Ask whether a generated change can include anything executable, and whether it can touch prices, legal text or checkout.
How does the vendor measure lift?
Ask whether there is a holdout group that never sees any change, how big it is, and whether the same visitor always stays in the same group. Ask what minimum sample the tool requires before it calls a winner. A vendor that reports lift only as before versus after, or only between its own variants, cannot tell you whether the tool beats doing nothing.
Also ask how conversions are attributed. A purchase three days after a change should be credited once, to the right exposure, within a stated window.
How does it fail safe?
Every tool will be slow or wrong sometimes. What matters is what the visitor sees when it is. Good answers: the page renders as normal and changes apply only if they arrive within a short budget; a kill switch stops every change at once; a rolled back change restores exactly what was there.
- Does the page wait for the tool before showing content?
- Is there a kill switch, and how fast does it take effect?
- Can you require approval before any change goes live?
- Is every change logged with what it replaced?
What data does it collect and store?
Ask about IP addresses, session recordings, keystrokes and form fields, and where data is stored. In India the DPDP Act applies to personal data, and a tool that records more than it needs is a liability you inherit. Behavioural intent can be read from page views and engagement without storing who the person is.
What will it really cost?
Pricing models differ: per seat, per monthly tracked user, per session, per credit, or custom contracts. Model your real traffic and your peak month, then ask what happens when you go over. A low entry price that jumps with traffic can cost more than a higher flat price within a year.
For reference, IntellQ answers these as follows: four operations on marked slots plus offers from your own guardrails, a 20 percent holdout by default with a 200 visitor minimum per arm, a 2.5 second decision budget after which the page stays unchanged, no IP addresses stored, and published monthly plans with stated credits.
How should you run a trial with a shortlisted vendor?
Give each shortlisted tool the same two weeks on the same pages, outside a sale. Ask it to propose changes, approve only the ones that fit your brand, and judge it on orders per visitor against its holdout rather than on its dashboard summary. Write down how long setup took, how many proposals you rejected and why, and whether anything broke.
A tool whose proposals you keep rejecting is telling you its idea of your customer differs from yours. That is worth knowing before you sign an annual contract.
Questions
Is an AI personalisation tool better than manual A/B testing?
It runs more tests with less of your time, and can tailor to each visitor. It is only better if it measures against a control group and stays inside limits you set.
Should the AI be allowed to publish changes on its own?
Start with approval required for every change. Loosen it only for low risk changes once you have seen the tool's judgement over several weeks.
What is a red flag in a vendor demo?
Lift shown only as before versus after, no answer on what happens when the tool is slow, and no written list of what the AI can change.