REngine: a graph-based recommendation engine
Recommendation engine that moves a Sylius / MySQL shop's data into a Neo4j graph, computes affinities and customer similarities, and serves suggestions through FastAPI.
- Context
- Side project, still ongoing: recommending products from the purchase history of a Sylius / MySQL shop.
- My role
- The whole project: MySQL to Neo4j ETL, affinity and similarity computation, recommendation API.
- Outcome
- A prototype answering in 19 ms on average locally, thanks to affinity links precomputed in the graph.
The idea
An e-commerce shop’s data is relational: customers, products, orders. For recommendations, the interesting question is precisely the relationship: “customers who look like this one, what else did they buy?”. In SQL, that quickly turns into a chain of joins. In a graph, it is a traversal.
So REngine moves the data of a Sylius shop (MySQL) into Neo4j, and computes recommendations in the graph.
The pipeline
Four steps, each in its own service:
- Sync: customers, products and orders migrated from MySQL to Neo4j, filtering out the noise (see below).
- Affinities: each customer is linked to the products they are interested in (
INTERESTED_IN). - Similarities: customers with close profiles are linked together (
SIMILAR_TO). - API: recommendations are served over REST by FastAPI.
The code is split into migrators, services and repositories, so one step can change without touching the others.
The problem: products everyone buys
In the first tests, basic items (reusable bags, everyday products) linked almost every customer to every other one. The result: similarities that meant nothing.
The fix: an analytics service finds the top 1% of sales, labels them CommonProduct and leaves them out of the similarity computation. Free or promotional products are also set aside, to keep only real purchase choices.
Measuring how close two customers are
Two customers are candidates if they share at least three products. A first similarity pass runs in memory with Pandas; to go further, I use the Neo4j Graph Data Science (GDS) extension, which writes weighted SIMILAR_TO relationships straight into the graph.
Suggestions then rely on the GraphAware Recommendation Engine framework, and on APOC for basket analysis.
The principle: look at what the closest customers bought, and suggest it.
Avoiding duplicates
A useful recommendation is not a variant of the product already viewed. If a customer looks at a box of 30 capsules, suggesting the box of 60 adds nothing: the engine sets these variants aside and suggests something else.
Performance
Affinity and similarity links are precomputed. A recommendation query only has to traverse the graph: 19 ms on average, locally.
What I take away
- The shape of the data matters as much as the algorithm: moving from tables to relationships made the problem much simpler.
- Cypher replaces a lot of procedural code with a single graph query.
- Specialized tools (GDS, GraphAware) save reimplementing similarity algorithms by hand.
What’s next
It is a prototype I keep working on. The next step is to recompute affinities and similarities periodically, to take new purchase behavior into account as it happens.