Visual recognition and product matching for furniture

Why photos, in furniture

In electronics, a manufacturer reference is often enough to link two offers. In furniture, the same sofa can be called Oslo at one retailer, Nordik at another, and carry a bare numeric reference on a marketplace. Photos change too, but far less than names: the silhouette, proportions, arms, stitching and legs remain those of the product.

That is why visual recognition has become the baseline method for matching furniture that shares no identifier.

How visual recognition works

A computer vision model reads a photo and derives a numeric representation from it, often called a vector or embedding. Two photos of the same product, or of very close products, give close vectors. Comparing one product with a whole market then means finding its nearest neighbors among thousands of vectors, which specialized tools do very fast, even on very large volumes.

Two broad families of models serve as starting points:

• models trained on images alone, which produce robust visual representations without human labels;
• models trained by pairing images with text, trained on hundreds of millions of image-text pairs.

These models know that a photo shows a corner sofa. They are much less sure that a sofa with slim arms and splayed legs is not the same as its neighbor with solid arms. Hence the next step.

Fine-tuning on furniture pairs

For a model to separate nearly identical products, it is trained on examples from the trade: pairs of photos labeled “same product” or “different product”. Every pair validated by a person teaches it what really matters in a sofa, and what is just staging.

Photo preparation matters as much as the model:

• compare the product only: the background, the rug or the plant should not weigh in the comparison;
• prefer front or three-quarter views, comparable from one site to another;
• compare several views when they exist, rather than a single lifestyle shot.

What a photo does not show

Vision has blind spots that matter in furniture:

• size: a two-seater and a three-seater of the same model look very alike in photos;
• mechanisms: a closed sofa bed looks like a fixed sofa;
• corner orientation, since a mirrored photo is enough to fool the model;
• fabric, as velvet and chenille in the same color can be confused.

That is why photos must be crossed with specifications: dimensions, seat count, functions. A good system rules a pair out as soon as one of these rules contradicts it, even if the photos look alike.

Measuring quality

A matching engine is judged on two figures:

• precision: among the pairs it suggests, how many are right;
• recall: among the true pairs that exist, how many it finds.

In price monitoring, a false pair costs more than a missed one, because it produces a false price gap and then a bad decision. It pays to be demanding, and to hand doubtful cases to human validation. Those validations in turn become training examples.

Searching the web by image

Consumer image search engines find a product from a photo and show where to buy it. They sometimes surface sellers missing from the usual lists. Their results need the same checks, though: same size, same function, same product.

Sources