language models, but today Mark and Warren are going to talk about how small language models are actually a powerful tool to match freelancers and companies. Mark is the director of data science at Malt, and Warren is leading

the R&D for advanced matching topics. So please give a warm round of applause for Mark and Warren.

Yeah, thank you for the introduction, and hi everyone. So I'm Mark, and uh, with my colleague Warren here, we're super

excited to be here today to talk to you about how we scale freelancer matching at Malt, um, thanks to a small language

model that is a bit special. Uh, so first a few words about Malt, because you might not all know the platform. So we're the

freelancing marketplace leader in Europe with more than 700,000 freelancers, um, and we make the

perfect match happen between freelancers and companies. So, as you might have guessed, making that perfect match is a big focus

of the data science team. Um, and as a company, you have two main ways of finding your talent. The first one

is through a search. You can browse the profiles, select the one you're interested in, and contact them. The

second way is by completely describing your project, and then our work commander system is going to take care of

contacting the best freelancer for you. So for today, we're going to focus on that recommender system. We had a recommender system

in place for a while. Um, at its core, we find the ranker, which is basically in charge of, um, seeing like,

um, how good the fit is between a freelancer and a project, to in the end come up with a ranking. The problem with

that approach is that it can take a long time. Uh, when you think about it, if you want to have the global ranking and

that approach is that it can take a long time, when you think about it, if you want to have the global ranking and

If you put aside the filtering that you can apply at the beginning, you might have to call your model up to 700,000 times to get the score for each pair of projects and freelancers to get the global ranking.

So a usual way to deal with that is to add a retrieval phase, also known as candidate generation. In our case, it means finding a way to go fast, but of course, approximately from 700 freelancers to a few hundred good candidates.

Once you have these candidates, you're back to normal and you can send them to the ranker. So that looks great, but where and how could we build such a retriever?

Okay, so before jumping to a solution, what we can do is have a look at what data we have at our disposal. So whenever a company contacts us to do automatic matching, we ask them to fill in a brief, and this brief contains three types of information.

The first one regarding what kind of freelancer they're looking for: job title. We ask them also to select a job category from our taxonomy and then some mandatory and bonus skills that the freelancer should have.

Then we ask them some business information: how much they're willing to pay, when should the project start, how long should it be? And finally, we ask them to describe what will be the mission with a mission title and a mission description.

On the other hand, the freelancers have profiles on our platform. This profile contains information about them, so with a job title, a job category, also some skills that they possess, but also a description about who they are and what they can do.

Uh, and uh, some past experiences. So with those two documents, uh, we could do a perfect word match, uh,

meaning that we will retrieve the profiles that contain most of the keywords from the project brief.

And in short, that is lexical matching using BM25. So that sounds great. It's not actually perfect, but uh, is it? Did we solve

retrieval with BM25? Uh, not really, because otherwise we wouldn't be here during this presentation. So BM25 is quite

fast, uh, but uh, lexical matching doesn't grasp the meaning and the context uh, behind words, and these in our case have some

issues because, uh, freelancers and companies may not speak the same language and may not use the same

dialect. Uh, they may mean the same thing but not express it the same way. So some examples: uh, a company may

actually deem it important to mention Git as a skill, whereas for a developer it's actually obvious and they don't even

mention it on their profile. Another example is maybe a company is looking for someone that can do deep learning, but for a freelancer,

they may rather choose to, uh, define what kind of technologies or what kind of framework they're

using. And lastly, uh, what we would also like to do is to be able to match freelancers and companies even

though they're not using the same language in their profiles and their briefs. So we could actually tackle keeping

BM25 and tackle all of those issues separately. But Mark, is there a magical solution that can, uh, tackle all of those

issues at the same time? Yeah, of course, Warren. So generative AI? No, I'm kidding. Um, neural retrieval. So

let's take a look at what exactly neural retrieval is. Um, to build a neural retriever,

You need an organized latent space. So what is a latent space? That's where the embeddings lie. And what are embeddings?

Embeddings are basically the coordinates for our profiles and our project. Uh, here we've got an example project in 2D, and we see some kind of organization with one job category being in one part of the space and another family in one part of the space.

That organization, in our case, would mean that the distance equals the skill match level. So if two freelancers or a project and the freelancers are next to each other, there's a good skill match. If they're far away, it's a bad skill match.

Um, to get that latent space, we will need encoders. So since profile uh differ a bit from the project briefs, we will need two encoders. But if we manage to get these encoders, then it's fairly easy to do the retrieval because once we get a new project,

we just have to encode that project and get its embedding—so basically, its coordinates—and then we just have to look around, um, who are the closest freelancers. So here, the nearest would be the best matches that we would send to the ranking phase after.

Um, so that looks good, but how do we get these encoders? Okay, so since we're working with text, we can actually have a look at the language model landscape. So, a bit of a side note: this figure is actually outdated, as we all know things are moving quite fast with language models.

But this figure is still interesting because you can see that, depending on the model, they may have different advantages, weaknesses, and strengths. And if we zoom a bit back and look at how they're trained, we see a...

A pattern is emerging, and uh, when a model is trained doing auto-regression, meaning that it's just next-word prediction, seeing only what is before this word, they're actually better for content generation. Whereas when models are trained using auto-encoding, seeing

the full sequence, uh, they're actually better for extracting information. So in our case, we don't want to use generative models; we actually want to

use encoder models. But if you recall what I said earlier, we have another constraint, which is that we want

cross-lingual matching. And so the big question is, how do we do cross-lingual matching? Okay, so the answer is that, um,

a multilingual language model is probably what we need, um, because there are two properties of these languages that are

interesting for us. The first one, and I already talked about it, is that the latent space is going to be organized so

similar things should be close to each other, and things that are different should be far away from each other. But

the other constraint in a multilingual language model that we want is to have the alignment between the languages. What

alignment means is that for the same concept in different languages, we should have—roughly, they should be roughly

located in the same location in the latent space. But we should be careful because not all latent spaces are born

equal. Um, depending on how you train your model, uh, you might get these nice properties, or you might not. So you need

to carefully pick which multilingual language model, uh, uh, you want. A good candidate for us is to take a look at, uh,

the multilingual SentenceBERT, uh, for two reasons. The first one is they are trained on sentence similarity, so that's

uh, what will get us the nice organization, um, in one language, and then they have the language alignment.

through distillation. Um, so now we had that we could say, okay, we have a model; it's aligned

between the languages; it's organized, so we could use it as is. The problem is that, if you remember at the beginning, we

want the distance to represent the skill fit, but it was trained on sentence similarity, so the distance is more about

semantics. So it's not exactly what we want. So then you could say, okay, let's fine-tune this model on our data. Um, the

problem that's going to happen is that you're going to break the alignment between the

languages. So, for example, at Mall, we have a lot in France, Germany, and Spain—so these countries and the languages they

use might actually favor this language and break the generalization over languages. In addition, in both cases,

as you remember, we presented—like, we have some structure in the profiles and the project brief—but here we're just

aggregating everything and losing the section and the structure. So now the next question is, how can we adapt to our use case

this pre-trained model while keeping the interesting properties I just mentioned? Okay, so to build this in-

between solution, what we decided to do is to create and develop our own architecture that is able to process

documents—so briefs or profiles—into a unique embedding vector. And the big picture is that this architecture is

made of four steps, two of which are sub-models and two of which are formatting steps. So the first model is actually a

multilingual model that we use as a front-end backbone to keep the language alignment, and the second one is a

trainable head based on a BERT model that will be trained to fit and adapt to our skill matching.

case. So that is for the big picture. Now, we can dive a bit into each of those steps. And so, for the first step, we

need to decide how we can model uh documents. So here, you have JSON profiles, profile, sorry, as an example, and

you can see that there's different sections in it. What we could do is just concatenate all the sections into one

text, but we will be losing the structure of the document. So what we could do in order to

keep the structure is to separate each section with a special keyword. Uh, but in both those cases, what we saw in our

experimentation is that we introduce too much noise, giving the model all this text at once. It's actually

not the best way to do it. What we could do, however, is to use a larger model and more data, but that is

not the path that we decided to follow. What we actually did was to process each section independently, so first obtaining sequences of token

IDs, and then processing those token IDs into the same backbone. We obtain one sequence of token embeddings per

section. So after this first step, just before the second step, what we have is multiple sequences of token embeddings,

where each token embedding contains the semantic content, which is language aligned because we keep the backbone frozen, but also it contains a bit of information about the position of the token in the sequence,

and also, due to the attention mechanism, some of the section context. But we are processing sections at that point, and what we want to do at

the end is process documents. So we could do a simple concatenation of all those sections, uh, but we will be losing one piece of information:

the information that we will be losing is the section of origin of each token. So what we did for that is we just

Added some section encoding for those who are familiar with it. It's a bit like positional encoding, but instead of

encoding the position in the sequence, we encode the section in the document. Okay, so now we're getting happier and happier about it. Uh, we have

a unique sequence, and we are not losing the section of origin, but we are still talking about semantic

content. To move from the semantic world to our specific skill matching world, we, uh, we need to train ahead, and we also

use the skip connection. The skip connection is a well-known method to simplify training, and in our case, since

we are keeping the backbone frozen, uh, you can see it as the head is actually learning what is the

difference between the output of the backbone and our specific domain. And each of the tokens also contains a bit of the document context

due to the attention mechanism. So now there is one last thing that we, uh, we need to do: we need to move

from sequence to a unique embedding. While we can just do pooling—we can just use a pooling operator—uh, often we use average

pooling, but in our case it's not well-suited, uh, because the description contains many more tokens than all other

sections. So if we use average pooling, we will just, uh, the final embedding—the final resulting embedding—will, uh,

contain mostly description content, and, uh, we—it will be unbalanced. So we, uh, what we do instead is we use a weighted

Jouanneau & Palyart - SLM-powered retrieval to scale freelancers matching at Malt

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

We aim to be an accessible, community-driven conference, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.