BNP Captcha authentication: a reverse engineering case

I wanted to retrieve my data from my bank to build an app. That app would help me to manage my budget and monitor what’s going on. Unfortunately, BNP does not provide an API and there’s no Yodlee (service used by Mint) for French banks.

So here is what I did with Ruby (♥), Selenium and Image Magick.

Step one: How does the authentication works

First, let’s see how BNP’s authentication works. You have to fill a form with your ID and click with your mouse on the right numbers and then click a validate button. Accessibility: Zero !

BNP authentication image

To “secure” their authentication, they provide a new image, rearranging the numbers, every time a GET request is sent to the image URL. So to by-pass the authentication, everything had to be done with only one request.

Step two: Let’s find the numbers

If you inspect the BNP’s HTML code, you will find the key you expected. A HTML map is provided with the image and it looks like this:

<map name="MapGril">
 <area onclick="Javascript:Grille('01')"  shape="rect" coords="5,5,27,26">
 <area onclick="Javascript:Grille('02')"  shape="rect" coords="32,5,54,26">
 <area onclick="Javascript:Grille('03')"  shape="rect" coords="59,5,81,26">
 <area onclick="Javascript:Grille('04')"  shape="rect" coords="86,5,108,26">
 <area onclick="Javascript:Grille('05')"  shape="rect" coords="113,5,135,26">
 <area onclick="Javascript:Grille('06')"  shape="rect" coords="5,32,27,53">


 <area onclick="Javascript:Grille('25')" shape="rect" coords="113,113,135,134">

No need to be a genius to understand that what is sent to the server is not the code but the coordinates of your numbers composing your password.

We always have a 5x5 grid. So if we could make our script to click to the right coordinates, we’re all set !

Step 3: Image magick and Ruby for the eyes

Image Magick offer a bunch of wonderful filters. We only need to reduce the noise of the image in order to recognize the numbers. An easy way is to transform the image in black and white in order have the numbers in black.

pixels, blacks = [], []
img.each_pixel do|pixel, c, r|
  blacks.push({c: c, r: r}) ifcolor=='#000000'

Every number will have a unique path with black pixels coordinates. in order to know those coordinates, I drew those path in a shell with 0 and 1 chars.

"1111111111111111111111111111111111111111111111111111111111111111111111" "1111111111111111111111111111111111111111111111111111111111111111111111" "1111111100111111111111111111111111101111111111111111111111111111111111" "1111111011011111111111111111111111010011111111111111111111111110011111" "1111110111001111111111111111111111111011111111111111111111111110011111" "1111111111011111111111111111111111111011111111111111111111111111011111" "1111111111011111111111111111111111110111111111111111111111111011011111" "1111111110111111111111111111111111111011111111111111111111110111011111" "1111111101111111111111111111111111111011111111111111111111110111011111" "1111111011111111111111111111111111111001111111111111111111110000001111" "1111110000001111111111111111111110011011111111111111111111111111011111" "1111110000001111111111111111111111100111111111111111111111111111011111" "1111111111111111111111111111111111111111111111111111111111111111111111"

Since we can read with our eyes the numbers, so can the computer. We can extract from this ASCII art that for example the number 2 can be defined with a path like this:

if blacks.include?({r: r, c: c}) &amp;&amp;
  blacks.include?({r: r+1, c: c}) &amp;&amp;
  blacks.include?({r: r, c: c+1}) &amp;&amp;
  blacks.include?({r: r+1, c: c+1}) &amp;&amp;
  blacks.include?({r: r, c: c+2}) &amp;&amp;
  blacks.include?({r: r+1, c: c+2}) &amp;&amp;
  blacks.include?({r: r, c: c+4}) &amp;&amp;
  blacks.include?({r: r+1, c: c+4})
  return cr 

Once we know where are the numbers, with the HTML map, we can check in which case the number is located then we have the case number where to click. With our example, we have the result of : [25, 3, 7, 8, 9, 12, 14, 21, 22, 24]

0 is in the case 25.
1 is in the case 3
2 is in the case 7
etc …

Step 4: Selenium and Ruby for the hand !

Now we’ve got the cases, a minimal selenium script is enough to click for you.

image =

account = driver.find_element(:name, 'ch1')
account.send_keys ENV['BNP_ACCOUNT']

(1..25).each do|area|
  where = "//*[@id='corps']/div[2]/div/div[2]/div[2]"
  where = where + "/table/tbody/tr/td[1]/div/center/"
  where = where + "map/area[#{area}]"
  self.instance_variable_set("@num#{area}", driver.find_element(:xpath, ))

numbers = image_url


button = driver.find_element(:xpath, "//node()[@id='active'][1]/a[1]")

We crop the image first, it has a useless border. Then we type our ID. Now for each case of the map, we associate the selenium element to click on. After have decoded the cropped image and retrieved the numbers position, now we can click the numbers composing the BNP password. After that, you can retrieve what you need from your BNP account like a charm.

That’s it ! Happy hacking :)
You can find the code here


Many services like Bankin’, Linxo gives you the opportunity to do exactly this. You provide your bank account password and then, they parse your bank data, rearrange it and gives you beautiful graphs. I don’t think they have partnerships with the banks because of that password they store.

For the user, that’s ugly and pretty dangerous! Even if they are certified to be Norton secured or whatever certification-like, the fact is they store your password and that’s very bad. That means if they’re compromised, chances are your bank account is not safe at all.

Currently, I don’t know any french bank service using a two-factor authentication, so this logic can be used on any bank using number grids. In order to avoid bad surprises, those company MUST conclude a partnership with the banks, and provide to customers a another password working for their platform.

Note: This article is for your information and I’m not responsible of what you could do with it. Thanks for reading !

Licence  CC BY-SA 4.0