Hi Julien,
Azerbaijani legislation does not quite differentiate foreigners from locals. That is why openning a bank account should not be a problem , if you are working/living here for a long time. All residents of Azerbaijan may open accounts. The legal definition of "resident" is "a person who stays in Azerbaijan more than 182 calendar days". You may approach the following banks along with your ID/passport and an employement reference, where you can find English-speaking front-desk personnel:
Bank Respublika
Access Bank
Bank Standard
Demir Bank
You can search these banks on google for their website and addresses.
I highly discourage you going to International Bank of Azerbaijan as they are very un-friendly and extremely bureaucratic.
Call me (pm) for help if you need any or need one to speak the local language.
Good luck !
Ramin