If you have IE 6, you are out of luck (or need a plugin?). For IE 7 and IE 8, it's not so bad. Go to the Tools menu and select Internet Options. In the Internet Options popup, select the General tab, and push the Languages button. In the Languages Preference popup, push the Add... button and add the language you want (French in your case). You might need to restart IE after that, but then your domain name should show correctly.
In Firefox, type about:config in the address bar where you would usually type the URL. You will get a warning. If you proceed, you should see a long list of settings. Right-click anywhere on this list, and in the popup menu select New / Boolean. Then enter this Preference Name, with this exact capitalization: network.IDN.whitelist.com and then in the next box, set the value to "true". Your domain should now show up properly.
Be aware that there is a reason the browsers make this process complicated. There are a lot of non-ascii characters that look like ascii characters, and evil sites can use these look-alike characters to disguise themselves. So once you open up your browser to non-ascii urls, be careful.
For more IDN info, by the way, I recommend idnforums.com and idndemystified.com. I think you'll discover that IDNs are not a waste of money, once you learn to avoid beginners' mistakes.