I'm not sure if you have already configured the groups yet in JCE, but there is one for 'default' and that is where you choose how the icons/buttons are arranged in the JCE that you see when you create or edit an article.
What you will look for is one that has two links of a chain (linked together) with a green circle w/ a + inside of it - click on that and it will open a lightbox-ish thingie popup for you to enter your link into - you can also specify within it where you want the link to open (parent, new window/tab, etc.), which you can also do within the JCE component configuration, if you add a lot of links like me that you all want them to be uniform... just to save myself some precious time.
Hope that helps~