What you need to do is find the sqldeveloper.conf file. You can just do a search on your system, though keep in mind that some systems (like Mac OS) consider this a system file and don't expose it in the GUI. You may need to use a command line to run your search.
On a Mac you'll likely find the file here: /Applications/SQLDeveloper.app/Contents/Resources/sqldeveloper/sqldeveloper/bin/sqldeveloper.conf
Just open a terminal, use your regular UNIX editor and open the file. Add this line to the file:
If you find you still can't open your file and that you have more resources available, increase that number until the file opens. Do pay attention to system resources though and leave some memory available for the OS and other apps.