Inline::Java has an EXTRA_JAVA_ARGS option you may mention when doing "use Inline". There you can supply arguments for the JVM start and here the -Xmx option limits the heap memory the JVM has available. For an example have a look into the perldoc of Inline::Java and search for EXTRA_JAVA_ARGS. Note that the real memory footprint of the JVM will be 10-20% higher as you just limit heap memory but this is usually sufficient to keep the JVM's memory consumption under control.
Christian