Hi Sean, I have a 5kw 12 volt powerjack at my shed. It has problems too with large loads, it definitely will not run 5kw loads. It will not start my air compressor at all but will run all my other power tools. Oztules pointed out that in trying to start the compressor (1.5kw, which would draw around10kw to start) it would be trying to pull 2000 amps from the batteries. Not much chance.
My solution was to get a 12 volt air compressor designed for pumping 4wd tyres up. I then plug it into my compressors air tank and then presto I have a 12 volt portable compressor for my air tools.
On the issue of cooling the transformer. Depends on you application. You could always do something really radical and mount the transformer outside the powerjack case.
It could be mounted in a position where air flow could be improved such as hung vertically , or better still in a tank of transformer oil.
I do suspect that your batteries will not last long if you keep pulling massive loads though.
The only other way is to change to a 24, or 48 volt battery configuration and get another powerjack for the big loads.
Others will have some more ideas I am sure.
Cheerio
Pete
Oh yep I run a powerstar w7 at the house but with a inductor as per Oztules design. That limits the standby current to around 3.5 amps. I only turn it on when I want it. I have 12volt lighting, and a 1 volt refrigerator.