I would tend to agree with RockyMtnRay, since he's talking mountain towing. You don't say where you'll be driving, but here in Michigan, the midwest and the east-coast, I find the Highlander to handle
just fine. Of course the 2619 does weigh a little less than your model. Make sure to tap the Overdrive disengage button when towing.
Rather than have second thoughts, put on a hitch and try it!
The trans-cooler is already there as well as most of the wiring. The Draw-Tite trailer hitch that is custom made for the Highlander is great, easy to install and in the worst case, you can use it for attaching a bike carrier. Don't skimp on the other components of a WDH though, you'll probably need it even if you change Tow vehicles.