I have tried running the following command to expand the heap size but it does not work.
The way I found is to use XLConnect library.
Instead of using write.xlsx(myDataset, file = "myfile.xlsx") from the xlsx library, I used library(XLConnect).
It has a function to clear the memory, xlcFreeMemory().
If all else fails, try writing to CSV format then convert into xlsx from Excel.