Newton applications in general use a small amount of heap space, even when they are not running (although considerably more when running, of course). It depends on the application, but various things must be kept in the heap: RemoveScripts, which can run when the application is removed, otherwise would not be able to execute when the card is pulled from the system (without generating the "Newton Still Needs the Card" message). Heap space in this case is not "wasted" - it is used as intended.
Applications that are well-behaved should set to NIL as many references to heap objects as they can when their application is closed; that will help conserve heap space. But users also want applications to open quickly. This can require the tradeoff of keeping around some context, so that the application doesn't have to rebuild or reload from soup everything it needs to run each time it is opened. Soup access is relatively slow compared to slot access, something like the difference between memory and disk.
In this case the particular heap I'm talking about is the NewtonScript heap. (There are other heaps in the system.) The NewtonScript heap holds the run-time data structures of the NewtonScript "world." (There are other "worlds" in the system.) This "world" doesn't exist for one application in specific, but for all the applications, including the built-in ones, where most of the application is in real ROM, and packages you install, where most of the application is in pseudo-ROM. It doesn't go away when you put your Newton to sleep. You can't compare the NewtonScript heap to the system heap or individual application heaps on the Mac -- the Newton is a very different animal.
Remember, on the Newton, battery life is king, and every cycle that the Newton is not asleep uses battery power. You don't want to go through the redundant loads, execution of install and startup code, and saves to flash or a RAM card every time you launch or close an application. Writing to flash RAM is especially expensive to battery life (and slower than RAM).
Also, applications don't modify themselves. That's a safety no-no. They're in pseudo-ROM. Anything they need to store permanently has to be in a soup (or in a slot in the root view, I believe, but I won't go into that now.)
A Newton application is not a file that is read into memory and launched when it is run. The situation is more complicated than that. The NewtonScript heap is a dynamic, garbage-collected memory environment. The language was actually designed from the ground up to allow minimal use of real memory. The complaint that some portion of that real memory is devoted to an application without open views is valid, I suppose, and heap is a precious resource, but (in well-written applications at least), this heap space isn't used without reason.