Date: Saturday, April 8, 1995 From: mheiskan@hut.fi [Mika Heiskanen] Re: Re: Card slot #2 memory question! [Note: Somebody in France posted an explanation of why some programs crash in high ports of the GX. It was poorly written and very difficult to understand. Mika came to the rescue. -jkh-] >How to fix the problem : > >First : Never use, in an LM program the Garbage Collector or every subroutines >who made it . For example : don't use : >GOSBVL 0613E % Garbage collector >GOSBVL 05B7D % Res String >GOSBVL 039BE % Res Room Translation from French: =GARBAGECOL EQU #0613E =MAKE$N EQU #05B7D =GETTEMP EQU #039BE >Second : Instead of doing the Garbage Collector in the ML program >do : >PROG (02D9D) >$ 5F42 @ Force the Garbage collector >Here put your ML program >END (0312B) Translation from French: :: GARBAGE CODE ENDCODE ; >In your Program, to make a string or memory do : >To reserve String: Instead of using 05B7D do > >ST=1 10 >GOSBVL 05B80 > >To reserve memory : Instead of using 039BE > >ST=1 10 <-------+ >GOSBVL 039C1 | > | >Explanation : The ST=1 10 before calling these subroutines force the HP48 | >to make an Insufficient Memory error instead of doing a Garbage Collector. | > | > | >Now, Why Programs don't work in port 2 ?? | > | >Simply because they don't use this ----------------------------------------- Translation from French: Gibberish. The status flag merely indicates a second attempt at allocation (after GC) so that the program knows not to try GC again. It has nothing to do with succesful allocation. >Further explanation : The Garbage Collector's subroutine is bugged... >When you run a program from port 2 , the 48gx copy the program into >temporary object's memory >( between #80000 and #C0000 or between #80000 and #FFFFF ) Translation from French: GC does not update objects referenced by the machine language return stack since by the design of the GC algorithm any reference must point to a location in memory from which continous SKIPOB calls will lead to the end of the tempob memory slot where the referenced object resides. The addresses are meaningless, the only thing that matters is that when you run a program from a covered GX port the program will be copied to temporary memory area for execution. By definition any object in temporary memory is subject to removal if not referenced, which by convention cannot be done bye machine language addresses as explained above. >When a memory problem is encountered (whith 05B7D for example ) the hp48 >made what we call "Garbage Collector", it remove all the objects witch are not >referenced by the hp48 ( a program is referenced when it's in the Stack, in last >arg's memory etc...) ... >So, when the HP48 does a Garbage Collector , it removes the program wich is >running !!!! So , it losts the ports 2 program .. Is it clear ????? Translation from French: A running machine language program in temporary memory area may be lost when GC occurs, for example due to a data allocation call. Now, so that this article would actually contain any useful information, there are 2 good ways of doing machine language allocation calls safely from machine language: RPL GC before code ^^^^^^^^^^^^^^^^^^ :: GARBAGE CODE ENDCODE ; Above is safe because 1) There are no unreferenced objects in tempob, including the code itself which is referenced by the RPL interpreter pointer (D0 contains the address of SEMI) 2) Only one data allocation call is done, thus the fact that it is unreferenced is meaningless since GC doesn't have a chance to occur again during the execution of the machine language. Notes: 1) You can actually do several data allocation calls just as long as you reference them properly when needed. Pushing the newly created objects to data stack is a simple way to ensure this. 2) GC is slow, thus above is not suitable when above program is called often and speed is required. Thus above is best suitable when only a single call is done. Typical application would be in games, in fact several of the Frencg games would perfectly from GX port 2 if you would just add the missing GARBAGE call in RPL. 3) You should not pop any objects from stack before the allocation because it may cause an unreferenced object to appear in tempob. Better do SAVPTR first, then possibly use POP# etc, then finally at exit to RPL drop the arguments to the code object. RPL GC in code ^^^^^^^^^^^^^^ CODE GOSUB PassGC CON(5) =DOCOL Start RPL CON(5) =GARBAGE GC CON(5) =COLA Drop current stream CON(5) =DOCODE Start code again REL(5) EndOfCodeObject PassGC C=RSTK Code to execute the RPL GOVLNG =GETPTREVALC (EQU #15C77) EndOfCodeObject ENDCODE Above is safe because: 1) During GC the object is referenced by the RPL interpreter pointer 2) The interpreter pointer will lead to the end of the object when SKIPOB is called since the size field of the embedded code object points to the end of the actual object, thus GC will not crash due to an invalid pointer. 3) No data allocation call is done before checking for sufficient memory, this can be done directly with CREATETEMP or with proper calculations with any type allocation. 4) The RPL COLA instruction will terminate the RPL stream started by executing DOCOL, thus nothing beyond the code object will be executed. Note that you could remove the COLA provided that you would put SEMI between EndOfCodeObject and ENDCODE to terminate RPL, however using COLA makes the trick compact and thus less prone to errors just because you forgot to add the SEMI at a possibly distant source location. Notes: 1) GC is done only when absolutely necessary, thus there are no unnecessary slowdowns. Thus the method suits better when speed is required and the extra memory required and by the memory tests is small compared to the speed benefits. 2) You can do several data allocation calls with the same GC trick. 3) Machine language return stack is still not updated, thus the trick cannot be used from a machine language subroutine. Since the latter trick might seem a bit confusing, below is an example from BZ uncompressor which uses the trick for speed. On the other hand the compressor uses the first trick since it need to do GC to have all available memory for the data tables. (Using the second trick would be possible but there is no point since it would just take more memory.) CODE ST=0 sGARB Flag no GC yet * Restart entry after possible GC restart GOSBVL =SAVPTR A=DAT1 A Address of compressed string D1=A D1=D1+ 10 Skip $ prolog & size field D1=D1+ 4 Skip BZ identifier (actually BZ tests it) C=DAT1 A Read size of the uncompressed object RSTK=C Save it GOSBVL =CREATETEMP Try to allocate it GONC success * Didn't succeed in allocation, try with GC if allowed ?ST=1 sGARB 2nd pass already? GOYES memerr Yes - error GOSUB PassGC CON(5) =DOCOL CON(5) =GARBAGE CON(5) =COLA CON(5) =DOCODE REL(5) BZend ST=1 sGARB Flag GC has been done now GOTO restart Restart. PassGC C=RSTK GOVLNG =GETPTREVALC Exit to RPL at address C[A] * Note that restart includes reading all addresses again. Sizes etc stored * in scratch registers would remain valid over the RPL GC though. * Now we have a succesfull allocation so uncompress can start. success C=RSTK AD0EX R0=A ->target object address A=A+C A R1=A ->target object end address ... ... the rest is just usual uncompression code Now if you studied above trick you may realize why some programs don't work from GX covered ports. Typical reasons are: 1) The author did not know that GC is not safe from GX port 2. I would count most French games in this category. 2) The complexities caused by the tricks are such that they shadow the point of the program / waste too much memory. Typical examples are hacking libraries. 3) The program handles tempob directly and thus it might have to move itself. This is not usually feasible. Best example is Jazz library which for speed handles tempob directly. To see the alternative consider a stack display program which would have to force GC before displaying each line on the stack.. 4) The program is compiled to a fixed address. Fixed addresses won't work since the code is copied to tempob for evaluation. <-RPL-> is an old example. Finally note that GC may arise from pretty harmless looking calls and thus the programmer may simply have overlooked it. Just compare the two programs below and see if you have understood from above which one may cause a crash and why (although the chances are small). ( Program1: negate # ) ( Program 2: negate # ) CODE CODE GOSBVL =POP# GOSBVL =POP# GOSBVL =SAVPTR GOSBVL =SAVPTR A=-A A A=-A A GOVLNG =PUSH#ALOOP R0=A ENDCODE GOSBVL =PUSH# LOOP ENDCODE For those interested in GC, here is a clip from an email from HP: ---------------------------------------------------------------------- Here's a brief overview of GARBAGECOL history. We originally investigated two methods and a variant: a. mark-and-sweep b. copying c. both of these with generational extensions While copying is clearly superior in speed/efficiency, the 2K total RAM of the 28C made it utterly impractical. We incorporated some of the generational techniques into the overall structure of RAM (no updateable pointers in composites, no GC in USEROB, etc.) and further investigated incremental GC versions, but they never got to the point of paying back the overhead. We concentrated next on what we determined was the critical quantity distinguishing between the two natural loop orderings for mark-and-sweep (that is, "For I in {obs in TEMPOB} For P in {object ptrs} ..." vs "For P in {object ptrs} For I in {objects pointed to by P}...") based on actual usage statistics. This resulted in a revised, faster, but somewhat more fragile GC algorithm which hasn't changed in some time. ---------------------------------------------------------------------- Summary: faster, more fragile -- --- --> Mika Heiskanen mheiskan@gamma.hut.fi