Announcement

Collapse
No announcement yet.

Saving An Entire Array At Once ?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Michael Mattias
    replied
    Talk about memories....and relevant, too!
    Code:
    [72030,3563] Michael Mattias
    RPD.ZIP
      Bin,  Bytes:    21248, Count:   141, 22-Aug-95
    
      Title   : REDIM PRESERVE plus Virtual Array Memory for PB
      Keywords: ARRAY REDIM PRESERVE SAVE RESTORE SOURCE FREE 3.1
    
      Routines to allow PowerBASIC programmers to REDIM regular and
      HUGE fixed-element-size arrays (including TYPE arrays) with PRESERVE and
      use disk files as "virtual array memory." Includes Unit source, demo source
      and documentation. Requires PB 3.1+. Copyright but free. From Michael
      Mattias.
    There are some 'save and restore' array routine in here which ain't all bad...
    Attached Files

    Leave a comment:


  • Fred Buffington
    replied
    Coming From THEOS multi-user basic in the 80's, it was nice to see the addition of mat in the newer versions of pbwin and pbcc.

    THEOS not only allows mat for numeric but also string arrays and
    a MAT WRITE #file%[,RECORD KEY]:string_arrayname$ or numeric_arrayname

    This would write the array to a file starting with item 1.

    and then on the other side,
    MAT READ #file%[,RECORD KEY]:arrayname

    Leave a comment:


  • Michael Mattias
    replied
    Wow, what memories!

    VARSEG, STRSEG, DEF SEG, BLOAD, BSAVE,'$$' data types....

    Ah, they just don't make keywords like they used to......

    Leave a comment:


  • Joe Byrne
    replied
    Not sure how you're using the BSAVE command, nor how fast the following works with that large of an array, but I used this technique a lot back in my DOS days (even a time or two in Windows) and found it to be darn good for my needs. Sure glad I never delete any of these "old" tools
    Code:
    '                           ARRAYSAV.BAS
    '        ---------------------------------------           Eric J. Pearson
    '        ARRAY SAVE/LOAD ROUTINES FOR POWERBASIC 
    '        ---------------------------------------             August 1991
    '
    ' These sample programs demonstrate a method for BSAVEing and BLOADing
    ' numeric and flex string arrays (i.e. swapping data directly between memory
    ' and disk) with PowerBASIC.  This technique is much faster than saving and
    ' loading arrays as text files (i.e. creating data files with PRINT#), and
    ' takes less program overhead.  And because the files are undelimited (i.e.
    ' no CR/LF, commas, quotes, etc.) and because numeric values are stored in
    ' "compressed" form (compared to storing numbers as text) the data files
    ' take up much less disk space.  Finally, the technique allows numeric values
    ' to be saved/loaded directly, without the relatively slow task of translation
    ' to/from text.
    '
    ' Numeric variables of any type, and flex strings, in arrays with any number
    ' of dimensions, can be saved and loaded with routines based on these samples.
    ' (Sorry, but "normal" string arrays can't be BSAVEd or BLOADed.)
    '
    ' Generally speaking, a small "custom" sub is built for each array that is
    ' to be saved/loaded, and a "standard" sub handles the actual bload/bsave...
    '
    ' INITIALIZATION:
    
    	'define variable types (*REQUIRED* values)...
    	%IntegerVar =  2   'INTEGER (%)
    	%LongIntVar =  4   'LONG INTEGER (&)
    	%QuadIntVar =  8   'QUAD INTEGER (&&)
    	%SinglePrec =  4   'SINGLE PRECISION (!)
    	%DoublePrec =  8   'DOUBLE PRECISION (#)
    	%ExtPrec    = 10   'EXTENDED PRECISION (##)
    	%BCDFixed   =  8   'BCD FIXED POINT (@)
    	%BCDFloat   = 10   'BCD FLOATING POINT (@@)
    	%FlexString =  0   'FLEX STRING ($$)
    
    	'for convenience (arbitrary values)...
    	%Load       =  1
    	%Save       = -1
    
    ' DEMONSTRATION #1 (1-dimensional integer array):
    
    	'dimension the array
    	DIM STATIC Array1%(256)
    
    	'load demonstration data into the array
    	Array1%(1) = 10001
    
    	'save the array on disk
    	CALL ArrayOne(%Save)
    
    	'clear the demo value from the array
    	ERASE Array1%
    
    	'load the array from disk
    	CALL ArrayOne(%Load)
    
    	'display the demonstration data
    	PRINT Array1%(1)    'prints 10001 if everything worked right.
    
    ' DEMONSTRATION #2 (3-dimensional, double precision array):
    
    	DIM STATIC Array2#(1:64,0:10,0:2)
    	Array2#(2,2,2) = 20000000.2
    	CALL ArrayTwo(%Save)
    	ERASE Array2#
    	CALL ArrayTwo(%Load)
    	PRINT Array2#(2,2,2) 'prints 20000000.2 if everything worked right.
    
    ' DEMONSTRATION #3 (one-dimensional, flex string array):
    
    	DIM STATIC Array3$$(0:1024)  '(each element to be 8 characters)
    	'8 chars per element * 1025 elements = 8200 bytes, so...
    	MAP Big3$$ * 8200
    
    	'map the array into the big string
    	FOR Element% = 0 TO 1024
    		ElBase% = (Element%*8)+1
    		MAP Big3$$, FROM ElBase% TO ElBase%+7 AS Array3$$(Element%)
    	NEXT
    
    	'load demonstration data into the array
    	LSET Array3$$(16) = "TESTING!"
    
    	'save the array
    	CALL ArrayThree(%Save)
    
    	'clear the demonstration data
    	'(don't ERASE the array or the MAP will be lost)
    	LSET Array3$$(16) = "        "
    
    	'load the array
    	CALL ArrayThree(%Load)
    
    	PRINT Array3$$(16)  'prints "TESTING!" if everything worked right.
    
    	END
    
    SUB ArrayOne(Direction%)
    
    	'THIS IS A "CUSTOM" SUB, SET UP FOR A PARTICULAR ARRAY
    
    	SHARED Array1%()
    	LOCAL  Segment&,Start&,Finish&,VarType%
    
    	Segment& = VARSEG(Array1%(0))   'first element
    	Start&   = VARPTR(Array1%(0))   'first element
    	Finish&  = VARPTR(Array1%(256)) 'last element
    	VarType% = %IntegerVar
    	CALL LoadOrSave("Array1",Direction%,Segment&,Start&,Finish&,VarType%)
    
    END SUB
    
    
    SUB ArrayTwo(Direction%)
    
    	'THIS IS A "CUSTOM" SUB, SET UP FOR A PARTICULAR ARRAY
    
    	SHARED Array2#()
    	LOCAL  Segment&,Start&,Finish&,VarType%
    
    	Segment& = VARSEG(Array2#(1,0,0))     'first element
    	Start&   = VARPTR(Array2#(1,0,0))     'first element
    	Finish&  = VARPTR(Array2#(64,10,2))   'last element
    	VarType% = %DoublePrec
    	CALL LoadOrSave("Array2",Direction%,Segment&,Start&,Finish&,VarType%)
    
    END SUB
    
    
    SUB ArrayThree(Direction%)
    
    	'THIS IS A "CUSTOM" SUB, SET UP FOR A PARTICULAR ARRAY
    
    	SHARED Big3$$  'note that Array3$$() itself is not shared
    	LOCAL  Segment&,Start&,Finish&,VarType%
    
    	'(note use of STRSEG and STRPTR, not VARSEG and VARPTR as above)
    	Segment& = STRSEG(Big3$$)          'first byte of big string
    	Start&   = STRPTR(Big3$$)          'first byte of big string
    	Finish&  = Start& + LEN(Big3$$)    'last byte of big string
    	VarType% = %FlexString             '
    
    	CALL LoadOrSave("Array3",Direction%,Segment&,Start&,Finish&,VarType%)
    
    END SUB
    
    
    SUB LoadOrSave(FileName$,WhichWay%,SegmentAdr&,StartAdr&,FinishAdr&,WordLen%)
    
    	'THIS "STANDARD" ROUTINE IS USED BY ALL "CUSTOM" SUBS ABOVE
    
    	LOCAL Length&
    
    	FileName$ = FileName$ + ".ARR"  'MY DEFAULT EXTENSION FOR ARRAY FILES
    
    	Length&  = FinishAdr& - StartAdr& + WordLen%  'ARRAY LENGTH IN BYTES
    
    	DEF SEG = SegmentAdr&
    
    	IF WhichWay% = %Save THEN
    		BSAVE FileName$,StartAdr&,Length&
    	ELSE
    		BLOAD FileName$,StartAdr&
    	END IF
    
    	DEF SEG
    
    END SUB
    
    '------------------------------------------------------------- end of ARRAY.BAS

    Leave a comment:


  • Donald Darden
    replied
    Intel imposed the 64k addressing boundary on their early CPU architectures, because somebody decided that nobody would ever need more than 1 MB or RAM, ever. So instead of having the segment addresses do a full offset from the addressing range allowed with 16 bits, they caused an overlap where only four addressing bits were actually appended. Some old code used some carefully crafted segment+address math to obscure jump locations or table lookup values, trying to protect their code from prying eyes.

    MSDOS merely followed the hardware design. Later, when they wanted to address more memory, they used tricks with address line 20 to map in a different memory chunk at the very top end of the memory range, and we saw new drivers under DOS to allow us to address expanded and extended memory. People never really got a handle on which was which.

    With Windows, Microsoft finally decided to take over managing memory and try to make it transparent to the user. The expansion of the original architecture design added instructions that could actually address a full 32-bits of memory range, and Windows was able to take advantage of this. DOS, however, was designed for the older architecture, and though it could run on the newer platforms, it still had the same limitations, because it still relied on the old 16-bit design and legacy addressing modes.

    So the original shortsightedness was probably due to some director over an engineering department with Intel who decided that 1 MB of RAM was all that was ever going to be needed, and saw this as a way to keep the chip die small and not require ever more complex pin arrangements to support all the addressing lines that would be needed otherwise. The advances in chip design and multilayer fabrication processes would eventually prove him wrong, but that does happen. But Bill Gates once thought
    that 64k was all the RAM that would ever be needed, so at least he was in good company.

    Every time an interrupt is generated, the first thing that usually happens is that further interrupts are inhibited immediately. Some very short processes or time critical processes may complete their tasks before they permit further interrupts. Others, willing to give way to other possible interrupts, and may enable interrupts almost immediately, thus permitting some other process to preceed them if need be. You cannot tell which method is used by any one process without examining the code involved.

    Leave a comment:


  • Hans Ruegg
    replied
    do you know of an operating system like MSDOS that will handle all RAM as straight line memory
    It can be done in MSDOS itself, if you are careful. Do a Google search for "real mode flat memory". However, I never tried it, because I am not a very careful programmer... And it would not serve your purpose of saving an array of more than 64 KB, because for that you would still have to use the DOS services which are programmed for writing a maximum of 64KB(-1) at once.

    Leave a comment:


  • Michael Mattias
    replied
    it is possible to issue a command to stop all such interrupts and dedicate the machine fully to any task the programmer desires
    Yes, it is possible.

    See the Intel x86 (or whatever chip you are working with) programming reference for details.

    Leave a comment:


  • Michael Mattias
    replied
    But Windows/32 is kind of like MS-DOS, and that DOES use linear addressing within the entire 32-bit range from x000000 to xFFFFFFFF

    MCM

    Leave a comment:


  • Michael Mattias
    replied
    Memory should be straight line do you know of an operating system like MSDOS that will handle all RAM as straight line memory ..
    (Assuming by "straight line" you mean "linear addressing")

    If it did, then it wouldn't be like MS-DOS. MS-DOS uses a 20-bit "paragraphffset" address, which is only linear within each 64K block which starts from offset zero.

    Leave a comment:


  • Geoffrey Voeth
    replied
    Originally posted by Erich Schulman View Post
    It sounds like FreeDOS-32 is what you're looking for, other than it being far from ready for production use. And I am assuming Linux or *BSD would not meet your needs.
    For Goodness Sake that idea is three years old and no more progress is indicated it seems there is no serious effort for anyone to make an operating system with unlimited straight line memory.

    Leave a comment:


  • Erich Schulman
    replied
    Originally posted by Geoffrey Voeth View Post
    <====
    do you know of an operating system like MSDOS that will handle all RAM as straight line memory like in the old DIGITAL EQUIPMENT CORPORATION PDP1140 series industrial control computers ???
    It sounds like FreeDOS-32 is what you're looking for, other than it being far from ready for production use. And I am assuming Linux or *BSD would not meet your needs.

    Leave a comment:


  • Geoffrey Voeth
    replied
    Originally posted by Michael Mattias View Post
    Nothing is going to be faster than BSAVE.

    It is not worth the time to do it in assembly language... why not? Because BSAVE is compiled into what the PB folks think is prett-good assembly language for saving an array to disk

    You can't save any array > 64K under MS-DOS in one shot because MS-DOS can't handle more than 64k in one shot. (16 bit registers, you know). Whatever you do or the compiler does for you, be assured "somewhere" it's broken up into chunks of 64 K or less.

    FWIW..
    >BSAVE happened in less time then a TIMER tic

    If you need to save a whole array more than 18 times per second, you need some serious design assistance.MCM
    <====

    There just no need for such comments as that last line you injected the human wants and desires are unlimited even if unreasonable.
    Other than that your comments seem reasonable.
    Memory should be straight line do you know of an operating system like MSDOS that will handle all RAM as straight line memory like in the old DIGITAL EQUIPMENT CORPORATION PDP1140 series industrial control computers ???

    There exists the possibility that the BSAVE routine first disables all interrupts prior to executing its main routine and that is why no tic counter tics are counted ?? It is my understanding that the CPU is resposibl through interrupts to maintain the seconds after midnight but that it is possible to issue a command to stop all such interrupts and dedicate the machine fully to any task the programmer desires.
    Last edited by Geoffrey Voeth; 9 May 2008, 01:16 PM.

    Leave a comment:


  • Michael Mattias
    replied
    Nothing is going to be faster than BSAVE.

    It is not worth the time to do it in assembly language... why not? Because BSAVE is compiled into what the PB folks think is prett-good assembly language for saving an array to disk

    You can't save any array > 64K under MS-DOS in one shot because MS-DOS can't handle more than 64k in one shot. (16 bit registers, you know). Whatever you do or the compiler does for you, be assured "somewhere" it's broken up into chunks of 64 K or less.

    FWIW..
    >BSAVE happened in less time then a TIMER tic

    If you need to save a whole array more than 18 times per second, you need some serious design assistance.

    MCM

    Leave a comment:


  • Geoffrey Voeth
    started a topic Saving An Entire Array At Once ?

    Saving An Entire Array At Once ?

    Hello Folks, I am trying to save array DATA_1?(0:32770) as fast as possible to file "a.dat" and would like to hear feedback in how to do this. So far I have found the BSAVE command the very best but think there might be another way such as using assembly language for arrays bigger than 65536 bytes.
    BSAVE happened in less time then a TIMER tic but all other ways involving opening files took fraction of seconds.

    IS there any way to save an entire VIRTUAL array with a single command and not using contiguous loops ?
Working...
X