Announcement

Collapse
No announcement yet.

Help Using Pointers

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Help Using Pointers

    I have a routine that reads data from a disk file from one file
    8 bytes at a time into a quad variable. It then encrypts the
    quad variable and then writes it to a different file. I would
    like to speed the process by reading the data 32000 bytes at a
    time and then process through the data using pointers, only I
    can't get any code to work.

    Does anyone have an experience with this or better yet a piece
    of example code demonstrating something like this. I can read
    and write the data into strings using GET# and Put# on a file
    OPEN in binary mode. The problem is I haven't found a way of
    getting real quad data variables out of the strings to
    manipulate.

    ------------------
    Thanks,

    Doug Gamble
    [email protected]

  • #2
    Disclaimer: I'm writing this online without having a compiler handy to test this with.

    Code:
    'Psuedo code
    DIM pQ1 as QUAD PTR, pQ2 as QUAD PTR
    OPEN "FILE.TXT" FOR BINARY AS #1
    FOR x& = 1 TO LOF(1) STEP 32000
      GET$ #1, 32000, a$
      b$ = a$         ' create output buffer of same length
      pQ1 = STRPTR(a$) ' use STRPTR32() with PB/DOS.
      pQ2 = STRPTR(b$)
      FOR y& = 1 to (LEN(a$) \ 8)    ' for example purposes... optimize for efficiency
        CALL MyEncryptFunction(@pQ1) ' function takes a BYREF QUAD 
        @pQ2 = @pQ1
        INCR @pQ1
        INCR @pQ2
      NEXT y&
      CALL WriteEncryptedString(b$)
    NEXT x&
    This should give you the idea. For most efficiency, dont place the encryption code in a sub/function, but make it LOCAL within the same sub/function as this code, then use GOSUB to call it. Alternatively, place the encryption code directly in the FOR/NEXT loop.

    Another variation is to use Indexed Pointers:

    Code:
    'Psuedo code
    DIM pQ1 as QUAD PTR, pQ2 as QUAD PTR
    OPEN "FILE.TXT" FOR BINARY AS #1
    FOR x& = 1 TO LOF(1) STEP 32000
      GET$ #1, 32000, a$
      b$ = a$         ' create output buffer of same length
      pQ1 = STRPTR(a$) ' use STRPTR32() with PB/DOS.
      pQ2 = STRPTR(b$)
      FOR y& = 0 to (LEN(a$) \ 8) - 1   ' for example purposes... optimize for efficiency
        CALL MyEncryptFunction(@pQ1[y&])   ' function takes a BYREF QUAD 
        @pQ2[y&] = @pQ1[y&]
      NEXT y&
      CALL WriteEncryptedString(b$)
    NEXT x&
    I hope this helps!

    PS: If you are using PB/CC or PB/DLL (32-bit) then you can read the file in much larger chunks, say 1Mb or more. Personally, I would use 8192 byte chunks as a tradeoff for speed VS network traffic generation (large cunks tend to swamp a network more than smaller packets... 8192 bytes is a good "rule of thumb" chunk size for network based apps).

    ------------------
    Lance
    PowerBASIC Support
    mailto:[email protected][email protected]</A>
    Lance
    mailto:[email protected]

    Comment


    • #3
      Doug,

      Unless you have good reason to encrypt 8 bytes at a time, there are much
      more powerful and much faster ways of encrypting information at a file
      level, the size limit will only be dictated by available memory.

      The sub below is a rotating XOR type where you supply the string to be
      encrypted (bString$) and a key string to encrypt it with (KeyString$). I
      posted a variation of this algo some time ago

      KeyString$ can be any length and the longer and more varied the better, an
      image file is a very good source of key string and if its very long, it
      will break the heart of any brute force attack.

      The advantage of a very large key is that there are no repeat cycles on
      small blocks that can be easily cracked, there is no problem with using a
      key that is even larger than the file being encrypted.

      What you need to avoid is a short repeat of the key string as the data can
      be broken down to a short sample and cracked that way. If you use a key
      over a couple of K in size, this becomes a lot more difficult. The SUB
      below will use any length key and when it gets to the end, it loops and
      starts again.

      It is used as follows,

      XorString MyFile$, MyKey$ ' encrypt file

      ' MyFile$ is now encrypted.

      XorString MyFile$, MyKey$ ' decrypt file

      ' MyFile is now decrypted

      This one should be fast enough to blow the hubcaps of most encryption
      routines and powerful enough to break the heart of most forms of attack.

      Hope its useful to you.

      Regards,

      [email protected]

      Code:
      ' ###########################################################################
        
        SUB XorString(bString$,KeyString$)
        
            #REGISTER NONE
        
            LOCAL ln   as LONG      ' source length
            LOCAL lkey as LONG      ' key length
            LOCAL lref as LONG      ' counter reference for key char position
            LOCAL src  as LONG
            LOCAL lpBt as LONG
            LOCAL pcnt as LONG
            LOCAL bvar as BYTE
        
            ! push ebx
            ! push esi
            ! push edi
        
            ln   = len(bString$)
            lkey = len(KeyString$)
        
            lpBt = StrPtr(KeyString$)   ' get starting address of key string
            pcnt = lpBt                 ' copy it to another variable
            lref = pcnt + lkey
        
            ! mov esi, lpBt
            ! mov al,[esi]
            ! mov bvar, al      ; put 1st byte in bvar
        
            src = StrPtr(bString$)
        
            ! mov ecx, ln
            ! mov esi, src
            ! mov edi, src
        
          xsSt:
            ! mov al,[esi]      ; copy 1st byte of source into al
            ! inc esi
            
            ! xor al, bvar      ; xor al with next byte in bvar
        
            ! push eax
            ! push esi
        
          ' ====== This code gets the next byte in the key string ======
          '        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            ! inc pcnt          ; increment byte address
            ! mov esi, pcnt     ; put it in esi
            ! mov al,[esi]
            ! inc esi
        
            ! mov ebx, lref
            ! cmp pcnt, ebx
            ! jne xsNxt
        
            ! mov edx, lpBt
            ! mov pcnt, edx     ; reset pcnt to ariginal address
            ! mov esi, pcnt     ; put original address in esi
        
            ! mov al,[esi]
            ! inc esi
        
          xsNxt:
            ! mov bvar, al
          ' ============================================================
        
            ! pop esi
            ! pop eax
        
            ! mov [edi], al
            ! inc edi
        
            ! dec ecx
            ! cmp ecx, 0
            ! jne xsSt
        
            ! pop edi
            ! pop esi
            ! pop ebx
        
        END SUB
        
      ' ###########################################################################
      ------------------
      hutch at movsd dot com
      The MASM Forum

      www.masm32.com

      Comment


      • #4
        Thanks for the feed back. I'm using a form of the Blowfish
        encryption algorhythm which is much more secure than an XOR
        method but thanks for the suggestion. I'll try the code for
        using pointers and see what comes of it.

        ------------------
        Thanks,

        Doug Gamble
        [email protected]

        Comment


        • #5
          You can try this method.

          Code:
              OPEN "FILE.TXT" FOR BINARY AS #1
              X& = LOF(1)
              B$ = SPACE$(X&)
              GET #1,, B$
              CLOSE #1
              X& = X& \ 8 
              DIM Pq(X&) AS QUAD AT STRPTR(B$)
              FOR i = 1 TO X&
                 CALL MyEncryptFunction(Pq(i)) 
              NEXT
          HTH




          ------------------

          Comment


          • #6
            Originally posted by Steve Hutchesson:
            The sub below is a rotating XOR type where you supply the string to be
            encrypted (bString$) and a key string to encrypt it with (KeyString$). I
            posted a variation of this algo some time ago

            KeyString$ can be any length and the longer and more varied the better, an
            image file is a very good source of key string and if its very long, it
            will break the heart of any brute force attack.
            As a matter of common sense, this would seem to be a fast and secure technique.
            Unfortunately, when it comes to mathematics, common sense is not always a very
            accurate guide. This type of XOR technique has a number of severe weaknesses
            that make it a poor choice for secure encryption. It's also particularly
            sensitive to choice of passwords. It would be inadvisable to use a text
            password at all, here: you'd want to run it through a secure hash first, at
            least.

            I'd recommend a web search on "Bruce Schneier" (and reading any of his
            excellent books) for more background on encryption.

            ------------------
            Tom Hanlin
            PowerBASIC Staff

            Comment


            • #7
              So far I have had the best success reading data in 4096 byte
              batches instead of 32000. This may have something to do with
              the 4096 cluster size on my NTFS partition or something about
              the encryption algorythm.

              Thakns to everyone for the feedback...I'll let everyone know
              what I find out regarding maximizing the file I/O.

              ------------------
              Thanks,

              Doug Gamble
              [email protected]

              Comment


              • #8
                Doug,

                The actual encryption technique is not where the action is, you can have
                rotations, displacements, XORs etc... , its the key size and repeat of
                that key that make it weak or strong. Your problem will be that an 8 byte
                long chunk is easy to break by brute force technique which is the normal
                type of attack on an algo of that type.

                56 bit DES was broken publically over 2 years ago and some of the cracking
                groups have already broken 512 bit keys so if the data is high security,
                you will need a stronger method.

                The technique I suggested to you differs in so much as instead of using a
                64 bit key, it can use ANY size key so if you are encrypting megabyte size
                files, you can easily use a megabyte size key to encrypt it that has no
                repeats in it at all. Put simply, a megabyte key is many magnitudes more
                difficult to attack than a 64 bit key.

                A simple XOR on one byte can be broken in 256 attempts, thats why the algo
                uses a large key file so that there is no sequence that can be discovered.
                The algo XORs each byte with a different byte from the key string that
                cannot be predicted as it is a seperate file not contained in the source.

                The largest single weakness in any encryption method is the security of the
                key, no method can protect from leaked key data.

                Regards,

                [email protected]


                ------------------
                hutch at movsd dot com
                The MASM Forum

                www.masm32.com

                Comment


                • #9
                  Tom has an interesting point here, plaintext keys are dangerous for a
                  couple of reasons, they have a character distribution of about 70 or less,
                  (upper & lower case, numbers and some punctuation) which is a lot less
                  than the 256 possible, the other is that it leaves the data open to what
                  is called dictionary attack on the password.

                  Make an encryption system that will use short plain text passwords and you
                  will get "Bill", Fred" and other easy to remember words. A dictionary
                  attack lists a large range of common words, names and just brute force
                  tests them.

                  My own preference is to use a binary file with a simple one step
                  encryption process, a bitmap of a meg or so in 24 bit colour is well out
                  of the range of dictionary attacks and it does not have the depleted
                  character range of a plain text password.

                  Tom's suggestion of pre-processing the plain text password with one of the
                  standard hash algorithms solves the problem of character distribution so
                  if the password/phrase is large enough, the password is much harder to
                  attack.

                  For seriously secure data, I prefer multiple processed hybrids using very
                  large keys. Writing the individual algos in asm makes them fast enough to
                  increase the complexity to time ratio in this manner.

                  Code breaking is an interesting area, especially as modern computers can
                  be set up to test massive numbers of combinations in a relatively short
                  time. Relying on sheer quantity is becoming less effective as modern
                  computers become more powerful. From memory, the publically set up system
                  that broke DES 56 bit was testing some billions of keys per second and it
                  was by no means a smart method, sheer brute force can deliver results if
                  the combinations can be calculated in mathematical terms.

                  Where there is no smart method is in the protection of the password, you
                  may have a system that can wear out an array of Cray supercomputers but
                  if someone has a method of getting at the actual password/key text, it is
                  wasted.

                  Regards,

                  [email protected]

                  ------------------
                  hutch at movsd dot com
                  The MASM Forum

                  www.masm32.com

                  Comment

                  Working...
                  X