Announcement

Collapse
No announcement yet.

CryptAcquireContextA + This API is deprecated.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • CryptAcquireContextA + This API is deprecated.

    "Important This API is deprecated.
    New and existing software should start using Cryptography Next Generation APIs.
    Microsoft may remove this API in future releases."

    For years I have been using something like the code below to generate MD5 hashes from a string.

    Have any of you all switched to the Cryptography API: Next Generation?
    Should I actually worry about my old programs breaking in the future? (I know hard question to ask!)

    Doesn't look too difficult...
    https://docs.microsoft.com/en-us/win...-hash-with-cng



    Code:
    #COMPILE EXE
    #DIM ALL
    #INCLUDE "WIN32API.INC"
    
    
    GLOBAL hProv AS DWORD
    GLOBAL strHash AS STRING * 16
    GLOBAL strHashPtr AS BYTE PTR
    
    DECLARE FUNCTION CryptGetHashParamX LIB "AdvAPI32.dll" ALIAS "CryptGetHashParam" _
    (BYVAL hHash AS DWORD, BYVAL dwParam AS DWORD, BYVAL pbData AS BYTE PTR, _
    pdwDataLen AS DWORD, BYVAL dwFlags AS DWORD) AS LONG
    
    '------------------------------------------------------------------------------
    
    FUNCTION PBMAIN () AS LONG
    
    strHashPtr = VARPTR(strHash)
    
    LOCAL result AS LONG
    result = CryptAcquireContext(hProv, BYVAL %Null, BYVAL %Null, %PROV_RSA_AES, 0)
    
    
    
    REGISTER i AS LONG
    LOCAL hHash AS DWORD, hText AS STRING
    LOCAL strText AS STRING
    
    strText = "TEST_FILE_CONTENTS_STRING"
    
    CryptCreateHash( hProv, %CALG_MD5, 0, 0, hHash )
    CryptHashData( hHash, BYVAL STRPTR(strText), LEN(strText), 0 )
    CryptGetHashParamX( hHash, %HP_HASHVAL, strHashPtr, 16, 0 )
    CryptDestroyHash hHash
    
    FOR i = 0 TO 15
    hText += HEX$( @strHashPtr[i], 2 )
    NEXT
    
    MSGBOX hText
    
    END FUNCTION

  • #2
    A search fro BCryptCreateHash turned up this code from MIke Doty: (Note Post #3 following this code )

    https://forum.powerbasic.com/forum/u...868#post775868


    or this one?
    https://forum.powerbasic.com/forum/u...312#post789312

    Comment


    • #3
      Thanks Stuart!
      Great place to start.
      I should have Googled it like you did.

      Comment


      • #4
        Nope, I didn't Google it, I just used the forum Search box at the top of the page.

        I've stripped Mike's code in the second post down to just the Hash function as a demo:

        '
        Code:
        'Uses  Mike Doty's function at
        'https://forum.powerbasic.com/forum/user-to-user-discussions/programming/789154-encryption-question?p=789312#post789312
        #COMPILE EXE
        #DIM ALL
        #INCLUDE "WIN32API.INC"
        
        FUNCTION PBMAIN() AS LONG
        LOCAL s, h,sTxt,sKey AS STRING
        LOCAL i AS LONG
        
        sTxt = "This is some text that we want to produce a Hash for "
        sKey = "This is the secret key to Hash with"
        s = SHA256HMAC(sTxt,sKey)
        FOR i = 1 TO LEN(s)
        h += HEX$(ASC(s,i),2,1)
        NEXT
        ? h
        END FUNCTION
        
        FUNCTION SHA256HMAC( BYVAL sText AS STRING, BYVAL pbSecret AS STRING ) AS STRING ' Binary
        ' sText is message to be hashed  code by David Roberts
        ' pbSecret is secret HMAC key
        
          LOCAL hAlg, phHash AS DWORD
          LOCAL sBinHMAC AS STRING * 32 ' 32 for SHA256
        
          BCryptOpenAlgorithmProvider( hAlg, "SHA256"$$, $$NUL, %BCRYPT_ALG_HANDLE_HMAC_FLAG ) ' We need hAlg
          BCryptCreateHash( hAlg, phHash, BYVAL %Null, 0, STRPTR( pbSecret ), LEN( pbSecret ), 0 ) ' We need pbHash
          BCryptHashData( phHash, STRPTR( sText ), LEN( sText ), 0 )
          BCryptFinishHash( phHash, VARPTR( sBinHMAC ), 32, 0 ) '  32 for SHA256, We need sBinHMAC
          BCryptDestroyHash( phHash )
          BCryptCloseAlgorithmProvider( hAlg, 0 )
          FUNCTION = sBinHMAC
        
        END FUNCTION
        '

        Comment


        • #5
          I am wanting to come up with a unique hash for files, think MD5 is the best way to go?

          Comment


          • #6
            Do you need security that lasts more than seconds??
            Dale

            Comment


            • #7
              Dale, Actually I was trying to make a unique hash (MD5?) for each file on my network to look for duplicates - so not really a security thing in this case.

              Comment


              • #8
                What is "SHA256"$$, in BCryptOpenAlgorithmProvider( hAlg, "SHA256"$$, $$NUL, %BCRYPT_ALG_HANDLE_HMAC_FLAG ) ???

                When I try to define it as LOCAL SHA256 AS STRING or WSTRING it errors Parameter mismatches definition. Same if I add PTR.

                Microsoft says "A pointer to a null-terminated Unicode string that identifies the requested cryptographic algorithm. This can be one of the standard CNG Algorithm Identifiers or the identifier for another registered algorithm."

                SO, is "SHA256"$$ a constant? What data type?

                Just trying to figure out what is going on under the hood.


                Jose Includes has this:

                '//
                '// Common algorithm identifiers.
                '//

                $$BCRYPT_RSA_ALGORITHM = "RSA"$$
                $$BCRYPT_RSA_SIGN_ALGORITHM = "RSA_SIGN"$$
                $$BCRYPT_DH_ALGORITHM = "DH"$$
                $$BCRYPT_DSA_ALGORITHM = "DSA"$$
                $$BCRYPT_RC2_ALGORITHM = "RC2"$$
                $$BCRYPT_RC4_ALGORITHM = "RC4"$$
                $$BCRYPT_AES_ALGORITHM = "AES"$$
                $$BCRYPT_DES_ALGORITHM = "DES"$$
                $$BCRYPT_DESX_ALGORITHM = "DESX"$$
                $$BCRYPT_3DES_ALGORITHM = "3DES"$$
                $$BCRYPT_3DES_112_ALGORITHM = "3DES_112"$$
                $$BCRYPT_MD2_ALGORITHM = "MD2"$$
                $$BCRYPT_MD4_ALGORITHM = "MD4"$$
                $$BCRYPT_MD5_ALGORITHM = "MD5"$$
                $$BCRYPT_SHA1_ALGORITHM = "SHA1"$$
                $$BCRYPT_SHA256_ALGORITHM = "SHA256"$$
                $$BCRYPT_SHA384_ALGORITHM = "SHA384"$$
                $$BCRYPT_SHA512_ALGORITHM = "SHA512"$$
                $$BCRYPT_AES_GMAC_ALGORITHM = "AES-GMAC"$$
                $$BCRYPT_ECDSA_P256_ALGORITHM = "ECDSA_P256"$$
                $$BCRYPT_ECDSA_P384_ALGORITHM = "ECDSA_P384"$$
                $$BCRYPT_ECDSA_P521_ALGORITHM = "ECDSA_P521"$$
                $$BCRYPT_ECDH_P256_ALGORITHM = "ECDH_P256"$$
                $$BCRYPT_ECDH_P384_ALGORITHM = "ECDH_P384"$$
                $$BCRYPT_ECDH_P521_ALGORITHM = "ECDH_P521"$$
                $$BCRYPT_RNG_ALGORITHM = "RNG"$$
                $$BCRYPT_RNG_FIPS186_DSA_ALGORITHM = "FIPS186DSARNG"$$
                $$BCRYPT_RNG_DUAL_EC_ALGORITHM = "DUALECRNG"$$

                Comment


                • #9
                  Originally posted by David Clarke View Post
                  Dale, Actually I was trying to make a unique hash (MD5?) for each file on my network to look for duplicates - so not really a security thing in this case.
                  Then use it. You already are used to MD5. I would suggest getting an MD5 DLL, or source code you can compile yourself. Then your app will not die when a new version of Windows comes out.

                  SHA256 is a NIST secure hash. From your Microsoft quote it is a hash that takes Unicode 16 (ie WIDE) strings for input. (they seem to use $$ the same way PB does.) It would be time better spent than recoding your app for MD5. (even though you don't need the security.) The file names can/will be WIDE strings.

                  Your app! You pick!

                  Cheers,

                  Dale

                  Comment


                  • #10
                    Originally posted by David Clarke View Post
                    I am wanting to come up with a unique hash for files, think MD5 is the best way to go?
                    In the Forensics world we tend to avoid MD5 where possible and lean towards SHA1 or better SHA-2 (SHA256). I think MD5 will still work based on your need but a review of https://en.wikipedia.org/wiki/Secure_Hash_Algorithms might help.

                    Your use case is broken on collisions and I doubt you would ever see a collision with MD5. For that matter you might even try something less dense like FNV-1. It really depends on how many files you are hashing
                    <b>George W. Bleck</b>
                    <img src='http://www.blecktech.com/myemail.gif'>

                    Comment


                    • #11
                      Thanks George!
                      I will try them all! For this project accuracy is most important. I am on a 35+ year purge of old files. And I have a zillion duplicates, some with different file names!

                      Comment


                      • #12
                        If you decide to stick with MD5 then I've included some rather "mature" code that still seems to work on Win10 and PBCC 6. I suspect that there are some DLL's somewhere on the Internet but I've never gone looking for them. Personally, I would rather switch to the CNG API functions that are actively supported and well documented.

                        FNV-1 is nice - I use it for hashing things like user/password data. However, it only returns a 32-bit value so I guess there could be a higher instance of collisions. I've included some code that I use for this if you want to play with it.

                        '
                        Code:
                        ' zMD5.inc
                        
                        '******************************************************************************
                        ' Original code notice:
                        '
                        ' This code implements the MD5 message-digest algorithm. The algorithm is due
                        ' to Ron Rivest.  This code was written by Colin Plumb in 1993 and no copyright
                        ' is claimed. This code is in the public domain so do with it what you wish.
                        '
                        ' Equivalent code is available from RSA Data Security, Inc. This code has been
                        ' tested against that, and is equivalent, except that you don't need to include
                        ' two pages of legalese with every copy.
                        '
                        ' To compute the message digest of a chunk of bytes, declare an MD5Context
                        ' structure, pass it to MD5Init, CALL MD5Update as needed on buffers full of
                        ' bytes, and then call MD5Final, which will fill a supplied 16-byte array with
                        ' the digest.
                        '
                        '-----------------------------------------------------------
                        ' Translated to Powerbasic by Florent Heyworth  2000-04-10
                        ' Edited by Florent Heyworth                    2004-02-12
                        ' Edited by Jerry Wilson                        2004-04-22
                        '-----------------------------------------------------------
                        '
                        '******************************************************************************
                        #IF NOT %DEF (%ZMD5_INC_)
                        %ZMD5_INC_ = 1                ' Other modules can determine if this is loaded
                        
                        #INCLUDE "WIN32API.INC"       ' Include the Win32 API library if not done already.
                        
                        
                        '-- Constants for the md5_Transform routine.
                        %S11 = 7
                        %S12 = 12
                        %S13 = 17
                        %S14 = 22
                        %S21 = 5
                        %S22 = 9
                        %S23 = 14
                        %S24 = 20
                        %S31 = 4
                        %S32 = 11
                        %S33 = 16
                        %S34 = 23
                        %S41 = 6
                        %S42 = 10
                        %S43 = 15
                        %S44 = 21
                        
                        '-- MD5 context data structure.
                        TYPE MD5_CTX
                          lState(4) AS LONG         ' state (ABCD)
                          lCount(1) AS LONG         ' number of bits modulo 2^64 (lsp first)
                          bBuf      AS ASCIIZ * 64  ' input buffer
                        END TYPE
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' MD5_ShiftRight() - Bit shifting function (SUB in PB - what gives?).
                        '
                        FUNCTION md5_shiftRight(BYVAL lThis AS LONG, BYVAL lBits AS LONG) AS LONG
                          SHIFT RIGHT lThis, lBits
                          FUNCTION = lThis
                        END FUNCTION
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' MD5_ShiftLeft() - Bit shifting function (SUB in PB - what gives?).
                        '
                        FUNCTION md5_shiftLeft(BYVAL lThis AS LONG, BYVAL lBits AS LONG) AS LONG
                          SHIFT LEFT lThis, lBits
                          FUNCTION = lThis
                        END FUNCTION
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' MD5_Init() - Begins an MD5 operation, writing a new context.
                        '
                        SUB md5_init( BYREF ctx AS MD5_CTX )
                        
                          ctx.lCount(0) = 0
                          ctx.lCount(1) = 0
                        
                          '-- Load magic initialization constants.
                          ctx.lState(0) = &H67452301
                          ctx.lState(1) = &HEFCDAB89
                          ctx.lState(2) = &H98BADCFE
                          ctx.lState(3) = &H10325476
                        END SUB
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' MD5_Transform() - Do the core cacluations.
                        '
                        SUB md5_Transform ( BYREF ctx AS MD5_CTX, BYVAL bBuf AS LONG PTR )
                          LOCAL a AS LONG
                          LOCAL b AS LONG
                          LOCAL c AS LONG
                          LOCAL d AS LONG
                        
                          a = ctx.lState(0)
                          b = ctx.lState(1)
                          c = ctx.lState(2)
                          d = ctx.lState(3)
                        
                          '-- ROUND 1
                          a = a + ((((b)) AND ((c))) OR ((NOT (b)) AND ((d)))) + (@bBuf[ 0]) +&Hd76aa478
                          ROTATE LEFT a, %S11: a = a + b
                          d = d + ((((a)) AND ((b))) OR ((NOT (a)) AND ((c)))) + (@bBuf[ 1]) +&He8c7b756
                          ROTATE LEFT d, %S12: d = d + a
                          c = c + ((((d)) AND ((a))) OR ((NOT (d)) AND ((b)))) + (@bBuf[ 2]) +&H242070db
                          ROTATE LEFT c, %S13: c = c + d
                          b = b + ((((c)) AND ((d))) OR ((NOT (c)) AND ((a)))) + (@bBuf[ 3]) +&Hc1bdceee
                          ROTATE LEFT b, %S14: b = b + c
                          '
                          a = a + ((((b)) AND ((c))) OR ((NOT (b)) AND ((d)))) + (@bBuf[ 4]) +&Hf57c0faf
                          ROTATE LEFT a, %S11: a = a + b
                          d = d + ((((a)) AND ((b))) OR ((NOT (a)) AND ((c)))) + (@bBuf[ 5]) +&H4787c62a
                          ROTATE LEFT d, %S12: d = d + a
                          c = c + ((((d)) AND ((a))) OR ((NOT (d)) AND ((b)))) + (@bBuf[ 6]) +&Ha8304613
                          ROTATE LEFT c, %S13: c = c + d
                          b = b + ((((c)) AND ((d))) OR ((NOT (c)) AND ((a)))) + (@bBuf[ 7]) +&Hfd469501
                          ROTATE LEFT b, %S14: b = b + c
                          '
                          a = a + ((((b)) AND ((c))) OR ((NOT (b)) AND ((d)))) + (@bBuf[ 8]) +&H698098d8
                          ROTATE LEFT a, %S11: a = a + b
                          d = d + ((((a)) AND ((b))) OR ((NOT (a)) AND ((c)))) + (@bBuf[ 9]) +&H8b44f7af
                          ROTATE LEFT d, %S12: d = d + a
                          c = c + ((((d)) AND ((a))) OR ((NOT (d)) AND ((b)))) + (@bBuf[10]) +&Hffff5bb1
                          ROTATE LEFT c, %S13: c = c + d
                          b = b + ((((c)) AND ((d))) OR ((NOT (c)) AND ((a)))) + (@bBuf[11]) +&H895cd7be
                          ROTATE LEFT b, %S14: b = b + c
                          '
                          a = a + ((((b)) AND ((c))) OR ((NOT (b)) AND ((d)))) + (@bBuf[12]) +&H6b901122
                          ROTATE LEFT a, %S11: a = a + b
                          d = d + ((((a)) AND ((b))) OR ((NOT (a)) AND ((c)))) + (@bBuf[13]) +&Hfd987193
                          ROTATE LEFT d, %S12: d = d + a
                          c = c + ((((d)) AND ((a))) OR ((NOT (d)) AND ((b)))) + (@bBuf[14]) +&Ha679438e
                          ROTATE LEFT c, %S13: c = c + d
                          b = b + ((((c)) AND ((d))) OR ((NOT (c)) AND ((a)))) + (@bBuf[15]) +&H49b40821
                          ROTATE LEFT b, %S14: b = b + c
                        
                          '-- ROUND 2
                          a = a + ((((b)) AND ((d))) OR (((c)) AND (NOT (d)))) + (@bBuf[ 1]) +&Hf61e2562
                          ROTATE LEFT a, %S21: a = a + b
                          d = d + ((((a)) AND ((c))) OR (((b)) AND (NOT (c)))) + (@bBuf[ 6]) +&Hc040b340
                          ROTATE LEFT d, %S22: d = d + a
                          c = c + ((((d)) AND ((b))) OR (((a)) AND (NOT (b)))) + (@bBuf[11]) +&H265e5a51
                          ROTATE LEFT c, %S23: c = c + d
                          b = b + ((((c)) AND ((a))) OR (((d)) AND (NOT (a)))) + (@bBuf[ 0]) +&He9b6c7aa
                          ROTATE LEFT b, %S24: b = b + c
                          '
                          a = a + ((((b)) AND ((d))) OR (((c)) AND (NOT (d)))) + (@bBuf[ 5]) +&Hd62f105d
                          ROTATE LEFT a, %S21: a = a + b
                          d = d + ((((a)) AND ((c))) OR (((b)) AND (NOT (c)))) + (@bBuf[10]) +&H2441453
                          ROTATE LEFT d, %S22: d = d + a
                          c = c + ((((d)) AND ((b))) OR (((a)) AND (NOT (b)))) + (@bBuf[15]) +&Hd8a1e681
                          ROTATE LEFT c, %S23: c = c + d
                          b = b + ((((c)) AND ((a))) OR (((d)) AND (NOT (a)))) + (@bBuf[ 4]) +&He7d3fbc8
                          ROTATE LEFT b, %S24: b = b + c
                          '
                          a = a + ((((b)) AND ((d))) OR (((c)) AND (NOT (d)))) + (@bBuf[ 9]) +&H21e1cde6
                          ROTATE LEFT a, %S21: a = a + b
                          d = d + ((((a)) AND ((c))) OR (((b)) AND (NOT (c)))) + (@bBuf[14]) +&Hc33707d6
                          ROTATE LEFT d, %S22: d = d + a
                          c = c + ((((d)) AND ((b))) OR (((a)) AND (NOT (b)))) + (@bBuf[ 3]) +&Hf4d50d87
                          ROTATE LEFT c, %S23: c = c + d
                          b = b + ((((c)) AND ((a))) OR (((d)) AND (NOT (a)))) + (@bBuf[ 8]) +&H455a14ed
                          ROTATE LEFT b, %S24: b = b + c
                          '
                          a = a + ((((b)) AND ((d))) OR (((c)) AND (NOT (d)))) + (@bBuf[13]) +&Ha9e3e905
                          ROTATE LEFT a, %S21: a = a + b
                          d = d + ((((a)) AND ((c))) OR (((b)) AND (NOT (c)))) + (@bBuf[ 2]) +&Hfcefa3f8
                          ROTATE LEFT d, %S22: d = d + a
                          c = c + ((((d)) AND ((b))) OR (((a)) AND (NOT (b)))) + (@bBuf[ 7]) +&H676f02d9
                          ROTATE LEFT c, %S23: c = c + d
                          b = b + ((((c)) AND ((a))) OR (((d)) AND (NOT (a)))) + (@bBuf[12]) +&H8d2a4c8a
                          ROTATE LEFT b, %S24: b = b + c
                        
                          '-- ROUND 3
                          a = a + (((b)) XOR ((c)) XOR ((d))) + (@bBuf[ 5]) +&Hfffa3942
                          ROTATE LEFT a, %S31: a = a + b
                          d = d + (((a)) XOR ((b)) XOR ((c))) + (@bBuf[ 8]) +&H8771f681
                          ROTATE LEFT d, %S32: d = d + a
                          c = c + (((d)) XOR ((a)) XOR ((b))) + (@bBuf[11]) +&H6d9d6122
                          ROTATE LEFT c, %S33: c = c + d
                          b = b + (((c)) XOR ((d)) XOR ((a))) + (@bBuf[14]) +&Hfde5380c
                          ROTATE LEFT b, %S34: b = b + c
                          '
                          a = a + (((b)) XOR ((c)) XOR ((d))) + (@bBuf[ 1]) +&Ha4beea44
                          ROTATE LEFT a, %S31: a = a + b
                          d = d + (((a)) XOR ((b)) XOR ((c))) + (@bBuf[ 4]) +&H4bdecfa9
                          ROTATE LEFT d, %S32: d = d + a
                          c = c + (((d)) XOR ((a)) XOR ((b))) + (@bBuf[ 7]) +&Hf6bb4b60
                          ROTATE LEFT c, %S33: c = c + d
                          b = b + (((c)) XOR ((d)) XOR ((a))) + (@bBuf[10]) +&Hbebfbc70
                          ROTATE LEFT b, %S34: b = b + c
                          '
                          a = a + (((b)) XOR ((c)) XOR ((d))) + (@bBuf[13]) +&H289b7ec6
                          ROTATE LEFT a, %S31: a = a + b
                          d = d + (((a)) XOR ((b)) XOR ((c))) + (@bBuf[ 0]) +&Heaa127fa
                          ROTATE LEFT d, %S32: d = d + a
                          c = c + (((d)) XOR ((a)) XOR ((b))) + (@bBuf[ 3]) +&Hd4ef3085
                          ROTATE LEFT c, %S33: c = c + d
                          b = b + (((c)) XOR ((d)) XOR ((a))) + (@bBuf[ 6]) +&H4881d05
                          ROTATE LEFT b, %S34: b = b + c
                          '
                          a = a + (((b)) XOR ((c)) XOR ((d))) + (@bBuf[ 9]) +&Hd9d4d039
                          ROTATE LEFT a, %S31: a = a + b
                          d = d + (((a)) XOR ((b)) XOR ((c))) + (@bBuf[12]) +&He6db99e5
                          ROTATE LEFT d, %S32: d = d + a
                          c = c + (((d)) XOR ((a)) XOR ((b))) + (@bBuf[15]) +&H1fa27cf8
                          ROTATE LEFT c, %S33: c = c + d
                          b = b + (((c)) XOR ((d)) XOR ((a))) + (@bBuf[ 2]) +&Hc4ac5665
                          ROTATE LEFT b, %S34: b = b + c
                        
                          '-- ROUND 4
                          a = a + (((c)) XOR (((b)) OR (NOT (d)))) + (@bBuf[ 0]) +&Hf4292244
                          ROTATE LEFT a, %S41: a = a + b
                          d = d + (((b)) XOR (((a)) OR (NOT (c)))) + (@bBuf[ 7]) +&H432aff97
                          ROTATE LEFT d, %S42: d = d + a
                          c = c + (((a)) XOR (((d)) OR (NOT (b)))) + (@bBuf[14]) +&Hab9423a7
                          ROTATE LEFT c, %S43: c = c + d
                          b = b + (((d)) XOR (((c)) OR (NOT (a)))) + (@bBuf[ 5]) +&Hfc93a039
                          ROTATE LEFT b, %S44: b = b + c
                          '
                          a = a + (((c)) XOR (((b)) OR (NOT (d)))) + (@bBuf[12]) +&H655b59c3
                          ROTATE LEFT a, %S41: a = a + b
                          d = d + (((b)) XOR (((a)) OR (NOT (c)))) + (@bBuf[ 3]) +&H8f0ccc92
                          ROTATE LEFT d, %S42: d = d + a
                          c = c + (((a)) XOR (((d)) OR (NOT (b)))) + (@bBuf[10]) +&Hffeff47d
                          ROTATE LEFT c, %S43: c = c + d
                          b = b + (((d)) XOR (((c)) OR (NOT (a)))) + (@bBuf[ 1]) +&H85845dd1
                          ROTATE LEFT b, %S44: b = b + c
                          '
                          a = a + (((c)) XOR (((b)) OR (NOT (d)))) + (@bBuf[ 8]) +&H6fa87e4f
                          ROTATE LEFT a, %S41: a = a + b
                          d = d + (((b)) XOR (((a)) OR (NOT (c)))) + (@bBuf[15]) +&Hfe2ce6e0
                          ROTATE LEFT d, %S42: d = d + a
                          c = c + (((a)) XOR (((d)) OR (NOT (b)))) + (@bBuf[ 6]) +&Ha3014314
                          ROTATE LEFT c, %S43: c = c + d
                          b = b + (((d)) XOR (((c)) OR (NOT (a)))) + (@bBuf[13]) +&H4e0811a1
                          ROTATE LEFT b, %S44: b = b + c
                          '
                          a = a + (((c)) XOR (((b)) OR (NOT (d)))) + (@bBuf[ 4]) +&Hf7537e82
                          ROTATE LEFT a, %S41: a = a + b
                          d = d + (((b)) XOR (((a)) OR (NOT (c)))) + (@bBuf[11]) +&Hbd3af235
                          ROTATE LEFT d, %S42: d = d + a
                          c = c + (((a)) XOR (((d)) OR (NOT (b)))) + (@bBuf[ 2]) +&H2ad7d2bb
                          ROTATE LEFT c, %S43: c = c + d
                          b = b + (((d)) XOR (((c)) OR (NOT (a)))) + (@bBuf[ 9]) +&Heb86d391
                          ROTATE LEFT b, %S44: b = b + c
                        
                          ctx.lState(0) = ctx.lState(0) + a
                          ctx.lState(1) = ctx.lState(1) + b
                          ctx.lState(2) = ctx.lState(2) + c
                          ctx.lState(3) = ctx.lState(3) + d
                        
                        END SUB
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' MD5_Update() -  Update context to reflect concatenation of new buffer full
                        '                 of bytes.
                        '
                        SUB md5_Update( _
                          BYREF ctx   AS MD5_CTX, _
                          BYVAL bBuf  AS BYTE PTR, _
                          BYVAL dwLen AS DWORD _
                          )
                        
                          LOCAL t AS DWORD
                        
                          '-- Update bitcount
                          t = ctx.lCount(0)
                          ctx.lCount(0) = t + md5_shiftLeft( dwLen, 3 )
                          IF ctx.lCount(0) < t THEN
                            INCR ctx.lCount(1) 'carry from low to high
                          END IF
                          ctx.lCount(1) = ctx.lCount(1) + md5_shiftRight( dwLen, 29 )
                          t = ( md5_shiftRight( t, 3) AND &H3F ) 'Bytes already IN shsInfo->DATA
                        
                          '-- Handle any leading odd-sized chunks.
                          IF ISTRUE(t) THEN
                            LOCAL p AS BYTE PTR
                            p = VARPTR(ctx.bBuf)
                            p = p + t
                            t = 64 - t
                            IF dwLen < t THEN
                              CALL MoveMemory( BYVAL p, BYVAL bBuf, dwLen )
                              EXIT SUB
                            END IF
                            CALL MoveMemory( BYVAL p, BYVAL bBuf, t )
                            CALL md5_Transform( ctx, VARPTR(ctx.bBuf) )
                            bBuf = bBuf + t
                            dwLen = dwLen - t
                          END IF
                        
                          '-- Process data in 64-byte chunks.
                          DO WHILE (dwLen >= 64)
                            CALL MoveMemory( BYVAL VARPTR(ctx.bBuf), BYVAL bBuf, 64)
                            CALL md5_Transform( ctx, VARPTR(ctx.bBuf) )
                            bBuf = bBuf + 64
                            dwLen = dwLen - 64
                          LOOP
                        
                          '-- Handle any remaining bytes of data.
                          CALL MoveMemory( BYVAL VARPTR(ctx.bBuf), BYVAL bBuf, dwLen )
                        END SUB
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' MD5_Final() -  Final wrapup.
                        '
                        SUB md5_Final( _
                          BYVAL bDigest AS BYTE PTR, _
                          ctx           AS MD5_CTX _
                          )
                        
                          LOCAL lCount  AS DWORD
                          LOCAL p       AS BYTE PTR
                          LOCAL bBuffer AS BYTE PTR
                        
                          bBuffer = VARPTR(ctx.bBuf)
                        
                          '-- Compute number of bytes MOD 64
                          lCount = (md5_shiftRight(ctx.lCount(0), 3) AND &H3F )
                        
                          '-- Set the first char of padding to &H80.  This is safe since there is
                          '    always at least one byte free
                          p = VARPTR(ctx.bBuf)
                          p = p + lCount
                          @p = &H80
                          INCR p
                        
                          '-- Bytes of padding needed TO make 64 bytes
                          lCount = 64 - 1 - lCount
                        
                          '-- Pad out TO 56 MOD 64
                          IF lCount < 8 THEN
                        
                            '-- Two lots of padding: pad the first block to 64 bytes
                            CALL FillMemory( p, lCount, 0 )
                            CALL md5_Transform( ctx, VARPTR(ctx.bBuf) )
                            CALL FillMemory( BYVAL VARPTR(ctx.bBuf), 56, 0 )
                          ELSE
                            '-- Pad block to 56 bytes
                            CALL FillMemory( BYVAL p, (lCount - 8), 0)
                          END IF
                        
                          '-- Append length in bits and transform
                          CALL MoveMemory( BYVAL VARPTR(@bBuffer[14 * SIZEOF(lCount)]), BYVAL VARPTR(ctx.lCount(0)), SIZEOF(lCount))
                          CALL MoveMemory( BYVAL VARPTR(@bBuffer[15 * SIZEOF(lCount)]), BYVAL VARPTR(ctx.lCount(1)), SIZEOF(lCount))
                          CALL md5_Transform( ctx, VARPTR(ctx.bBuf) )
                          CALL MoveMemory( BYVAL bDigest, BYVAL VARPTR(ctx.lState(0)), 16) ' Move the MD5 value!
                        
                          '-- Zeroise sensitive information
                          CALL FillMemory( BYVAL VARPTR(ctx), SIZEOF(ctx), 0)
                        END SUB
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' MD5_Digest() - Returns the MD5 digest as a hex string.
                        '
                        FUNCTION md5_Digest( BYREF ctx AS MD5_CTX) AS STRING
                          LOCAL sHex     AS STRING
                          LOCAL szDigest AS ASCIIZ * 16
                          LOCAL bDigest  AS BYTE PTR
                          LOCAL sCheck   AS STRING
                          REGISTER i     AS LONG
                        
                          bDigest = VARPTR(szDigest)
                          CALL md5_Final( bDigest, ctx)
                          IF bDigest <> %NULL THEN
                            FOR i = 0 TO 15
                              sHex = sHex + HEX$(@bDigest[i],2)
                            NEXT
                          END IF
                          FUNCTION = sHex
                        END FUNCTION
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' MD5_ReadFile() - Calculate MD5 value for a file.
                        '
                        ' Reads a file and calls md5_Update on the buffer. md5_Init() MUST have been
                        ' called prior to calling this function - call md5_Digest() afterwards to
                        ' get the fingerprint.
                        '
                        SUB md5_readFile( BYREF ctx AS MD5_CTX,  szFile AS ASCIIZ )
                        
                          LOCAL hFile       AS DWORD
                          LOCAL szBuffer    AS ASCIIZ * 1024
                          LOCAL lBufLen     AS LONG
                          LOCAL lLen        AS LONG
                          LOCAL lResult     AS LONG
                          LOCAL lToRead     AS LONG
                          LOCAL lToReadHigh AS LONG
                          LOCAL lpBuffer    AS LONG
                        
                          lBufLen = SIZEOF(szBuffer)
                          lpBuffer = VARPTR(szBuffer)
                          hFile = CreateFile( szFile, %GENERIC_READ, %FILE_SHARE_READ, _
                                                  BYVAL %NULL, %OPEN_EXISTING, %FILE_ATTRIBUTE_NORMAL, 0& )
                          IF hFile <> %INVALID_HANDLE_VALUE THEN
                            lToRead = GetFileSize(hFile, lToReadHigh)
                            md5_readFile_loop:
                            lResult = ReadFile( hFile, BYVAL lpBuffer, lBufLen, lLen , BYVAL %NULL )
                            IF lResult = 0  THEN GOTO md5_readFile_close
                            CALL md5_Update(ctx, lpBuffer, lLen)
                            lToRead = lToRead - lLen
                            IF lToRead = 0 THEN GOTO md5_readFile_close
                            GOTO md5_readFile_loop
                            md5_readFile_close:
                            CALL CloseHandle(hFile)
                          END IF
                        END SUB
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' zCalcMD5Hex() - Calculate MD5 value for data and return as a hex string.
                        '
                        ' Call this function with a pointer to the data to be analyzed as well as the
                        ' length of the data in bytes. The function will create and return a 32-byte
                        ' string containing the hex representation of the MD5 value. Useful for text
                        ' based protocols (i.e. HTTP, SIP). This function is externally visible to PB.
                        '//////////////////////////////////////////////////////////////////////////////
                        FUNCTION zCalcMD5Hex ( _
                          BYVAL pData   AS BYTE POINTER, _    ' Pointer to raw data in memory
                          BYVAL dSize   AS LONG _             ' Length of data to use
                          )             AS STRING             ' Return 32-byte hex string of MD5 value
                        
                          DIM ctx    AS MD5_CTX
                        
                          CALL md5_Init(ctx)
                          CALL md5_Update(ctx, pData, dSize)
                          FUNCTION = md5_Digest(ctx)
                        
                        END FUNCTION
                        
                        
                        '//////////////////////////////////////////////////////////////////////////////
                        ' zCalcMD5Mem() - Calculate MD5 value for data and place value in memory buffer.
                        '
                        ' Call this function with a pointer to the data to be analyzed as well as the
                        ' length of the data in bytes. The function will place the 16-byte MD5 value
                        ' into the buffer pointed to by pMd5. This is useful for binary protocols that
                        ' required 128-bit MD5 values (i.e. SSL). This function is externally visible.
                        ' This is much faster than CalcMD5Hex which has to perform string calculations.
                        '
                        FUNCTION zCalcMD5Mem ( _
                          BYVAL pData AS BYTE POINTER, _  ' Pointer to raw data in memory
                          BYVAL dSize AS LONG, _          ' Length of data to use (in bytes)
                          BYVAL pMd5  AS BYTE POINTER _   ' Pointer to buffer to hold 16-byte MD5 value
                          )           AS LONG             ' Always return zero (not useful)
                        
                          DIM ctx    AS MD5_CTX
                        
                          CALL md5_Init(ctx)
                          CALL md5_Update(ctx, pData, dSize)
                          CALL md5_Final(pMd5, ctx)       ' Call to md5_final points to external buffer.
                        
                        END FUNCTION
                        
                        #ENDIF
                        '
                        '
                        Code:
                        '////////////////////////////////////////////////////////////////////////////////////////
                        ' FNV1_32 - Simple data hashing function returns a 32-bit value.
                        '
                        ' FNV (Fowler/Noll/Vo) is a very simple hash designed for high-speed hashing, but with
                        ' minimal collisions- especially with numbers and lower case strings. It works well
                        ' with small data elements (e.g. user name, URL). FNV-1 is preferred over FNV-0. This
                        ' is often used for indexes or name hashing for authentication.
                        '
                        ' Testing results in around 20M calculations per second (.12us per calculation) on an
                        ' Intel i5-4300M CPU @ 2.6GHz in a tight loop using a 32-byte string. Processing time
                        ' doubles when using a 64-byte string.
                        '
                        ' Interesting discussion and source code at http://www.isthe.com/chongo/tech/comp/fnv/
                        ' and in the PB forum source code library under topic "FNV (Fowler/Noll/Vo) 32-bit hash".
                        '
                        ' Creating a version of this that only supports strings that break on 32-bit boundries
                        ' MAY be faster so should be considered for future development.
                        '////////////////////////////////////////////////////////////////////////////////////////
                        FUNCTION FNV1_32( _
                          BYVAL lpData  AS DWORD, _   ' Pointer to buffer of data to hash
                          BYVAL dSize   AS DWORD _    ' Size of buffer
                          ) AS DWORD
                          #REGISTER NONE
                          ! mov esi, lpData        ;esi = ptr to buffer
                          ! mov ecx, dSize         ;ecx = length of buffer (counter)
                          ! mov eax, %FNV1_32_INIT ;offset_basis  ;set to 0 for FNV-0, or 2166136261 for FNV-1
                          ! mov edi, %FNV_32_PRIME ;FNV_32_PRIME = 16777619
                          ! xor ebx, ebx           ;ebx = 0
                          nextbyte:
                          ! mul edi                ;eax = eax * FNV_32_PRIME
                          ! mov bl, [esi]          ;bl = byte from esi
                          ! xor eax, ebx           ;al = al xor bl
                          ! inc esi                ;esi = esi + 1 (buffer pos)
                          ! dec ecx                ;ecx = ecx - 1 (counter)
                          ! jnz nextbyte           ;if ecx is 0, jmp to NextByte
                          ! mov FUNCTION, eax      ;else, function = eax
                        END FUNCTION  ' FNV1_32
                        '

                        Comment


                        • #13
                          Very cool Jerry! Thanks! Not that I understand it!!

                          Comment


                          • #14
                            Hi Jerry,

                            A bit curious, the difference between MD5 and FNV functions, which function is more reliable (lesser collisions) ?

                            Comment


                            • #15
                              Bear in mind that in this case, if file sizea are different, a collision doesn't matter
                              The chance of getting both a hash collision and a matching file size reduces the risk of false positives considerably.

                              Hashing speed for multi-megabyte inputs would be my major concern

                              Comment


                              • #16
                                I wanted to add a bit more here. I am hashing every file on my network (except c:\windows) looking for duplicates.
                                Here is the stripped down test code.

                                I was originally going to use MD5 but George Bleck made me realize SHA256 might be a better idea!

                                Code:
                                'Uses Mike Doty's function at 'https://forum.powerbasic.com/forum/user-to-user-discussions/programming/789154-encryption-question?p=789312#post789312
                                
                                #COMPILE EXE
                                #DIM ALL
                                #INCLUDE "WIN32API.INC"
                                
                                FUNCTION PBMAIN() AS LONG
                                
                                LOCAL sTxt,sKey AS STRING
                                
                                
                                OPEN "C:\Basic Source\FILE_AND_RECURSION\FILES_FIND_AND_DO\moo.mp4" FOR BINARY AS #1
                                
                                GET #1, LOF(1), sTxt
                                CLOSE #1
                                
                                
                                'sTxt = "This is some text that we want to produce a Hash for "
                                sKey = "This is the secret key to Hash with"
                                
                                MSGBOX SHA256HMAC(sTxt,sKey)
                                
                                END FUNCTION
                                
                                '------------------------------------------------------------------------------
                                
                                FUNCTION SHA256HMAC( BYVAL sText AS STRING, BYVAL pbSecret AS STRING ) AS STRING ' Binary
                                
                                LOCAL hAlg, phHash AS DWORD
                                LOCAL sBinHMAC AS STRING * 32 ' 32 for SHA256
                                LOCAL pbOutput AS BYTE PTR
                                
                                BCryptOpenAlgorithmProvider( hAlg, "SHA256"$$, $$NUL, %BCRYPT_ALG_HANDLE_HMAC_FLAG ) ' We need hAlg
                                BCryptCreateHash( hAlg, phHash, BYVAL %Null, 0, STRPTR( pbSecret ), LEN( pbSecret ), 0 ) ' We need pbHash
                                BCryptHashData( phHash, STRPTR( sText ), LEN( sText ), 0 )
                                BCryptFinishHash( phHash, VARPTR( sBinHMAC ), 32, 0 ) ' 32 for SHA256, We need sBinHMAC
                                BCryptDestroyHash( phHash )
                                BCryptCloseAlgorithmProvider( hAlg, 0 )
                                
                                pbOutput = VARPTR(sBinHMAC)
                                
                                LOCAL HASH_STRING AS STRING
                                LOCAL i AS LONG
                                
                                FOR i = 0 TO LEN(sBinHMAC) -1
                                HASH_STRING += HEX$(@pbOutput[i],2)
                                NEXT
                                
                                
                                
                                FUNCTION = HASH_STRING
                                
                                END FUNCTION

                                Comment


                                • #17
                                  FNV-1 is quite fast but has a MUCH larger collision window (depends on version bit depth). Using additional compare methods (like file size as was mentioned) helps to limit it quite a bit. I like FNV for it's use of a very small hash length but it is by no means forensically or cryptologically secure. But it does have it's place (like fast hash compares or as an indices for small sets of data items).

                                  Someone mentioned using hashes for passwords, although a good idea... its not enough. I am hoping you are also salting/peppering to thwart rainbow tables. Better yet... use a key derivation function like PBKDF2 (which time stretches the key) and an underlying cryptologically secure hash algorithm so GPU bruteforcing becomes fruitless.
                                  <b>George W. Bleck</b>
                                  <img src='http://www.blecktech.com/myemail.gif'>

                                  Comment


                                  • #18
                                    Here is a shorter version of Stuart's post #4. It uses BCryptHash introduced in Windows 10 so is not declared in "WIN32API.INC".

                                    BCryptHash

                                    Code:
                                    #COMPILE EXE
                                    #DIM ALL
                                    #INCLUDE "WIN32API.INC"
                                    
                                    Declare Function BCryptHash Lib "bcrypt.dll" Alias "BCryptHash"( ByVal hAlg As Dword, ByVal pbSecret As Byte Ptr, ByVal cbSecret As Dword, _
                                    ByVal pbinput As Byte Ptr, ByVal cbInput As Dword, ByVal pbOutput As Byte Ptr, ByVal cbOutput As Dword ) As Dword
                                    
                                    FUNCTION PBMAIN() AS LONG
                                    LOCAL s, h,sTxt,sKey AS STRING
                                    LOCAL i AS LONG
                                    
                                    sTxt = "This is some text that we want to produce a Hash for "
                                    sKey = "This is the secret key to Hash with"
                                    s = SHA256HMAC(sTxt,sKey)
                                    FOR i = 1 TO LEN(s)
                                    h += HEX$(ASC(s,i),2,1)
                                    NEXT
                                    ? h
                                    END FUNCTION
                                    
                                    FUNCTION SHA256HMAC( BYVAL sText AS STRING, BYVAL pbSecret AS STRING ) AS STRING ' Binary
                                    ' sText is message to be hashed code by David Roberts
                                    ' pbSecret is secret HMAC key
                                    
                                    LOCAL hAlg, phHash AS DWORD
                                    LOCAL sBinHMAC AS STRING * 32 ' 32 for SHA256
                                    
                                    BCryptOpenAlgorithmProvider( hAlg, "SHA256"$$, $$NUL, %BCRYPT_ALG_HANDLE_HMAC_FLAG ) ' We need hAlg
                                    BCryptHash( hAlg, STRPTR( pbSecret ), LEN( pbSecret ), STRPTR( sText ), LEN( sText ), VARPTR( sBinHMAC ), 32 ) '32 for SHA256
                                    BCryptCloseAlgorithmProvider( hAlg, 0 )
                                    FUNCTION = sBinHMAC
                                    
                                    END FUNCTION
                                    '

                                    Comment


                                    • #19
                                      Originally posted by George Bleck View Post
                                      FNV-1 is quite fast but has a MUCH larger collision window (depends on version bit depth). Using additional compare methods (like file size as was mentioned) helps to limit it quite a bit. I like FNV for it's use of a very small hash length but it is by no means forensically or cryptologically secure. But it does have it's place (like fast hash compares or as an indices for small sets of data items).
                                      Exactly right. I use FNV1 to hash user names in a large table and then create a 32-bit index of the values. Collisions are possible, especially as the table grows in size, so collision detection is used when user names are added to the table. And the user name lookup is VERY fast. For indexing, the hashes must be unique.

                                      Someone mentioned using hashes for passwords, although a good idea... its not enough. I am hoping you are also salting/peppering to thwart rainbow tables. Better yet... use a key derivation function like PBKDF2 (which time stretches the key) and an underlying cryptologically secure hash algorithm so GPU bruteforcing becomes fruitless.
                                      I agree. Clear text passwords should NEVER be stored - anywhere. At the very least, a salted hash of the passwords should be used.

                                      For David's project, It doesn't sound like security is an issue - just fast and accurate hashing of file names and data so cryptographically-secure hashing will probably add unneeded overhead to the process. In his case, perhaps an FNV hash of the file name may be sufficient to look for duplicate names. If one were to cycle through, virtually, all files on a computer (probably a LOT of files), and save the file name hash to an indexed table, duplicate file names would be detected. If the file name is also stored in the table but not indexed, hash collisions would probably be detected. If you hash the data in file (SHA256), then different versions of the file could be detected. It just depends on how deep of an inspection you want to do.

                                      It seems fairly likely that some duplicates will be found. Many vendors distribute applications with public DLLs (e.g. zlib.dll) that are stored in the apps directory. And, I suspect that there's more than one "config.txt" file.

                                      Comment


                                      • #20
                                        Mr Wilson

                                        You mentioned for FNV1 hash function :
                                        so collision detection is used when user names are added to the table
                                        how do you do this collision detection ? do you have some code to share with us? thanks

                                        Comment

                                        Working...
                                        X