Tweet
The key to these high order SHAs is the type of provider and its associated Cryptographic Service Provider (CSP).
With HashFile we use %PROV_RSA_FULL and $MS_DEF_PROV respectively to gain access to the inbuilt MD5 an SHA1.
If we use %PROV_RSA_AES and $MS_ENH_RSA_AES_PROV then not only do we gain access to MD5 and SHA1 but also SHA256, SHA384 and SHA512. Note that for XP < SP3 these values are not to be found.
These values are used in the function crGetDefaultRSAHandle. The provider type is used for the CryptAcquireContext API. If there is a default Cryptographic Service Provider (CSP) associated with the given provider type then it will be used without further ado. If there is not a default CSP then we force the issue with a second pass of CryptAcquireContext giving this time both the provider type and CSP.
It follows then if there is a default CSP the following code will work with both XP SP3 and Vista. If there is not a default CSP then the following code will work with XP SP3 but may fail with Vista.
The values we need, including all the necessary numeric equates, have been added to Don Dickinson's Wincrypt.inc in post #3.
For XP SP3 we have for the CSP:
$MS_ENH_RSA_AES_PROV = "Microsoft Enhanced RSA and AES Cryptographic Provider (Prototype)"
and for Vista we have for the CSP:
$MS_ENH_RSA_AES_PROV = "Microsoft Enhanced RSA and AES Cryptographic Provider"
That is sans "(Prototype)" with Vista.
For Vista without a default CSP associated with %PROV_RSA_AES then Wincrypt.inc should be edited. With a default CSP then no editing is required because the correct CSP will be used.
Right, that is the caveat out of the way.
This is a typical output of the test code in post #2 on a 100MB file and some text.
Much of PBMain is involved with the display and whether you are using PBWin or PBCC.
The core statment is: Hash( sText, WhichHash, HashHex, Flag )
The input sText is either a filepath or text.
The input WhichHash is one of MD5, SHA1, SHA256, SHA384 and SHA512. There are lesser algorithms available but the following code does not concern itsef with them.
The output HashHex is the resulting hash value in hex string format.
The input Flag is either 0 or 1 simply advises Hash() whether sText is either text or a file respectively.
On reading a file we don't need the data again so file cacheing is bypassed. In theory this should speed things up but I'm not sure how significant that is. However, we do avoid the possibility of existing cache data being put aside which may have been used again. The bypassing is, of course, achieved via FILE_FLAG_NO_BUFFERING in the CreateFile API. Buffer addresses for read and write operations should be sector aligned when using FILE_FLAG_NO_BUFFERING and to this end I have used VirtualAlloc API. Of course, this is ideal for the test code below as the file is read by each and every algorithm.
Some of the code is either a direct port or an adaptation of Don Dickinson's code which accompanied the publication of Wincrypt.inc, 10 years ago in a few weeks time.
Initial tests saw MD5 on files being slower than anticipated. I was already using my timer macro and did not want to move its position to different areas of the code - I wanted multiple timers so updated the timer macro here.
I was now in a position to do something like this
StartTimer(1)
Code ...
StopTimer(1)
zTrace("Timer 1:" + sTimeTaken(1,2))
where sTimeTaken(1,2) means using Timer 1 and display to 2 decimal places.
zTrace is Practice Terrier's superb little tracing utilty. If you have not seen this yet then you should check it out here.
To cut a long story short I found that the issue was clearly related to the first instance of Hash() and had nothing to with MD5, it was nothing to do with bypassing the file cache and seemed to be a small absolute value so its effect diminished as the file being tested increased in size. First instance smacks of some kind of initialization but I don't know what. I should add that we are only talking about 5 to 10 milliseconds.
To ensure that all trials are on a level playing field the simplest solution is to execute a dummy run of Hash() before the trials start.
The test code will fail with XP < SP3. The application should not fail, he says fingers crossed, as an adaptation of José Roca's API error trapper is used. If we use %PROV_RSA_FULL and $MS_DEF_PROV, as HashFile does, then MD5 and SHA1 will succeed and the high order SHAs will fail. You have been warned.
I should add that the following is not the first to gather these high order SHAs. Rick Kelly has done a monumental task - see his post here. It did not dawn on me at that time what was under the bonnet - a cursory glance and I was off like a shot. There are others much better qualified than I who could make good use of Rick's work.
As always, have fun.
With HashFile we use %PROV_RSA_FULL and $MS_DEF_PROV respectively to gain access to the inbuilt MD5 an SHA1.
If we use %PROV_RSA_AES and $MS_ENH_RSA_AES_PROV then not only do we gain access to MD5 and SHA1 but also SHA256, SHA384 and SHA512. Note that for XP < SP3 these values are not to be found.
These values are used in the function crGetDefaultRSAHandle. The provider type is used for the CryptAcquireContext API. If there is a default Cryptographic Service Provider (CSP) associated with the given provider type then it will be used without further ado. If there is not a default CSP then we force the issue with a second pass of CryptAcquireContext giving this time both the provider type and CSP.
It follows then if there is a default CSP the following code will work with both XP SP3 and Vista. If there is not a default CSP then the following code will work with XP SP3 but may fail with Vista.
The values we need, including all the necessary numeric equates, have been added to Don Dickinson's Wincrypt.inc in post #3.
For XP SP3 we have for the CSP:
$MS_ENH_RSA_AES_PROV = "Microsoft Enhanced RSA and AES Cryptographic Provider (Prototype)"
and for Vista we have for the CSP:
$MS_ENH_RSA_AES_PROV = "Microsoft Enhanced RSA and AES Cryptographic Provider"
That is sans "(Prototype)" with Vista.
For Vista without a default CSP associated with %PROV_RSA_AES then Wincrypt.inc should be edited. With a default CSP then no editing is required because the correct CSP will be used.
Right, that is the caveat out of the way.
This is a typical output of the test code in post #2 on a 100MB file and some text.
Much of PBMain is involved with the display and whether you are using PBWin or PBCC.
The core statment is: Hash( sText, WhichHash, HashHex, Flag )
The input sText is either a filepath or text.
The input WhichHash is one of MD5, SHA1, SHA256, SHA384 and SHA512. There are lesser algorithms available but the following code does not concern itsef with them.
The output HashHex is the resulting hash value in hex string format.
The input Flag is either 0 or 1 simply advises Hash() whether sText is either text or a file respectively.
On reading a file we don't need the data again so file cacheing is bypassed. In theory this should speed things up but I'm not sure how significant that is. However, we do avoid the possibility of existing cache data being put aside which may have been used again. The bypassing is, of course, achieved via FILE_FLAG_NO_BUFFERING in the CreateFile API. Buffer addresses for read and write operations should be sector aligned when using FILE_FLAG_NO_BUFFERING and to this end I have used VirtualAlloc API. Of course, this is ideal for the test code below as the file is read by each and every algorithm.
Some of the code is either a direct port or an adaptation of Don Dickinson's code which accompanied the publication of Wincrypt.inc, 10 years ago in a few weeks time.
Initial tests saw MD5 on files being slower than anticipated. I was already using my timer macro and did not want to move its position to different areas of the code - I wanted multiple timers so updated the timer macro here.
I was now in a position to do something like this
StartTimer(1)
Code ...
StopTimer(1)
zTrace("Timer 1:" + sTimeTaken(1,2))
where sTimeTaken(1,2) means using Timer 1 and display to 2 decimal places.
zTrace is Practice Terrier's superb little tracing utilty. If you have not seen this yet then you should check it out here.
To cut a long story short I found that the issue was clearly related to the first instance of Hash() and had nothing to with MD5, it was nothing to do with bypassing the file cache and seemed to be a small absolute value so its effect diminished as the file being tested increased in size. First instance smacks of some kind of initialization but I don't know what. I should add that we are only talking about 5 to 10 milliseconds.
To ensure that all trials are on a level playing field the simplest solution is to execute a dummy run of Hash() before the trials start.
The test code will fail with XP < SP3. The application should not fail, he says fingers crossed, as an adaptation of José Roca's API error trapper is used. If we use %PROV_RSA_FULL and $MS_DEF_PROV, as HashFile does, then MD5 and SHA1 will succeed and the high order SHAs will fail. You have been warned.
I should add that the following is not the first to gather these high order SHAs. Rick Kelly has done a monumental task - see his post here. It did not dawn on me at that time what was under the bonnet - a cursory glance and I was off like a shot. There are others much better qualified than I who could make good use of Rick's work.
As always, have fun.
I didn't get a migraine but it was close.
Actually, I don't drink from one month to the next - sometimes three or four months - and one day my grocer will notice me looking over at the booze cabinet and say "Captain Morgan?" to which I reply "Aye, why not".
Comment