Where should I stote data

[John Petty]

Normally as PB programs do not need the be registered I would not have the program located in “Program Files” but in its own folder elsewhere complete with DLL’s and configuration information thus avoiding the current security complexities. Any large data files I stored in another separate folder.
I am in the process finalizing a program to post in Source Code that may be of some interest and as it could benefit from individual user profiles am trying to decide where to store its data.
Kev Peel recently posted an update to his solution and Michael Mattias did similar in March 2008 , both gave examples which choose “documents and settings\user\application data etc” but from my observations that is not what Microsoft use, instead it puts the data in a hidden folder “documents and settings\user\local data\application data etc” so for instance this is where you would find the Outlook PST file. It appears the “documents and settings\user\application data etc” is only intended for roaming data if a user should have such a profile set up on a server.
Of course just to make it simple the whole concept CSIDL’s has been deprecated in Vista and I guess 7.

[Michael Mattias]

You can use the D&S/User Name/Local data.... folder (CSIDL_APPDATA) if you want, but the bigger problem you run into is trying to put anything there at installation time, since one user (probably admin since he/she has privileges to install)) does not have access to the folder of another user.

So, you might install "common" data in D&S/All USers/Application Data (CSIDL_COMMON_APPDATA) and the user-specific info in D&S/userName/Application data .... as long as your application is prepared to deal with the fact that on the first use for any user there will be no "user-specific" data.

(This topic comes up frequently on the Inno Setup newgroups. You can browse the newgroup archives starting at http://www.jrsoftware.org)

[John Petty]

Thanks Michael
I have never used Delphi so that is not much help. If i read you correctly it is exactly what I am trying to do, ie a user logs on to the computer calls the program which simply detects they have no personal configuration and so requests same, that should be simple the config (ini) exists or it doesn't, it is what MS seem to do. There is no common data, it is very user dependant, this is a mail app.
So should the user dependant data be stored in user\application data or user\local settings\application data?
John

[Michael Mattias]

So should the user dependant data be stored in user\application data or user\local settings\application data

User-specific data should be stored 'in or under' CSIDL_APPDATA - wherever that happens to be at the time.

Shared data should be stored 'in or under' CDISL_COMMONAPPDATA -whereever that happens to be at the time.

Yes, both are "user-movable' although I have never seen it done in Real Life.

MS suggests - and I use - the CSIDL_APPDATA or CSIDL_COMMONAPPDATA\CompanyName\ApplicationName folder, so there is no possibility of a folder or file name conflict should some other publisher want to use the same configuration file name.

You can use ShgetSpecialFolderPath() to find where it's at... or if you want to just cut and paste working PB code you can go to Move That INI File! and Get That Ini File Name!

That code will even automatically create any subfolders required... making it very handy if you want to use the CompanyName/ApplicationName scheme and don't want a phone call to roust you out of bed when a "new user" needs to use the application for the first time.

MCM

[Guy Dombrowski]

data storage

I NEVER but NEVER store My customer data files in anyplace that Microsoft recommend.
Better use a separate folder that will not get wiped if ( sorry... WHEN ) you will have to re-install Windows.
Better yet, a separate partition on the hard disk or yes, another hard disk.
Coming from somebody that spend a lot of time fixing and re-installing ailing Windows.

[Knuth Konrad]

Guy,

with this attitude, you'll run into major problems with Vista and upwards, due to it's locked down security approach.

The way Michael explained it, is the way MS recommended since NT 4. Basically, non-administrative users can't and shouldn't *write* anywhere besides Documents and Settings\<user>. This is done so that even if a user catches a virus, this virus (which runs under that user's account privileges) can't do harm to the whole system, but "just" to this user's data.

While I agree that this is a mess in terms of maintenance, as a company admin, I prefer security over maintenance these days.

Using disk images and identical hardware lessens the re-install hassle. But you can't look over the shoulder of all of your 200 users and stop them clicking on that newest attack vector.

We had to deal with proprietary software, which didn't behave well under W2K. It even wrote into the Windows folder. We spent two days (three people = six man days worth of company money) using FileMon and RegMon to figure out in which no-go areas that application would write and grant access to those specific files (even created 0 byte dummy files which that app uses) and registry keys so that it would still run under a restricted user account. If we had a choice, we would have immediately dumped that software and used another one, which would have been more in line with our security policy.

[Guy Dombrowski]

Security

Knuth,

None of my customers on Vista so far.
When they buy a new machine they always use the XP downgrade option.
As their accounting files are their most important files, it is understandable that they are a little bit fearful of trusting MS with those.
I always recommend also that the accounting machine not be used for any kind of idle Internet browsing but only with trusted sites.
If that computer or its Windows die, it take only one minute to copy the data file to a brand new or spare machine and since there is no lengthy re-installation process with PB the down time is kept to a minimum.
But I admit I will have to address the Win Seven problem some day.

[Michael Mattias]

While I agree that this is a mess in terms of maintenance, as a company admin, I prefer security over maintenance these days.

You mean program maintenance?

That's why BASIC provides the abilty to create your own functions. Write 'em once, use 'em many....

FUNCTION GetCurrentUserConfigFileName () AS STRING

FUNCTION GetCommonConfigFileName() AS STRING

After you write 'em, put these functions into a DLL so NOBODY can screw 'em up.

MCM

[Jeff Blakeney]

I've been reading about this issue for some time and is one of the things that makes me dislike the direction Windows has taken. My personal take on it is that the administrator should have total control over where everything is on the system and be able to set restrictions for other users (not that I've ever had a machine set up for multiple users) and the users should have total control over where stuff goes within their (possibly) constrained areas.

The way things are now, you need to use specialized software to backup your software when it should be as simple as dragging a folder to a removable drive. I'm not even sure specialized software could be used to back up individual programs seeing as there is nothing linking the folder in "Program Files" with any folders in "Documents and Settings" or with keys in the registry. The registry is another thing I avoid except for doing file type associations but those can easily be reset by a user using "Folder Options".

I'm still using XP and haven't done any real testing but I've read that Windows will not even return an error if you try to create or write to a file in a protected directory now. This means I can't simply check for an error anymore, I need to write extra code to make sure the changes went through and if they don't, then I can let the user know that the file needs to be put somewhere else.

Basically, what it boils down to is, I should be in control of the organization of my files and Windows should only be concerned with the its files (a few files in the root directory and the Windows directory) and leave the rest to me.

[Guy Dombrowski]

Amen to that

[Knuth Konrad]

Guy,

None of my customers on Vista so far.
When they buy a new machine they always use the XP downgrade option.
As their accounting files are their most important files, it is understandable that they are a little bit fearful of trusting MS with those.

As I wrote: these recommended locations for storing data have been around since NT4. Just that with Vista now MS (thankfully) enforces these recommendations.

Part of Windows security problem is this programmer's attitude to store files everywhere. Which makes it necessary to have write access everywhere. Which led to the habit to just run as administrator ... and each and every malware in the world gratefully screams "Hurray!"

Michael,

You mean program maintenance?

No, "machine/system" maintenance. On the programming side, it's like you wrote: spent once a bit time writing adequate helper functions and you'Re done.

Jeff,

the way you described it should be, is exactly the way it is:

- Admin has control of machine.
- Admin can move (system) folders around (You can move the Program Files or Documents and settings folder, if you like).
- Admin can give permissions as he sees fit.
- Windows only cares about root, %WIN% and below. Everything else is in Common files (MDAC, for example) or other such locations is not part of the OS itself

Basically, Windows moves finally to the direction where "real OS" (Unix) have always been. Users there where never able to write "outside of user land". Early Windows versions started this very fatal habit, we all learned to love in the past.

[Kev Peel]

Unfortunately this habit of dumping junk files about the system still persists - install any major application and it will scatter dlls, logs and other files all over the system, including the important "windows" and "system32" folders, very difficult to remove it all.

I for one am glad that M$ has forced applications to at least save data and settings in a single location (the registry is another issue). How I wish more system folders would be locked down and made untouchable

[Jeff Blakeney]

I started writing a long reply but no one is really going to change their minds about this so I've pared it down to just these two points.

Windows security issues don't really have much to do with file organization. The problem is Microsoft products allowing other computers to write stuff to you hard drive and then execute it without the user knowing. Malware doesn't need to be in a specific location to run, it just needs to run.

I never, even in my most disorganized state, would have come up with the design that Microsoft did with the "Documents and Settings" folder. I'm not a Windows expert but I have no idea what most of that directory is there for. Then there is the fact that there are nine "Application Data" folders inside the five "user" folders (All, Default, Jeff, LocalService and NetworkService and I'm the only user set up on this machine) and I only set up this machine in the past two weeks so D&S will program continue to get worse. Another reason I avoid "My Documents" is when sharing my drive with my laptop. If I share my C drive, I have to drill through a few more levels to get to my documents and I still need to change permissions to be able to view it unless I share "My Documents" separately. So Windows is worried about more than just its few files in the root directory and the files in the Windows directory.

[Kev Peel]

The folder configuration is quite clear for XP:

All Users - Can be accessed by programs running under any user
Default - Templates, Shortcuts, etc that will be used when creating new accounts
Current User - The current user's folder
LocalService / NetworkService - Service-specific folders

*On Vista and Windows 7 the layout is more complicated (unnecessarily so, IMO)

You may only "see" one account, but on all recent versions of Windows there are at least 4 - SYSTEM, LOCAL SERVICE, NETWORK SERVICE and the current user. Each account has various permissions, and processes are executed under the relevant account. The folder layouts are complex because Windows is complex, this is not DOS. The "File and Settings Transfer Wizard" included with Windows should help users migrate between systems.

The "Application Data" is just that, data files that the user shouldn't be able to access (or deterred from accessing). For storage of files that users are allowed to access, "My Documents" can be used

PS. "Map network drive" is your friend

[John Petty]

Whilst it all sounds reasonable there has been no answer to the purpose of the hidden folder "Local Settings" under which is another folder "Application Data" compared to the "Application Data" folder which all here seem to be recommending.
To me there is a problem with MS's current thinking i.e. Office Outlook stores the personal folders PST in the following location
"Documents and Settings\user\Local Settings\Application Data\Microsoft\Outlook\". If Outlook has a problem it tells you to repair it with the provided Inbox repair program "Scanpst.exe", however as the PST file is under a hidden folder you have to disable the supposed security system in order to be able to find it to repair it. I fail to see the logic, as it also makes it virtually impossible to do simple backups of data most business people consider one of their most important data files.

[Jeff Blakeney]

Kev, I was using "Map Network drive" for a while and logging on to my laptop so I could access the "My Documents" folder there but I didn't want to leave it mapped all the time because my laptop isn't powered up much and I found that Windows Explorer kept forgetting what user I logged as last.

I found it much easier just to open the "My Network Places" and it would remember network drives I've visited before and list them there so it was a lot quicker to get to.

[Kev Peel]

Jeff,

I know this a problem on XP Home but Windows 2000, XP Pro and Vista should remember the network login (even between reboots). I have drive X: mapped on all my PCs (4) to my PB source library folder and only a laptop with XP Home was problematic. I never did find out how to get it to remember the account password. The others were fine though.

Maybe NET USE would help? I know a shortcut with a link to the \\computername\sharedfolder can be a substitute for a mapped drive.