I scratch a living trading stock index futures on my own account. Over the years I have tried just about every piece of serious software/system available to the small professional retail trader. All have issues which render them less than optimally useful for the sort of trading that I undertake. My biggest and perennially ongoing issues concern the collection, cleaning, storage and post-collection processing/manipulation of large 'raw tick-data' files. I currently collect data from 2 separate sources and have giga-bytes of of the stuff. My problem is getting it into various forms that can be used by several of the more useful power/AI/GA analysis packages that I use.
Briefly, it is a doddle to get standard timed Open/High/Low/Close/Volume/Trades data; most 'trading software' processes real-time raw data into that form as standard. What is NOT such a doddle is to construct Volume/#Trades/Range bars which include both #trades and volume at Bid and Ask and correlated values for the content of the order book either side of each last traded price during the construction of said bars. I have finally decided that the only way I am going to get what I want from this data is to programming it myself.
So what are we talking about? Well it sounds simple enough (doesn't it always?). And, to the extent that the bulk of what I want to do is simply a matter of parsing flat text files, it probably is to an experienced programmer. Problem is my programming experience is limited to PAL (Paradox Application Language) - for DOS!! which, although extensive, ended over 15 years ago. I have just about settled on Power Basic because it is used by the authors of some of my more useful analysis packages to program various add-ons and indicators.
If any of this rings bells with anyone I'd be grateful for observations, questions and/or advice to a rank beginner before taking the plunge.
Briefly, it is a doddle to get standard timed Open/High/Low/Close/Volume/Trades data; most 'trading software' processes real-time raw data into that form as standard. What is NOT such a doddle is to construct Volume/#Trades/Range bars which include both #trades and volume at Bid and Ask and correlated values for the content of the order book either side of each last traded price during the construction of said bars. I have finally decided that the only way I am going to get what I want from this data is to programming it myself.
So what are we talking about? Well it sounds simple enough (doesn't it always?). And, to the extent that the bulk of what I want to do is simply a matter of parsing flat text files, it probably is to an experienced programmer. Problem is my programming experience is limited to PAL (Paradox Application Language) - for DOS!! which, although extensive, ended over 15 years ago. I have just about settled on Power Basic because it is used by the authors of some of my more useful analysis packages to program various add-ons and indicators.
If any of this rings bells with anyone I'd be grateful for observations, questions and/or advice to a rank beginner before taking the plunge.
Comment