Traders' Roundtable

Any regular users out there?

L

I have been using Octave, R and Gnuplot, along with a few sed scripts, for several years now for all my trading related activities.

Hi babelproofreader!

Great name
Have you read George Steiner's After Babel?

More on the topic. Do you use R & Octave for modeling or for writing trading systems? GnuPlot: use it for visualization, for trading charts or both?

Connection to TBB: do you in any way move data between them? How?

Thank you,
L

I've been using R for all of my trading model development and analysis and have been very pleased with it. One major caveat: if you use looping in your code, then as with most scripting languages, this is going to be slower than molasses. R is only fast if you are able to vectorize your code. Fortunately, it's very easy to write C code extensions, so if you know C programming, this is an easy problem to overcome (you just write a few helper C functions for the slow parts of your code). This is primarily an issue if you deal with low resolution data (e.g. 1 min or less). For daily R is plenty fast.

One other nifty program I thought I'd mention is SierraChart -- the reason I bring it up in this context is that it allows you to write C code that programmatically adjusts the position of the chart on the screen. This makes it very easy to write some trading code in R that generates your trades, then step through the trades via SierraChart, if you have a desire to visually inspect what the chart looks like at the time of the trade. There are also some user contributed R packages for simple charting, but not nearly as flexible as SierraChart.

-bbc

Hi Blueberrycake,

Since you write your strategies in R & graph in SierraCharts, what and how do you use TBB? Is R for trying out concepts and TBB for day to day operations?

Some guesses:
Do you recode everything in TBB?
If so, how do you handle scientific library, or lack thereof, issues?

Possible alternative interpretation of your note is that you write out signals from R, read them in TBB, run Money, Portfolio & Risk managers in TBB to generate orders?

Any other architectural solutions that you employ to communicate btwn to environments?

L

Hi longmemory,

Great name

Thanks

Have you read George Steiner's After Babel?

No

Connection to TBB: do you in any way move data between them? How?

I do not use TBB, or any other trading software, in any way. Frankly; and I don't mean this as a criticism of either TBB or those who have paid for and use it; I don't understand why anybody would pay for something when excellent quality and, in my opinion much more powerful and versatile, free alternatives are available.

I use sed scripts to pre-process data where necessary for Octave, R and Gnuplot. Octave is my main number crunching software which I use to calculate indicators, write trading systems, to produce actionable reports etc, for which purposes I use especially written C++ .oct functions where speed is an issue. Octave output is then visualised with Gnuplot or is used as the input to R for statistical analysis of back tests. I would add that I run all of this on a Linux OS.

bableproofreader,

Good to see other fans of R here. Just curious, what kinds of things can Octave do that R can't? And would Python/matplotlib be an equal substitute for Octave/gnuplot in your opinion? I'm approaching the opinion that R can do just about anything. However, there are a few things that Python tends to have some libraries for that R does not as yet (GAs for example).

There are a very few... but existent things that matlab appears to have that R and Python doesn't, particularly on the signal processing side. But seeing as my last license expired about 15 years ago, I've decided not to rely on matlab.

Hi Qsquare,

"However, there are a few things that Python tends to have some libraries for that R does not as yet (GAs for example)."
Yes, Python has HUGE collection of AI stuff. Almost anything to do with AIhas been re-implemented from Lisp or wrapped and runs 'in' Python. (Well, really, C...)

"There are a very few... but existent things that matlab appears to have that R and Python doesn't, particularly on the signal processing side. But seeing as my last license expired about 15 years ago, I've decided not to rely on matlab."

Octave is matlab compatible. Free. Octave is slower then matlab.

Might wish to reconsider:
Very close personal friend teaches Digital Signal Processing (grad school) and used to use matlab. The Engineering school continues to use matlab for 1st / 2nd yr grad students, but doctoral candidates all use Python. Look into
numPy
sciPy

Bigger variety DSP impls can be had in sciPy then matlab.

Speed: sciPy with Pyrex runs ten to twenty times faster then matlab.
Cython is faster then Pyrex, about twice as fast.

Real speed:
Since both R and sciPy (Cython or not) call the same Cray based vector algorithms from the '70s, '80s, & '90s, there is really no difference in speed between a well written sciPy and R language program. Both will run close to C and Fortran speeds.

(Need to turn 64 bit on, and may have to flash your bios and set compiler switches for Intel Advanced Vector Extensions. Recent Gnu gcc does support SIMD instructions.)

Real difference is in surface grammar.
R has lots of fans.
Cython allows you to write C with Python syntax.

Someone with recent R experience: is Scheme accessible from R? Used to be in S.

L

Thanks longmemory,

Your comments pretty much jive with what I suspected (or knew). So Octave/Gnuplot is essentially a substitute for matlab and variants. I've tried Scilab before, and the syntax or interface was horribly non-user friendly, IMO. In addition the supposed matlab converters (to scilab) suck pretty bad at any decent sized scripts.

Every now and then I'll come across something in matlab, however, that I know can be implemented in Python, but the conversion tends to be very time consuming and challenging. Does the Octave program have any conversion scripts that are remotely better than Scilab in your opinion?

I'm also proficient in Python (numpy/scipi incl), but find myself gravitating more and more towards R (primarily because it's just so easy to translate thoughts, and there are so many packages available nowadays) over time.
I know there is an RPY interface, but it doesn't seem all that easy to pick up or put together, IMO.

Q

"Does the Octave program have any conversion scripts that are remotely better than Scilab in your opinion? "

Octave is basically free Matlab with minor syntax differences.

Do you exchange data with TBB? Do you use them both in daily trading or one is research, other operations?

L

longmemory wrote: Since you write your strategies in R & graph in SierraCharts, what and how do you use TBB? Is R for trying out concepts and TBB for day to day operations?

I don't use TBB. Long time ago I went through phases of using AmiBroker, TradeStation and WealthLab, and while all had their advantages, I found that I do enough non-standard analysis that using a more generic analysis system such as R works better for me.

It takes a bit longer to get going than with a specialized trading analysis tool, but once you've got data model and function library set up (there is lots of stuff available for free online, and/or you can write your own), you can do all the same things and then some even faster. Of course if you have zero experience writing any type of code and have no background in basic statistics, then it's probably not a wise route to take.

One other comment on speed and optimization -- it's very easy to waste lots of time trying to optimize performance. Usually it's a waste of time, given how fast modern computers are. You should write your systems and analysis code in a way that enables you to write it quickly, and with fewest chances for error. Then if you find that if it runs slow, you take the specific piece that's slowing it down and optimize it for performance.

-bbc

Qsquare

Just curious, what kinds of things can Octave do that R can't?

What I currently code in Octave I could do equally well in R. I just happen to use Octave because I was introduced to it first and have many .oct functions and scripts already written for Octave. However, like you, I can envisage that over time I may move fully over to the R environment.

Here's another vote for R.

1) huge library of free code.

2) vectorize your problem and it is incredibly fast.

3) reasonable graphics with a little effort.

I find it easy to transfer stuff between TBB and R as both can easily read and generate text files.

I like the ease of use / reporting / graphics / debugging of TBB - it would take me forever to reproduce that in R!

I haven't really started looking at it.

I have TradingBlox - a new user - but at some point will probably move over to R when I get the chance.

Writing code from scratch isn't hard, but it does take time. Plus, I am not a great coder, so I'd make logic errors that would take a while to discover.

Did you make your own R code - or did you start with something else?

Wow for the guys doing it in R.

I can see myself doing some backtesting in R, but before to get this in an industrial way you need to do alot of infrastructure work ... I have a few question for you guys because I cannot decide between buying OpenQuant and doing it from scratch by myself since I work full time ...

Are you guys running scripts to store the timeseries ? (are you using daily or intraday data)
Are you trading manually or are you fully automated ?
So that means that you have to recode the strategy in another language in fine ?

Ideally, I would like to hear from your experience, I would like to get started but I would like to use the right tools from the start ...

I just use R for off-line analysis - it is not suited to building a trading system as it is horribly slow for loop processing i.e. working through this time interval then the next and so on - you cannot vectorize in this case because of the time-series dependencies so why use R?

Mike S wrote "Writing code from scratch isn't hard, but it does take time. Plus, I am not a great coder..."

Mike : The "does take time" part can be rather painful. I have 10 years of code in Trade Station Easy Language that I do NOT have the human resources to recode. Writing even a simple looking trading system in Easy Language, say half a page, or one page of code, and translating it to Trading Blox can take one or three weeks.

Why?
(a) I have 10 years worth of library functions and TS has an order of magnitude more (and useful) functions then TBB.

(b) There are subtle differences in semantics between platforms. Tracking down DIFFERENCES is VERY HARD.

(c) Once you apply portfolio controls the output changes dramatically. You'd need to build an automated "compare TBB to Platform X results" toolkit, else you'll be scratching your head a lot.

Similar concerns would apply to R, but with greater level of difficulty.

Blue Trin : Are you BUYING Open Quant. To the best of my limited understanding, Open Quant is FREE, open source and you can download it from code.google.com.

Is there a possiblity that you are confusing Open Quant with Smart Quant, a Russian, Windows only, commercial package (of no obvious attractiveness) ?

L

Eventhorizon wrote "use R for off-line analysis - it is not suited to building a trading system as it is horribly slow for loop processing..."

Eventhorizon : the quick and dirty approach be to get <<Revolution Analytics>> formerly Revolution R. It is a university spin off adding parallel processing to R. Given CAREFUL coding, your code MIGHT benefit from parallel execution.

Of course, writing code in a vector friendly manner would make more of a difference...

L

use R for off-line analysis - it is not suited to building a trading system as it is horribly slow for loop processing

There is the Rcpp package which allows C++ to be used from within R. This will considerably speed up loops.