Corsair Consulting LLC

So you want to use Genetic algorithms to do Forex trading, eh?  Where should we start?  I'm going to use the specifics of JGAP (since that is the platform I've picked for my development) but the concepts should work with any genetic structure.

Let's start at the very beginning:  what exactly is the basis of what we are trying to implement? Is it the case that:

1. we have a specific trading system or set of indicators that we think will do the trick when tweaked just so?  In this case the goal is to evolve a system with optimal parameters.
2. we don't really have a specific system in mind.  We would then have a completely naive system, which starts from a large set of indicators and builds its own trading methods.

The first case, that of a specific trading system which is to be optimized, is the simpler case. Let's plan a Chromosome for a trivial trading system.  Define the trading system Genes as:

1. a Long position is entered when a shorter-duration moving average crosses above a longer-duration MA, AND
2. RSI rises from below a certain threshold to above a certain threshold
3. No short positions will be taken
4. position is closed when a fixed number of pips are gained, OR
5. position is closed when a fixed number of pips are lost from the maximum advancement of the position (a trailing stop)

Gene 1 implements the cross of moving averages.  How many variables can we extract from that operation?

Really, it seems as though this should be something that is evolved on on its own. For a system of sufficient complexity the combinatorial expolsion of possibilities means that it would take much longer to find an optimal solution for any single gene within the whole.  So let's split this task out into a separate Chromosome.

Let's define the Moving Average Chromosome as having the four genes above.

Evolution of this should be tested by many runs through the Open Forex Platform backtesting framework.

But what is the fitness function? See The next post for that!  :)

More Googling has uncovered 2 new ways to invoke Java/Scala code with MetaTrader. Both are commercial products but licenses are reasonably priced.

The first takes an approach similiar to the one demonstrated in this blog posting.  The name of the product is JavaDllBuilder.  It compiles Java code into DLLs but uses a JNI translating layer to make the code available. 

The second is more interesting.  It is called DDE for Java, a DDE client for java (naturally!)  Using the approach of a Java program accessing MetaTrader through DDE I think I can take advantage all of the heavy lifting done by MetaTrader (indicator calculation, actual communication with the broker) and do all of the fun work in Scala!

Edit:  crap -- DDE is being dropped from MetaTrader 5.  Scratch that option!

This is a follow-up to the post A Toolchain for Integrating MetaTrader 4 and Java or Scala.

Success!

Using the method described in my previous post I have successfully invoked Java code from an Expert Advisor in MetaTrader 4.  My single deviation from the original article is that I corrected the misspelling of "Interface" throughout.

I developed this solution in the following environment:

• Mac OSX 10.6
• Windows 7 Professional running under VirtualBox on the Mac
• JDK 6 (1.6.0_15)
• Eclipse Galileo (any IDE will do, or none at all)
• Visual Studio 2005, SP1 for Vista and Windows 7
• MetaTrader 4.00 Build 225
• IKVM 0.40.0.1

Here is my method:

First, follow the steps as directed by this CodeGuru.com article.  My tutorial assumes that you have already successfully completed all of the previous steps.

I created a simple Java project in Eclipse called MetaTraderImportTest.  I then created a package com.corsairllc.mt4Import.  You should use whatever you like.  Keep in mind that the package declared here will be used as the namespace in the CppStdcallInterfaceWrapper.cpp file.  The I created a new Java class within that package called JavaClass with the following code:

package com.corsairllc.mt4Import;

public class JavaClass {
public static byte[] Hello(byte[] name) {
String retval = new String(name) + ", hello from Java!";

return retval.getBytes();
}
}

I know that the method name violated method naming conventions but I am trying to make this class a drop-in replacement for the C# version used previously.

Now export a JAR of the project. I called mine JavaClass.jar (but in retrospect JavaAssembly.jar may have been a better name)  You do not need to include the source.  Obviously, for such a trivial Java class an IDE is not necessary.  But I assume that if you want to code Java in MT4 then your code will be non-trivial.

Copy your JAR file to the same directory to which you copied the DLLs generated in the previous steps (probably the C:\Program Files\MetaTrader 4 directory).

If you haven't already, download IKVM, extract the zip file file (ikvmbin-0.40.0.1.zip) and put its BIN directory into your PATH.  The BIN directory contains other DLLs that you will need to reference later.

Compile the .NET dll from your JAR at the command prompt with the following command:

ikvmc -target:library JavaClass.jar

This will create a file called JavaClass.dll.  This DLL will have dependencies on the other DLL files from the ikvm\BIN folder.

Now we must modify the Visual Studio solution created in the previous exercise.  Edit the CppStdcallInterfaceWrapper project, adding a Reference to the JavaClass.dll file that is in your C:\Program Files\MetaTrader 4 directory.

Next, add a Reference to IKVM.OpenJDK.Core.dll.  This file is found in the IKVM\BIN directory.  For both of the References, make sure that all of the Boolean Build Properties are set to True (Copy Local, Copy Dependencies, etc).

Open the CppStdcallInterfaceWrapper.cpp file.  Comment out the dll import for the C# assembly and update the namespace as follows:

//#using "CSharpAssembly.dll"
using namespace com::corsairllc::mt4Import;

around line 34 or so, update the code that loads the C# class and change it to read

array^ char8ManArr =
JavaClass::Hello(nameManArr);


Now you can build the solution.  On the first build you will see in the Ouput window the other IKVM dll files being copied into your project.

Copy all of the DLL files from the debug folder where the build files were placed, into the C:\Program Files\MetaTrader 4 directory.  I have not tested whether the other IKVM DLL files are actually needed for this solution to work. Copying them will not hurt though.

Run the EA created in the CodeGuru tutorial, make sure to Allow DLL imports.  You should get the following popup:

My Google-fu is strong... 

I believe I have found a way to invoke Java and Scala code from MetaTrader 4. It is not pretty but it should work.  I have not actually tested this -- this is just my thinking out loud after some searching.

1. The starting point is a compiled JAR file... either Scala or java code will do.
2. IKVM (http://www.ikvm.net).  I think this project is a product of the Mono people.  It is an implementation of the JVM designed to run under Mono/.NET.  The bytecode from the JAR file is compiled to some format that runs on the IKVM JVM runtime. This glob of bits (a dll) is then made available as a .NET assembly.
3. This little article on CodeGuru.com explains how to make a .NET assembly available as a plain old importable DLL.  It is written from the context of importing .NET code into MetaTrader.

The third step of that process would be unnecessary using NinjaTrader since it natively supports .NET stuff.

I guess the big thing is what level of JDK is supported by the IKVM JVM.  The webpage claims JSE 6 compatibility with the exception of AWT/Swing and Security classes.  Neither of those will be an issue for this project.

Edit 17 Oct 09 : Here is my followup implementing this mechanism.  Success!

Here are some of the platforms available for automated Forex trading.

MetaTrader 4: (http://www.metaquotes.net/metatrader/)

MetaTrader has a scripting language for automation, MQL4.  It allows loading foreign code via Windows dlls.  There is a lot of tool-chain distance between Scala/Java bytecode and a Windows DLL but my Google-fu seems to have found a path.  Look here for a subsequent post describing it.

MT4 has a big advantage in that many Forex brokers use it for their trading platform.

NinjaTrader: (http://www.ninjatrader.com/)

NinjaTrader automates via a .NET-derived language (C#).  Its  automation language seems capable of importing outside .NET assemblies.  The tool chain for MetaTrader involves making the JVM bytecode available as a .NET assembly.  That means that this route would work with the tool chain I found for MT4 but with one less step.

NinjaTrader works with less than a dozen brokers but the ones it works with tend to be very reputable.

Tradestation: (http://www.tradestation.com/)

This gets good reviews but is limited to the TradeStation brokerage...

QuickFix protocol (http://www.quickfixj.org/)

FIX protocol is a standard for all kinds of financial maneuvering.  There is a Java implementation.  Basically, my dream setup for this project is

1. QuickFix/J with a Scala Wrapper
2. Compose a Domain Specific Language (DSL) in Scala
3. ???
4. Profit

FIX protocol is supported by a few brokers but they are the majors: Oanda, Dukascopy, Interactive Brokers, etc.  The problem is that because you are dealing with the major players, it requires a not-insignificant outlay of money (in account minimums and monthly fees) during the development process.

The cost probably makes this route a no-go, for now.  But just for now...