Overview Of Asynchronous Tools in C# and C++

Asynchronous programming is satisfying, it feels good to write code that doesn't block.

The tools to do it have evolved a lot over the past few years, both in C# and C++.

Adapter And Dependency Injection Without The Pretty Diagrams

How I used Adapter, Singleton and Dependency Injection (in real life, without the dry diagrams).

Confidently Assert Fallacies with Statistics, Surveys and Luck

You can be as rigorous and thorough as you like, with a bit of luck you can prove relationships between things that don't actually exist.

Java vs C++ vs C#: What Is The Best Programming Language?

From Akihabara in Tokyo how to choose a programming language?

Hide legacy C++ APIs behind WCF to simplify deployment

The problem

We are using a number of in-house C++ APIs for various systems (static info repository, Excel calculation libraries, internal market data repository, configuration repository). Those APIs were originally designed to be run in Excel spreadsheets and date back to a time when .NET didn't exist.
Some of those APIs now come with 50 or more native Windows DLLs and sometimes require installation of database drivers on the client machines.

To use them in a .NET solution you have to write C++/CLI wrappers. Interop is not enough because APIs have complex data types so you must write the marshalling code in C++/CLI yourself. Adding mixed (managed/native) libraries to .NET solutions makes builds slightly more difficult to manage.

If you are creating Winforms or WPF GUIs that happen to use one of those APIs directly you have to deploy those native DLLs or ODBC drivers to users machines. It's just painful and ClickOnce doesn't really like that.

Proposed solution

Hide those APIs along with their native DLLs and db drivers behind WCF endpoints. Deployment complexity is limited to the server. All the clients have to do is make a WCF call.

This brings other problems though:

• how to avoid building a single point of failure whereby if one API call gets stuck then all clients trying to access the service are stuck.
• how to manage user's credentials to apply proper authorisation? (because the APIs are now called from a server).
• how to manage user's session state? (an example of state is in-memory cache which is sometimes maintained to increase performance between API calls)

In our case it turns out that some of those APIs

• have state. They require to connect and disconnect (create an environment for the duration of the user's session then destroy it). 
• are not thread-safe: they were originally designed to run inside single-threaded clients such as Excel and do not have a predictable behaviour when several instances of the API are created in different threads inside the same process. 
• can be very memory-hungry with long-running operations and direct access to various databases (Oracle, SQL Server). Because it's still 32-bit code, the hunger for memory can be a problem as we've seen situations when passing arguments in bulk to an API call breaks the 2GB memory limit.

WCF allows you to have a per-session instantiation mode, which sounds good here: it makes sense to create an instance of each API for each client WCF proxy. However we really can't run all those instances inside a single process with multiple threads. The code is not thread-safe and because it's 32-bit we would quickly run out of memory.

WCF automatically manages creation of a thread pool to run services concurrently but it doesn't manage the creation of separate processes, which is what we want to do here really. The only way to ensure the native APIs instances will not create side effect is to run them in the same way they were designed and tested: one per process.

How to do that?

Let's say the contract's interface is called IMyContract.

You could create a dispatcher service implementing IMyContract with binding netHttpBinding for instance. This would be addressable from any host in the company.
Then the implementation of IMyContract would just redirect the calls to other services, hosted on the same server in different processes.

The dispatcher service would be configured with InstanceContextMode = PerSession to have one instance created for each client proxy.

Every time a DispatcherService instance is created, it would spawn a process. That process in turn would create a WCF endpoint with binding netNamedPipesBinding. Named pipes would do fine for low overhead interprocess communication on the same machine.

Because there is only one dispatcher service instance per user session, there would also be one process per user session. This would ensure that those native C++ non re-entrant client APIs would be completely isolated from each other.

(To be continued...)

Algorithmic Trading Resources

Books

• Numerical methods in Finance with C++ very concise and programming-oriented. You have to be familiar with the methods before reading it (binomial method, finite difference, implied volatility with Black Sholes...)
• All About High Frequency Trading very unpretentious book, easy to read. Gives practical tips about the physical architecture of an automated trading platform.
• C# for Financial Markets very dense. Not much prose, many bullet points. Describes patterns for bond pricing, binomial method, finite difference method, interaction with Excel, C++/CLI... 
• Financial Instrument Pricing using C++ (review by QuantStart)
• Top 5 Essential books for algorithmic trading (QuantStart)
• Paul Willmot on quantitative finance
• Architects of Electronic Trading (SafariOnlineBooks)
• Encyclopaedia of Financial Models (SafariOnlineBooks)
• Large reading list from QuantStart

Overviews

• NumericalMethod courses the slides give a rough overview of the tools/strategies used for algorithmic trading.

Strategies

• Cheat sheet for algorithm trading (PDF) Deals with margin accounts.

Sites

• Quantopian algorithmic trading platform: web app that allows you to create algorithms and back-test them using historical data.
• Quantstart has a list of free resources here.

Open Source Projects

• Algo Trader: written in Java, there is a an open source version and full featured enterprise version. The architecture diagram is interesting.

Technology

• Trading floor architecture diagram from Cisco 
• Large file storage: HDF5
• High speed time series databases: kdb, OneTick, Reuters Velocity Analytics (formerly Vhayu)
• Big data in banking: how four financial giants crunch big data.

Courses

• Computational investing 8-week online course on coursera.

Learning Japanese

What can you do when you take the tube? Read the free version of the Evening Standard or learn Japanese with flashcards.

Same as for Chinese I use cram.com so store my vocab.

I started re-arranging the vocab list in 3 columns, separating Kanji from Romaji. like this:

You can import this format into cram.com which supports 3-sided cards.

Then you download those cards to Flashcards Deluxe where you can choose which side to use for the prompt: either English or Kanji.

One backup is never enough...

I had trouble with my backup lately when switching to a new MacBook. I tried restoring the data to the new machine as a test.

The WD Passport software restore failed (it restored a 6-month old version of my files for some reason). Since the WD Passport software doesn't have a backup tool for Mac, I tried to configure it with Timemachine instead. The WD Passport USB drive died. I reformatted it from Windows: impossible to write to it any more. Surface tests with the WD diagnostic tools failed. It probably didn't like the whole traveling thing.

The Mozy restore worked fine.

So here is the new setup:

Off-site backup

• Automatic backup to Mozy using local encryption with passphrase. 

On-site backup

• Automatic backup to a USB drive with TimeMachine and Apple encryption.
• Manual backup every now and then of sensitive files to a USB key protected with Truecrypt. It's good to have a simple copy of your important data. A simple, straight copy that doesn't require backup software. Because DVDs are out, a simple USB key should do.

The main disk is encrypted with FileVault. 

I'm still using Mozy but I'm considering moving to Backblaze that also offers local encryption with a passphrase that no one else knows and unlimited storage. Also Backblaze claims to be able to backup data from a USB drive as well without getting confused. I tried doing this with Mozy and it got mad, deleting all my files from the server as soon as I unplugged the drive.

Learning Chinese

I bought a couple of phrasebooks while traveling. Making out in Chinese is a fun one, the Rough Guide Phrasebook: Mandarin Chinese fits in your pocket.

I also used the flashcards by Brainscape. They contain some English > Chinese vocab (+audio), some Chinese > English, some grammar and even character radicals.

I maintain a vocabulary list in a spreadsheet which I export to flashcards using cram.com (used to be flashcardexchange) for long-term memorisation. When I'm in the tube I play the flashcards on the iPhone with Flashcards Deluxe, one of the many flashcards apps that support cram accounts. I haven't tried many flashcard apps actually. I like this one because it allows you to fine tune the spaced repetition algo. So you can set how long it takes before they prompt you with a word you got wrong. And when you get a word right you can set how long it takes before the app prompts you again to check you didn't forget.

Cram allows you to work with 3-sided cards. This is good for languages like Chinese or Japanese where you need to learn characters on top of the romanised version of the word.

For a while I listened to Serge Melnyk's free podcast as well as One Minute Mandarin from Radio Lingua network.

I gave a quick try to Rosetta Stone in a shop. It gave me the impression of being no more than a very polished overpriced phrasebook, I wasn't convinced.

So far the best tool I found is the course by chineseclass101. Courses are structured in a large number of short lessons, each lesson has a 15min podcast with pdf notes, online quizzes, vocabulary and grammar points. It's very progressive and they actually explain you the grammar from the ground up. The vocab always comes with the Simplified Chinese, Traditional Chinese and pinyin representations. The beginner lessons have 4 lines of dialog each. The advanced lessons have entire texts. You have to use at least the premium option for the material to be useful. The podcasts happen to be lively and not too scripted.

The same company that does chineseclass101 sells an iPhone app WordPower. This one is useful to learn vocab. The vocab is organised in 10 levels of flashcards of about 100 words each starting with the most basic. You can also tackle the flashcards by category.

Letting go of the pinyin

At the beginning I was learning the Roman representation of Chinese words while completly ignoring the actual Chinese characters. Then after using the line-by-line audio from chineseclass101, I realised recognising characters is not that difficult.

Pinyin exists only to give a phonetic representation of the sound for each Chinese character. However anything else than the International Phonetic Alphabet is actually misleading. It's just as lame as those English phonetic approximations you find in Lonely Planet phrasebooks. If you have access to an online resource / electronic dictionary that plays out a word out loud when you click it (such as Google Translate), then you don't need pinyin. All you have to do is read the characters at the same time as you listen to the way they're pronounced in order to associate each character with its sound and tone.

The objective is to learn to visually recognise the Chinese characters, pinyin is just a way to guess the sound associated with the character when the actual sound is not available. If you have access to the real sound, forget pinyin. By the way from my last trip to Asia I got the impression that most Chinese people don't know about pinyin anyway.

For pinyin to be useful you actually have to learn how to pronounce it with a chart like this one, that lists the 4 tones of each syllable combination.

To recognise characters it helps to know the radicals they're composed of. It helps memorisation as well. Sometimes they make sense... Here is a list of radicals together with their simplified version. Brainscape here is a good flashcard tool to learn them. You can find a fun visual way of associating radicals with characters on Chineasy.org.


Texting

I've also found out that a good exercise to train recognising the characters is to text on the iPhone using the Mandarin keyboard. The keyboard expects pinyin as input then suggests a list of matching homophone characters sorted by frequency. Choosing the right character from the list makes your brain work a little bit.

Preparing for the FRM exam

The exam for the FRM certification (Financial Risk Manager) takes place twice a year in May and November. It has two parts and you can take the exam for each part either separately or both on the same day.

According to GARP it takes somewhere between 200 and 400 hours of reading to prepare each part. So obviously you have to spread it out over a few months to make it digestible.

Topics

This is Part 1 in a nutshell:

• Foundations of risk management: some generalities about risk + CAPM in great detail
• Quantitative Analysis: probabilities, distributions, statistics, lots of regression, Monte Carlo methods
• Financial Markets and Products: futures, swaps, options, bonds... Lots of chapters from the Hull book.
• Valuation and Risk Models: this is where it gets interesting. VAR calculation, binomial trees, Black Sholes, Greeks, hedging, risk metrics...

Material

• The official material is an aggregation of chapters extracted from a variety of finance books. The FRM study doesn't rely on one single manual. This makes the reading more interesting because you navigate through publications written by different authors with different specialties. The FRM study guide gives the full list of books and chapters to read. Another guide called "AIM statements"gives the same list plus a highlight of what you're expected to take away from each chapter.

If you wanted you could absorb the whole FRM knowledge without spending a penny: just go through the reading plan available on the website and borrow the books from libraries. This is what I did while I was traveling since I didn't want to carry heavy books with me.

• Obviously having all readings in one place is much more convenient so you can buy the 4 books from GARP for $250 + shipping. This is what I did when coming back to London to prepare for the last 2-week cram.

• There is also a book called Financial Risk Manager Handbookwritten by Philippe Jorion and available in Kindle format. It's a complete textbook covering FRM Part 1 and Part 2 and broadly addressing the same topics as the 2013 GARP's study guide (but not exactly). That handbook is not mentioned on the GARP website so I guess it has been obsoleted. Anyway this is a very very convenient book because you find pretty much everything in one place and in electronic format. It is not as detailed as the readings themselves but it gives a very decent overview of what you're supposed to know. It also contains a good number of exam questions with answers. I had this book on my iPhone, iPad and MBP.

Hong Kong skyline seen from Cafe Habitu: study with free Wifi and a cool view

Doing FRM on the road

Apart from a few documents available online, 95% of the FRM reading are from books that are not available in electronic form (boooooh!). Because I was traveling and didn't walk around with a camel I relied on the following:

• GARP's AIM statements for the reading plan. 
• Libraries (in particular the Singapore national library which had most of the books I needed, the Hong Kong main library on the other hand was no help).
• Jorion's FRM handbook in Kindle format. This is how I kept doing FRM reading on the boat in Thailand, on the resort's peer in Langkawi, in the studenty coffee shops of Taipei, in Hong Kong's Habitu cafes, in the Beijing hutong and during long ferry trips in the Philippines...

Boracay 

Time Allocation

According to my logs I spent about 230 hours on FRM study and I felt like I could have used 50 more hours to really complete the material.

Rather than going through the chapters sequentially I've iterated through the readings, jumping to the next chapter when one chapter was taking too long. Sometimes a chapter was building up on a concept introduced in a previous chapter I had not completed. In that case I had to move back to the previous chapter to understand what was going on... That made the reading more interesting I think. I did 4 iterations that were somewhere between 2 and 4 weeks in length.

I took the sample exam questions during the last week of revisions, which was a mistake. I should have taken them earlier because they're very useful.

The questions are multiple-choice questions with 4 possible answers every time. Some questions require calculations (nothing too complex and beyond add/multiply/divide/log), others just judgement. You would think that doing all the non calculation questions first would be easier, well it's not the case :) The qualitative questions can be quite tricky and require some thinking even if you're familiar with the material.

The Spider House in Boracay:
read about quantitative analysis with a cocktail