Hi guys, Andrea Unger here.
Robustness of a system!
This is something often debated in the industry and it’s a bit in line with the myths or stereotypes to destroy because sometimes there are excessive thoughts about the evaluation of the robustness of a system and there is the idea that this might be something extremely algorithmic or so.
Well, there is a part of the truth, but sometimes there is also something that is not properly explained, so let’s try to clarify the concept.
First thing, there are many tools out there permitting to evaluate how stable or robust the performance of a system are, based on the choices we made of the parameters used in the system.
We have inputs, we choose the inputs after the optimization and these choices lead to the performance of the system.
There are tools, useful tools, that permit you to move around the levels you chose for these inputs and to verify how eventually the performance of the system changes.
These tools are useful for sure, but if you think about that, we should avoid the cherry-picking of some values.
Here we have two categories of developers: the tricky developers, those who are sort of scams, who build on purpose something that looks very nice but they already know that it is absolutely unstable, because they really chose the parameters inputs of the system so that the performance looked nice but in reality they knew that it could not last the real-time challenge; the second category of developers are sort of “stupid developers”, “dumb developers”, I mean no offense in this, of course, but those are cheating themselves.
If I develop something and I see that on a specific value of an input I get a wonderful performance but any other value shows a disaster, and I want to take that and I’m happy… I’m cheating myself, of course, and I try to use something that will fail five minutes later.
So, I mean, you can choose what words best fit this kind of behavior.
So, that said, the evaluation in any case of robustness, in this sense, might be useful because if you make the choices properly, you are able to have an overall scenario of the stability of your system and this is something that is useful of course, if doing so you understand that there is stability.
Another point is, this is a hidden message, what happens if the edge you base your system upon disappears?
If you build a system based on a specific characteristic of a market, or a specific edge, or on a specific something and this thing disappears, then your system is no longer good?
Let’s imagine you go to the office and you are there at 9 a.m. because you are able to catch the bus at 8:20, the bus just in front of your building.
If for some reason, like summer bus schedule changes from 8:20 to 8:35, you will arrive late to the workplace.
The environment changed and your system, to take that bus and get to work on time, failed, so there is a change.
The point in this specific case you just need to follow the information of the bus company and they will tell you when they change the schedules of their busses and you’re able to find a solution in advance.
The markets are not telling us when they change their schedules and it is our problem.
So we have to be careful to really, deeply, try to understand what we put in our systems.
We have to try to isolate the motor which is making the decisions for the trades of that system.
Why did we put these rules together?
Because we are looking for that specific edge.
If we isolate the edge we are working on, we might be able to find out in advance or at least, in time, that that edge is going or is gone.
Normally when you work on a specific edge than you add other filters, you combine other patterns, and in the end, you find something that might survive a period where the edge disappears, because of other conditions.
By the way, the higher the number of conditions the higher of the chance of an unstable system, because obviously, the higher the degree of freedom of your system the better.
When you put many conditions, you force your system into something that might be very unstable, like a beautiful castle of cards which looks very nice, but when your cat passes by with the tail everything is going nuts.
So actually, you have to be careful of it, but let’s imagine that the additional conditions help in surviving the initial phase of that market, no longer having the edge you work on.
If you have your isolated system only based on that, maybe something not tradable, but that shows the results, without any additional condition, you might be informed on time that the edge is gone and that might help, of course.
This is very difficult, it needs to be experienced and the only way we can act if we don’t do this is to create some performance indicators that tell us when to stop a system.
This opens up to a completely new world, but it’s something that is the only solution to this.
We have to get indicators of performance that are telling us when a system is to stop.
The bad news is that there is no specific indicator of performance.
You have to choose what you prefer.
You have to build something that makes sense, of course, and then choose among these.
You might be wrong, we will never be right all the times, but if you have something that tells you when to switch a system off, this is obviously ahead and it’s the only way to avoid losing a lot of money with a bad dying or ill system in your hands.
This is maybe not very scientific because you are not evaluating the robustness, you are simply evaluating the performance, but unfortunately, for this kind of situation, where we have to fight against a disappearing or a disappeared edge, it’s, in my opinion, the only way to go.
I hope this was useful for you and in any case write down in the comments if you have some specific thought about this or some concern, I will try to answer as usual and in any case, we meet at the next video.
Ciao from Andrea Unger.
Click the link below
and you will be noticed
as soon as new material is available