Foolish Workshop
Characteristics of a Good Screen
One "Fool's" Opinion

By Moe Chernick (Workshop)

EL SEGUNDO, CA (Sept. 28, 1999) -- With all the talk of screening strategies, the question always arises: How do you know a good mechanical strategy from a bad one? Of course, answering this question is a matter of opinion. There are many, particularly among the "Wise," who think that there is no such thing as a good mechanical strategy.

Then there are those who never met a screen they didn't like as long as the returns were great. I take a more middle-of-the-road point of view. I invest in mechanical strategies because, since 1991 when I tried my first "Beat the Dow" screen, my best returns by far have been from investing in mechanical strategies.

However, I am also aware of the critics and do not dismiss their arguments out of hand. Therefore, before I invest in a screen, my number one interest is determining the probability that the screen will work in the future. To me a strategy that has "returned" an average of 30% per year based on backtesting is a much better strategy that one that has "returned" an average of 50% IF the 30% strategy is more likely to continue generating those returns.

So how do you determine if a strategy will be a future winner? To help me make a determination, I examine each strategy's stock screening protocol according to my five rules for a good mechanical strategy. This is only one Fool's opinion. Just because a screen adheres to these rules doesn't guarantee that it will work through the ages, although I do think it increases the likelihood that it will.

Can a screen work that doesn't pass all these rules? Of course it can, but for me, at least, it is too dangerous to touch. Here are the first two rules (we'll discuss the last three next week):

  1. The strategy must follow a simple and logical protocol.
  2. The rationale must make sense and be easily explainable.
Rule 1: The strategy must follow a simple and logical protocol.

A protocol is the set of steps followed to generate a stock screen. For example, the protocol for the Relative Strength (26 week) screen is:
  1. Start with the stocks ranked #1 for timeliness by Value Line.
  2. Sort by "Total Return - 26 weeks." Select the five stocks with the highest returns.
  3. Repeat these steps each month, hanging on to any stock still on the list and replacing any that have fallen off.

Simple enough. Simple rules means that the number of criteria are limited and easy to understand. The more complicated and obscure the criteria, the greater the chance that it may lead to datamining. If it takes 20 steps to run a stock screen, then there is good reason to be suspicious.

I would also be suspicious of complicated formulae. For example, if a screen has a step that states, "Select only stocks where After-Tax Cash Flow minus Inventory is two-thirds higher than revenues," I'd run.

Besides being simple, the protocol must be logical. Our relative strength screen starts with stocks that are recommended by Value Line, and from that group picks the ones that have gone up in price the most over the last six months. Since Value Line has already checked the financial soundness of these companies and since good stocks that go up tend to continue to go up, at least for a while, it's not hard to see why this screen works so well.

Now suppose the protocol said "screen for x, y, and z and buy the third ranked stock." Why the third and not the number one or number two stocks? If the answer is that the third-ranked stock had the highest average return over the last 10 years, uh.... sorry. That doesn't cut it. That's like investing because hemlines are up.

Or suppose you see a criteria like "Inventories/sales ratio is greater than 23.7%." Why is 23.7% chosen? Why not use 20% or 25%? Maybe that cutoff point is the spot where you manage to include a really wonderful stock one year and exclude a really bad one. That can do wonders for your average return numbers -- but only in retrospect.

These are red flags that scream "datamining!"

Rule 2: The rationale must make sense and be easily explainable.

If a screen doesn't make sense to you, don't invest in it. If you can't easily explain why it works, then you shouldn't use it. Don't get "Wise" disease and think you are not smart enough to understand the logic behind the screen. If you don't understand why it works, ask questions. If you still don't understand, don't use it. But I go even one step further by saying if you can't tell your best friend why it works, don't use it.

The more complicated the rationale, the more likely that the high returns are the result of datamining. For example, I can tell you why 12 different factors are all signs of a good company, and it might even make sense that, combined in a certain way, you can come up with a screen that scores big when backtested. But what is probably happening is that certain variables were manipulated, consciously or unconsciously, to either include a few terrific stocks or exclude a few really bad ones. Would a combination like that really repeat in the future? Not likely.

Next week we will take a look at some tougher rules. Meanwhile, what do you think about the rules above? We welcome discussion on the Foolish Workshop message board.

New Rankings | Workshop Returns