See my latest episode on data intregation which discusses handling nulls in data integration:
I am writing up the following companion piece to go along with my Tableau visualization of the stock market crashes of 1929, 1987, 2008 and 2020. I was very curious about these events and found my analysis very interesting and telling.
The crash of 1929 is a valauble lesson in just how bad things can get. And this is to say, they can get really, really bad. Even without the historical context of the Great Depression, the chart is breathtaking. In this crash period, we see a world broken by the depression and later seared by World War II. By the time the Dow Jones Industrial Average regains its peak in 1954, the world has been completely and utterly transformed in every conceivable way.
When we see the massive interventions in the markets of governments and central banks in 2008 and 2020, we can look to 1929 to see just what it is they are trying to prevent.
The crash of 1987 comes on suddenly but after its dramatic start, it begins a slow and steady recovery. Of all the historic crashes I analyzed, it is the mildest. And yet, it still takes two years to complete a full recovery on the index.
The crash of 2008 comes on slowly, coming off a peak in 2007, but picks up in the fall of 2008 and begins a dramatic drop. Governments and central banks take massive stimulus action. This crash is brought on by a collapse in the United States housing market and the accompanying Great Recession. The Dow Jones average recovers its peak by 2013.
The crash of 2020 is sudden and dramatic, but then starts a quick recovery and then gets very choppy. We are seeing unprecedented market news like the West Texas Intermediate oil price dropping into negative territory and also some significant bankruptcies are being announced. This does not bode well for what is to come.
Conclusion: We are nowhere near out of the woods yet. As we can see, even in the mildest market crash (1987), it took 4 months to touch bottom and 2 years to recover. In the most severe case (1929) it took almost 3 years to touch bottom and 22 years to recover. We have likely not seen the bottom of the market, and are likely at least 18 months away from a recovery. As Sven Henrich put it:
I choose to use the Dow Jones Industrial Average for my analysis. I know that the Dow Jones has a survivorship bias, as it only includes the 30 top US companies of the day. But I used it because it is more historic, and also because for many the Dow Jones Industrial Average is the stock market. It also allows us to take a snapshot in time by seeing the components of the Dow (see below). I could have used a more broad market index but for the purposes of my analysis it tells the story I wanted to tell.
I set each stock market crash starting point at the pre-crash peak, and then used a percentage decline to show the losses against each other, counted out in days post-peak.
For historical interest, here are the Dow Jones Industrial Average components for the crash periods:
|Allied Chemical and Dye Corporation||International Nickel Company|
|American Can Company||Mack Trucks, Inc.|
|American Smelting & Refining Company||Nash Motors Company|
|American Sugar Refining Company||National Cash Register Company|
|American Tobacco Company||North American Company|
|Atlantic Refining Company||Paramount Publix Corporation|
|Bethlehem Steel Corporation||Radio Corporation of America|
|Chrysler Corporation||Sears Roebuck & Company|
|Curtiss-Wright Corporation||Standard Oil Co. of New Jersey|
|General Electric Company||The Texas Company|
|General Foods Corporation||Texas Gulf Sulpher Company|
|General Motors Corporation||Union Carbide Corporation|
|General Railway Signal Company||United States Steel Corporation|
|B.F. Goodrich Corporation||Westinghouse Electric Corporation|
|International Harvester Company||W.F. Woolworth Company|
|Allied-Signal Incorporated||Goodyear Tire and Rubber Company|
|Aluminium Company of America||International Business Machines Corporation|
|American Can Company||International Paper Company|
|American Express Company||McDonald’s Corporation|
|American Telephone and Telegraph Company||Merck & Co.|
|Bethlehem Steel Corporation||Navister International Corporation (formerly International Harvester Company)|
|The Boeing Company||Phillip Morris Companies|
|Chervon Corporation||The Procter & Gamble Company|
|The Coca-Cola Company||Sears Roebuck & Company|
|E.I. du Pont de Nemours & Company||Texaco Incorporated (formerly The Texas Company)|
|Inco Limited||Union Carbide Corporation|
|Eastman Kodak Company||United Technologies Corporation|
|Exxon Corporation (formerly Standard Oil Company of New Jersey)||USX Corporation (formerly United States Steel Corporation)|
|General Electric Company||Westinghouse Electric Corporation|
|General Motors Company||F.W. Woolworth Company|
|3M Company||Hewlett-Packard Company|
|Alcoa Inc. (formerly Aluminium Company of America)||The Home Depot, Inc.|
|American Express Company||Intel Corporaiton|
|American International Group Inc.||International Business Machines Corporation|
|AT&T Inc.||Johnson & Johnson|
|Bank of America Corporation||JPMorgan Chase & Co.|
|The Boeing Company||McDonald’s Corporation|
|Caterpillar Inc.||Merck & Co.|
|Cheron Corporation||Microsoft Corporation|
|Citigroup Inc.||Pfizer Inc.|
|The Coca-Cola Company||The Procter & Gamble Company|
|E.I. du Pont de Nemours & Company||United Technologies Corporation|
|Exxon Mobil Corporation||Verizon Communications Inc.|
|General Electric Corporation||Wal-Mart Stores, Inc.|
|General Motors Corporation||The Walt Disney Company|
|3M Company||JPMorgan Chase & Co.|
|American Express Company||McDonald’s Corporation|
|Apple Inc.||Merck & Co.|
|The Boeing Company||Microsoft Corporation|
|Caterpillar Inc.||Nike, Inc.|
|Chevron Corporation||Pfizer Inc.|
|Cisco Systems, Inc.||The Procter & Gamble Company|
|The Coca-Cola Company||Raytheon Technologies|
|Dow Chemical Company||The Travelers Companies, Inc.|
|Exxon Mobil Corporation||UnitedHealth Group Inc.|
|The Goldman Sachs Group, Inc.||Verizon Communications, Inc.|
|The Home Depot, Inc.||Visa, Inc.|
|Intel Corporation||Walgreens Boots Alliance, Inc.|
|International Business Machines Corporation||Walmart Inc.|
|Johnson & Johnson||The Walt Disney Company|
“Some are born great, some achieve greatness, and some have greatness thrust upon them” – William Shakespeare, Twelfth Night
There is only one constant in technology, and that is change. Change came to me on the heels of coronavirus a few weeks ago and like the flip of a switch, the times had changed. I have been here before: in 2017, in 2009, in 2001 and in 1995, I was caught up in sweeping change. But in change comes great opportunity, a chance of renewal, a chance to reinvent, a chance to be reborn.
I have always loved technology, and my how the times have changed. My first home computer was an Atari 800XL and was Atari’s answer to the Commodore 64. I would tinker with code in Basic, attempting to write simple video games and such. In high school I would upgrade to an IBM 286 with the at-the-time incredible memory of 640K. It is funny now to picture what we considered advanced computers of the day. I remember being amazed by the first computer I coveted that had one thousand kilobytes in memory, the Atari ST. To put that in perspective, my current laptop has 16 million times more memory.
I went on to business school, and to a job in the oil patch where I learned about databases and SQL code and how to watch a business go down the drain. I was laid off in round six. The department I had worked for went from 100 employees eventually down to 2. Shortly after the firm was bought out. So it goes.
Keen to move on from the oil patch, I briefly considered law school when a lawyer mentor told me “You must love the law.” Not loving the law, I went back to computers, the love of my career. I picked up a job in 1999 with a company called Newcomp Solutions in Toronto, where I learned about Cognos and data warehousing and ETL development. It was the heady days of the internet boom, and I had arrived.
Since those days I have seen continuous change and innovation, from Cognos series 5 to Cognos series 11, from Microsoft SQL Server 2000 and Data Transformation Services (DTS) to Azure and Power BI, from Informatica to Talend and Alteryx, to Tableau, to Big Data, to Cloud, to ever bigger and ever faster. I became a Cognos expert and a technical instructor. Along the way I picked up data strategy, data governance, enterprise architecture and broader data management concepts, at a variety of employers and clients, until I arrived where I am today, an independent consultant in data management consulting and technical education.
And now as we stand amidst a global pandemic, wondering what will come next, seeing what I have seen, I can tell you what it will be.
Ever more change. Bring it on. I am ready.