Developing Frameworks for ICO Investing

In the first quarter of 2018, Initial Coin Offerings (ICOs) have raised over $3.4 billion, exceeding the total amount raised from January to November of 20¹⁷¹. This explosive growth has fueled excitement from speculators, scammers, and smart investors alike, and shows no sign of slowing down halfway into Q2. While the ICO has become the new paradigm for fundraising, its infancy means there is a severe lack of objective time-tested data-points to analyze or draw meaningful conclusions from. While we have some preliminary statistics on the successes and failures of various ICOs, it will take several years to fully explore and begin to understand the different forces which impact ICO success. It remains to be seen whether particular tokenized models will be successful, and it remains to be seen whether companies may run out of funding and need to raise additional rounds in the future.

In this post, I will discuss ways we can begin to develop frameworks around ICO investing, both at a macro-level (all ICOs) and a more detailed level (specific ICOs), by applying lessons from the Venture Capital (VC) industry, as well as tools built from a historical analysis of the philosophy behind the development of AI. These frameworks can and should apply both to individual investors, as well as larger firms looking to understand and enter the space. None of the content in this article should be interpreted as investment advice. This post was inspired by Coinlist’s article, The Current State of Initial Coin Offerings, and an assignment for Nan Li and Zavain Dar’s excellent Stanford seminar on Venture Capital, SYMSYS 161.

The ICO as Distributed Venture Capital

For the purposes of this post, I will consider ICOs as a form of distributed VC: Prospective investors must choose from numerous potential projects, evaluate their technological and financial value, and ultimately choose whether or not to support a project with an investment. Rather than limiting these opportunities to larger investors, the ICO leverages crowdfunding, and is designed to be accessible to a broader audience. Due to the uncertain regulatory guidelines around ICOs, however, citizens of many countries are restricted from participating in ICOs unless they are accredited investors. I will also set aside issues of regulation for this post, and leave the question of whether the tokens created during an ICO are to be classified as securities to be answered by legal professionals. This has also been covered in depth by Coinlist’s informative article linked above.

With the help of this Dilbert cartoon, it is important to note that ICOs differ from VC several important ways:

1. ICO investors are traditionally not buying shares (or a stake) in the company; rather, they are buying tokens which are to be used in the company’s ecosystem (whether as a security, financial instrument, or utility token). While there are exceptions, i.e. Siafund, the majority of ICOs raise funding without releasing company equity.
2. ICO investors traditionally receive liquidity much faster than traditional VC. For a traditional VC, most companies take at least 7–10 years before reaching a point of liquidity where investors can realize a return on their investment. While ICOs are beginning to move in this direction, with longer lock-ups for institutional and pre-sale investors, the time-to-liquidity is still substantially shorter.

While these differences are substantial, they do not invalidate the underlying philosophy that ICOs are best viewed as distributed VC. Rather than playing a role exclusively via funding, VCs must now bring more to the table to receive allocations for the most competitive projects — connections, resources, and partnerships are king in the current landscape. With the widespread availability of funding for quality projects, early investors must now offer more than just cash — and this is where VCs can excel when they find promising projects.

In the Darkness of the Cave

As more established firms look to move or invest into players in the cryptocurrency space, there remains a deep chasm between the frameworks used by established institutional investors and the current state of the cryptocurrency ecosystem. Most companies are in very early stages during the ICO process, and have little more to show than a whitepaper and a team; at best, maybe an alpha or beta product. Like traditional VC, the challenge is to separate out the noise and pick winners at this early stage. These winners can then be guided toward, and given the resources to achieve, success. Unlike traditional VC, there is minimal historical or industry data, and cryptocurrency projects have not yet found their own Silicon Valley — there is no proverbial Sand Hill Road, and no established “hub” for these global projects.

The ICO is yet another escalation of a growing long-term trend across VC: The commoditization of fundraising rounds. As more VCs vie for the same promising companies, the race to secure a position in a promising company early drives larger investments in earlier rounds. While companies in the mid-2000s would raise $10–15 million in a Series B or C round, these rounds now have much higher raises, while Series A has become the new Series B, at least for fundraising. Higher early rounds lead investors to find their portfolio gems at even earlier stages, and this requires more risk-taking, as companies are younger and less-established at these earlier rounds. The ICO is the next step in this process: ICOs are raising $15–50 million, yet often the company maturity is closer to a seed or early Series A round. While it remains to be seen whether companies that did ICOs will raise additional rounds in the future, the current trend seems to be to raise sufficient funds all at once.

In addition to the higher initial rounds, to build a portfolio of high-quality blockchain-based companies requires a global reach, as well as a clear thesis about the industry itself. Moreover, within the blockchain industry, there are numerous verticals, each with their own winners and losers. As an investor or VC, it is thus essential to not only build proper position sizes to support companies and hedge risks, but also identify and select promising verticals within blockchain technology. It is insufficient and unwise to simply “invest in blockchain” because of the wide array of sectors under the same umbrella. This is a complicated problem, made even more difficult by uncertainty, volatility, and the early stage of the industry — to begin to tackle the problem, it is necessary to develop short-term and long-term theses which guide investments.

Within this context, we want to define a set of rules and guidelines, a “function,” which takes an ICO as input, gives an investment decision, amount, and timeline as output. To do this, we need to build an understanding of what a “successful” ICO looks like, and then boil this success down to a series of if-this-then-that statements which could help us make investment decisions. This is similar to a canonical problem in artificial intelligence — finding whether a picture contains a cat, given a random picture as input².

Platonism and Nihilism: AI as a Model

To accomplish our “cat-finding” mission, we must be able to recognize and describe rules for determining if something in a picture is a cat. Moreover, we cannot use descriptors such as “cute” or “fluffy” without clearly defining them. Nor can we describe a cat by its taxonomy or smell — neither of these things help us answer the question when given a picture. To accomplish this task, we might sit down and conjure up an image of the “perfect” cat — the Platonic form of Cat. How does it look? What defines it? Maybe you started with descriptions like these:

• A cat has four legs
• A cat has pointy ears
• A cat has a tail
• A cat has whiskers
• A cat has paws

… But in the middle picture, we can see only one of these five “defining” characteristics despite us knowing that it contains a cat, and in the third, we might mistakenly think that the picture contains two cats! How can we do better? This was the problem that early AI researchers faced.

With more advanced algorithms and machine learning tools, we can now create ways for computers to “learn” what a cat looks like. In fact, we may not be able to explain the features that the computer learns when classifying an image as “cat” or “not cat.” Many of these features may take place at the pixel-level, and may not be meaningful for a human to classify a picture, despite being very helpful for computers to do so. This is a Nihilistic, results-driven approach — as humans, we do not care how the computer identifies a cat, so long as it is correct. This unexplainable process is viewed by some as deeply unsettling, while it is championed by others as evidence of the power and success of AI and machine learning tools. The EU’s “Right to Explanation” regulations directly address this point, banning fully automated and unexplainable decision-making when it impacts real people.

While AI has made great strides in cat-finding, replicating the approach to machine learning in other industries is a difficult challenge. Most notably, cat-finding requires a great deal of training data in order for an algorithm to learn about what is a cat and what is not a cat. Without sufficient data, this would not be possible.

What have we learned from this example? In fact, “cat-finding” is an analogy for a substantial portion of what VCs do today — in a space with lots of noise, uncertainty, and constantly changing landscapes, VCs are attempting to find their “cats:” Successful companies which will become portfolio-leaders. Unfortunately, VCs, unlike cat-finders, generally do not have access to large amounts of useful data to use in their decision-making processes. They are thus reduced to the earlier method of cat-finding: Defining heuristics, based on “ground truths” they believe to be important, to identify characteristics and features of successful companies, then using those heuristics to evaluate each new investment opportunity they come across.

What is a Cat? Defining Success

ICOs take the idea of cat-finding to an even more challenging level: Currently, we do not know what a cat (or success) looks like! Before we can attempt to predict any success, we need to know what we are looking for. We have short-term information about successful projects, but even the earliest ICO, Mastercoin, has faded into obscurity over the last 5 years. For the typical VC, this is only half the lifespan of a fund! Even quintessential success stories such as Ethereum are now facing a barrage of new competitors, offering new solutions to smart-contracts, scalability, and trustless transactions. No doubt, Ethereum’s 1500x return to date from its ICO should be defined as a resounding success, but most projects do not even come close to achieving these massive returns, and many fail before even launching anything.

In the world of ICOs, then, how should we define success? Is it simply returning an amount greater than invested? Is it that the coin never trades below ICO price? What time-horizons should be used to determine success? Arguments could be made for countless different answers, but ultimately defining success depends on both perspective and risk. Consider the following challenges:

• Success for investors may not mirror success for projects, especially in the long-term. For investors, ROI is the primary (and often, only) focus, whereas for projects, building and releasing a successful product may be more important, especially for long-term growth.
• Given that many ICOs fail, success for an investor must offset the risk of failure, as well as counteracting other promising investments which ultimately failed. Like other industries dominated by startups, we can expect the majority of projects to fail, so the ROI for success must offset these losses for investing to be worthwhile.
• Success in ICOs can be measured on many different time-frames thanks to token liquidity, and each approach to measuring success has many tradeoffs. For example, did the project increase in value at initial exchange listing? After 1 month? 3 months? At its all-time high? Is the project still in development? Are road-map goals being met?

A case can be made, in fact, for success criteria to be specific to each investor. Everyone has their own risk tolerance, expectations, and philosophies toward investment. While 1500x cases such as Ethereum would be included in everyone’s lists, projects that increased 20% or 100% over 1 month or 1 year may or may not be. Similarly, an ICO which trades at 5x but does not ever (or is extremely slow to) develop a product may not be an unequivocal success.

I will share my personal definition of success and its predictors below, but keep in mind I am not here to argue for any singular definition of success; rather, I encourage everyone to build their own robust definition of success which could be programmed into a computer using simple rules. Just like cat-finding, success-finding requires a clear understanding of what success looks like before you can even begin to build a reliable framework. Only once you are able to define success can you begin to analyze or understand what factors may impact it.

For me, as an investor, my general definition of success involves financial returns. As a young investor with (very!) limited capital, I have a higher risk-tolerance and longer time-horizon than many others. My goal is to pick projects that are likely to be successful in 1–2 years which I believe will contribute to the blockchain ecosystem. Because of the high failure-rate of start-ups broadly, I diversify my capital across numerous projects — as a consequence, to expect positive returns, I look for projects that I expect, if successful, to provide at least 5–10x returns. This framework is not intended to be a boiler-plate, and it will likely evolve over time, but it provides a more concrete definition for “success.”

Cat-Finding in ICOs: My Framework

Now that we have tentatively defined what success looks like, the question then becomes, how can we predict this success? This is essentially the same process as cat-finding, but instead of our input being an image, it is the ICO’s information.

For me, to be a successful in 1–2 years and achieve an outsized ROI, a project must achieve all of the following. The sub-bullets are potential inputs to each feature, with some impacting short-term ROI, and others impacting more long-term ROI. Since my primary focus is the latter, I naturally weight those factors more highly.

• An active and supportive community
  • How is expert sentiment? Twitter sentiment? Newbie sentiment?
  • What is the quality of discussion in their communication channels?
  • How often do team members share updates and project details?
  • Did they do an airdrop? To whom?
  • Does the community evangelize or promote the project?
  • How is the project’s marketing and public communications?
• A clear and ongoing use-case and/or utility for any token/coin involved
  • Why is the token/coin necessary instead of using BTC/ETH/Fiat?
  • What incentives are there to hold the token/coin?
  • What is the supply of the coin, and how does that change over time?
  • How compelling is the use-case versus other alternatives?
• A clear reason for utilizing blockchain over other existing solutions
  • Does the problem need decentralization?
  • Does the problem need a secure, unalterable record of transactions?
  • Does the problem have an easier or less-onerous solution?
  • Has someone else solved the problem using something non-blockchain?
• Large SAM and a clear plan for achieving market penetration
  • Is the ICO’s vertical positioning in the market lucrative?
  • Will the current SOM be sufficient to sustain the project for 2–3 years?
  • What is a reasonable market-share for the industry?
  • How have non-blockchain competitors performed?
  • How have blockchain-based competitors performed?
  • How will the project onboard new customers and capture competitors’?
• Regular Github updates and improvements
  • Is there active and ongoing development?
  • How old is their Github? How long have they worked on the project?
  • Are updates exclusively made by 1 person? The entire team?
• A growing, well-qualified and/or accomplished team
  • Where did the team work before joining the project? Is it verifiable?
  • Is the team academically qualified? Technologically qualified?
  • Have new advisors or team-members joined after the project began?
  • What have team-members accomplished before starting the project?
  • How big is the team? How many developers?

Additionally, I believe most successful projects have at least two of the following qualities:

• Meeting or exceeding projected roadmap deadlines
• Regular product updates and/or press coverage
• Working product or prototype

These criteria are quite similar to what many VCs examine when evaluating any startup. When projects meet these criteria in a promising manner, financial success will likely follow. While some projects may dip below ICO price initially, I am more concerned with the long-term viability of each project. By using this heuristic and evaluating the likelihood of each of the above factors (using their inputs), I have found that most projects’ financial success beyond their initial hype aligns nicely. Of course, this is merely a heuristic: My attempt to define what a “cat” looks like, and how we can go about finding those cats.

A Data-Driven Look at ICO Success

My success heuristics have guided me relatively well. Despite this, there are other factors I may have overlooked, and there are other factors which I do not believe are important when they may, in fact, be crucial to determining an ICO’s success. To test these factors, it would be necessary to have historical data that contains this information. Surprisingly, despite the prolific data-analysis and technological community around ICOs and blockchain as an industry, no major reports or open datasets exist. While gathering exhaustive data about ICOs is a challenging undertaking, I do believe that it is an important task moving forward, and one that should be undertaken with diligence and care. This is a crucial step for being able to make more accurate and profitable investments, as without the data, we cannot even begin to think about replicating the success of AI with data when considering investments.

While more in-depth data does not exist, I did manage to gather some data from other research sources to attempt to take a more data-driven approach, but this data is lacking in many ways — its depth, process, and verifiability all leave some clarity to be desired. With these caveats out of the way…

In their article on ICO Developments and Trading, Sherwin Dowlat and the Satis Group define ICO success using three primary heuristics: Product/chain deployment, transparent roadmap, and Github code contributions in the past 3 months. Using these metrics, in addition to classifying ICOs as scams and failures, they develop a framework for examining the current ICO landscape. Unfortunately, the raw data and time-horizon of their analysis is not outlined, but their conclusions (assuming their analysis is/was valid) are quite interesting:

The first piece which should jump out is, of course, the astonishing number of scams. While they were unclear on their precise classification for a scam, if we include the hundreds of Bitcointalk ANN posts (as they say they have done, at least to an extent), this ratio seems accurate. Of the scams, we only hear about a few — but our availability bias does not mean they do not exist. Scams are rampant enough that the SEC even launched a fake ICO website, Howeycoins, aimed at better informing and protecting potential investors from the countless money-grabs popping up on a nearly daily basis. While most scams can be relatively easy to spot, some promising-looking projects have also ultimately turned out to be scams — an unfortunate problem which remains unresolved.

Let us leave aside the scams, however, as without knowing the precise data used, it is difficult to determine its precise accuracy. Instead, let’s focus on the non-scam cases: Given that an ICO was not a scam, about 42% went on to trade. Of this 42%, just under half met their criteria for success. If we assume that we can exclude scams, this success-rate is approximately on-par (or slightly above) with start-up industry standards of ~20% success rate. This validates diversification of investment, as well as the risk inherent in ICO investing.

In the case of these authors’ analysis, their definition success focuses on the broader market reaction to the ICO, and its continued development and improvement. This framework similarly focuses on long-term returns: Even if price performance is not stellar immediately, projects that are building out their products are more likely to succeed and grow over time. These measures, thus, do not directly address profits, rather, financial success is implicit in their criteria. This is particularly evident in their visualization of ICOs by current market-cap:

As we can see, “successful” ICOs are more likely to be valued higher than dwindling ones. This is not a coincidence: While financial performance in the short-term is driven by far more volatile and unpredictable factors (hype, supply-squeezes, Twitter influencers, pump and dump schemes etc), overall long-term performance is driven by fundamental factors such as team quality, product-market-fit, meeting established deadlines, and continual improvement — all factors in my initial input function. While both the short- and long-term frameworks are potentially profitable opportunities to take advantage of, focusing on the latter provides a more useful framework for strategic investing.

AI and ICO Winters: The Cyclic Nature of Technology

Everything works great until it doesn’t anymore. In AI, as researchers and investors both realized that the axiomatic cat-finding approach was an exercise in futility, they shifted their focus elsewhere and both money and talent dried up. It was not until the computational power, data-analysis tools, and machine-learning frameworks of the past decade were sufficiently developed that AI could return to prominence. AI’s resurgence has driven a new round of investment, excitement, and advancement, and we seem to have cycled back to a “Golden Age” for the technology — VCs, academics, and entrepreneurs alike are flocking back to the new AI. Similar processes have occurred with data-driven insights in numerous other industries as well.

In VC, winter is coming. As deals become commoditized, more expensive, and scarcer, VCs must find new high-yield outlets for their capital. As exits decline and IPOs become rarer, VCs must find new avenues for liquidity. Cryptocurrency and blockchain are potential outlets for these problems, but they are far from immune. In fact, we have already seen at least one “ICO winter,” and we may see another soon: As more money flows into ICOs, deals become similarly commoditized, and competitors vie for an all-too-slowly expanding pool of precious resources (community, developers, government support and more). Projects that fall out of the “hype cycle” struggle to stay afloat and keep their high valuations at ICO. We saw this process last summer, as Ethereum’s price skyrocketed to nearly $400 before crashing again, with numerous projects doing ICOs. We saw it again in February-April of this year, as the poor market conditions added a further challenge.

The good news and bad news are ultimately similar: Hype, markets, and even technologies have a cyclical nature. We can learn from past cycles and behaviors, and generalize to help inform future decisions. Increasing the availability of data-driven resources in the cryptocurrency space will be an important component of ensuring future cycles can be capitalized upon as much as possible.

Recap and Summary

Using AI as an analogy, I have walked through frameworks and challenges in ICO investment for both the private and institutional investor. We currently remain firmly in Plato’s cave in the cryptocurrency market, and lack the historical data to begin to move toward a more Nihilistic framework. As a consequence of the newness of ICOs, we could not even build a reliable dataset for making any long-term predictions today. It will be exciting to watch the space mature further in the upcoming months and years, and hopefully, with time, we will be able to more accurately identify the drivers of industry success.

To fuel this process, we must work toward building long-term robust data-sets of information about cryptocurrency markets and ICOs, and we are still in the early stages of doing so. Until then, our approaches to investment will continue to take unverified and potentially inaccurate axiomatic and heuristic approaches — while these may often be “good enough,” they are hardly satisfactory.

Thus:

… To be continued at a later date, with later data.