Historically, space has been an industry only governments and heavyweight airspace corporations (such as Boeing) could handle. Uses of AI in space technology might be expected to be even more expensive, but the last decade of innovation has made space more accessible – and thanks to AI – data from space is becoming much more useful for businesses and governments.
At the time of this writing it costs $200k to build and launch a satellite into orbit, a reduction of cost from $200M 10 years ago (given pretty same capabilities) These technological innovations were summed up well by Steve Jurvetson, managing partner at DFJ:
“Compared to other industries, I have never seen such an enormous margin for improvement. There’s this canonical thing about a startup needing to pitch a 10X improvement to be a worthwhile investment. You rarely see an entrepreneur pitch a 100X improvement. But in space we’ve seen 1,000X, and really we’ve seen 10,000X.”
Today’s $200k satellites are capable of sending terabytes of images of our planet on the daily basis, enabling myriads of AI-driven applications that provide previously inaccessible insights on global-scale economic, social and industrial processes.
This article explores some of the most promising applications of AI in space technology (with a focus on visual satellite data), and many of the trends and startups that are driving these applications forward. This article will be broken down into the following sections:
- Overview of investments and investment trends in space tech
- Acquisitions in satellite imagery machine learning companies
- Companies and use cases
- Applications of AI an satellite data
- “One step” satellite data applications
- “Multi-step” satellite data applications
- Applications of AI an satellite data
- Market potential for AI and space tech, and three trends to watch
- Concluding thoughts and take-aways
We’ll begin by looking at investments:
Investments in Space Tech
Disruption and plummeting costs in space tech has been driven by two factors: Technology and venture capital.
An important recent development has been the utilization of commercial-off-the-shelf (COTS) electronics in satellites. Many of the electronic parts used in a modern refrigerator are perfectly viable for use in space, and this access and availability has helped to drive prices down significantly.
Commercial launch services have also disrupted the space tech industry. Companies like SpaceX and Blue Origin are granting easier access to space and pushing costs down by an order of magnitude by bringing a Silicon Valley mindset (and healthy commercial competition) and culture to previously very conservative, state-run environment.
Not surprisingly, such cost reductions caught attention of VCs. In 2015 venture capitalists nearly doubled the amount of venture cash invested in the industry in all of the previous 15 years combined, driven by $1B SpaceX and $500M OneWeb mega-rounds. In 2016, $1.415B have been invested, influenced by OneWeb’s $1,2 bn. round led by the SoftBank Group.
And these numbers are comparable with VC investments in other emerging hardware-enabled technology sectors – $3.6B in Internet of Things and $1.8B in AR/VR in 2016 (data from CB Insights) – it’s clear investors are aware of space-startups and not afraid to put their money into orbit.
Acquisitions in Satellite Imagery Machine Learning Companies
It should be noted that there are likely to be plenty of important space tech or satellite-related startups who don’t use artificial intelligence at all. As Canvas Ventures VC Ben Narasin told us in his “AI in Industry” podcast interview, AI is secondary to the business model and goals of the company. There may be satellite companies who can achieve their aims more effectively with simple software, or hardware with no AI at all. Given that this article covers image recognition and analysis from satellite data, AI and machine learning are essential to the task at hand, but that isn’t always the case.
For the sake of this article we’ll focus on computer vision techniques applied to satellite imagery – which is one of the most promising applications of AI in space technology today. Certainly AI and machine learning can be applied in the wide spectrum of space-related problems, such as development of robotic moon landers or rockets failures causes analysis, and those applications likely deserve their own articles in the future.
The earth observation (EO) industry is a good example of the current promise of AI’s satellite image applications. DigitalGlobe, one of the leading companies in EO business, has acquired 2 companies, working on analyzing satellite imagery with ML algorithms. Radiant Group (an established EO business working with US intelligence customers) was bought by the firm for $140M. Timbr.io – data science startup made some interesting work with satellite imagery – was also bought up by DigitalGlobe for an undisclosed amount.
BlackSky Global is a Digital Globe competitor who raised around $50M in VC funding has acquired OpenWhere — a software company developing web-based platform for satellite imagery analysis – in 2016.
In addition, National Geospatial-Intelligence Agency (NGA) – a US intelligence agency and the largest buyer of satellite imagery on the market – has clearly stated shift in its strategy from plain data towards data analytics. Citing Robert Cardillo, Director at NGA: “I envision the future where we will move from analyzing Big Data towards realizing a potential of Fast Data. We’ll buy basic imagery analysis as a commodity – much like we buy foundation data today”.
AI-driven satellite data applications are in demand for a number of important reasons, including:
- Major leaps and bounds in machine vision in the last 5-10 years have made challenging tasks (such as identifying cars, buildings, or changes in landscape over time) do-able by machines
- The exponentially increasing amount of data being produced from satellites (thanks in part to the proliferation of satellites, improvements in camera technology, and improvements in data storage and transfer capabilities)
- The heavy expense of imagery analytics and interpretation – which is still often performed by performed by human imagery analysts
Thus, while 163 remote sensing satellites have been launched in 2006-2015, next decade this number is expxected to increase in by a factor of 2.5 – to 419 satellites. Value-added services (data analytics, for example) for satellite and visual data is predicted to grow quickly in the coming years as well. Nearly a dozen startups (including Planet) are planning to launch large remote sensing satellite constellations.
VC’s seem to share in some of the enthusiasm about AI’s applications for earth observation. Orbital Insight recently raised a massive $50M Series C, led by Sequoia with the participation of GV, Lux Capital, CME Ventures and $20M Series B in June last year, led by GV and supported by Lux Capital, Bloomberg Beta and CME Ventures. Also, in November 2016 startup called Cape Analytics raised $14M Series B led by Formation 8, supported by Lux Capital, Data Collective, Khosla Ventures and Promus Ventures.
Generally, the dynamic of VC investments in “EO + ML” startups is shown below — it’s definitely on the rise and I expect we’d see a couple of deals until the end of the year. Below is a graph representing the data I was able to collect on the last four years of activity:
Startups with AI Applications for Satellite Imagery
Detailing “EO + ML” startup landscape, there are nine companies on my radar (see graphic below) – most of which are US-based and have been founded in the last two or three years, more information about the funding is presented above.
Of the companies listed above, it’s important to note that some (e.g. Orbital Insight, Spaceknow and Rezatec) are delivering wide-range of AI-driven solutions on top of the satellite imagery, while others are focusing on a particular vertical (Cape Analytics, Descartes Labs and Tellus Labs).
Applications of ML and Satellite Data
In this segment of the article we’ll look at the particular use-cases of artificial intelligence, which we’ll further segment into two distinct groups:
- “One-level” applications in which machine learning techniques are applied straight to satellite imagery
- “Multi-level” applications – applications built on complex pipeline – when information, extracted from satellite imagery is just a set of feature in more sophisticated models, also consuming data from “non-satellite” sources
To clarify “one-level” applications, it’s worth to mention that from technical perspective, these applications require complex machine learning pipeline (details ahead).
“One-step” Satellite Data Applications
We’ll dive first into object detection. Actual information on buildings, road segments, and urban area boundaries is important for municipalities, government agencies, rescue teams, military, and other civil agencies. However, while object detection task has been enabled by a number of integrated ready-to-use pipelines using convolutional neural nets (FasterRCNN, YOLO), there are some issues with satellite imagery which make this task more challenging.
First, objects in satellite imagery are often very small (~20 pixels in size), while input images are enormous (often hundreds of megapixels) and also there’s a relative scarcity of training data. Thus, the methods mentioned above are not optimized to detect small objects in large images, and often perform poorly when applied to Earth observation data. Adapting these methods to the different scales and objects of interest in satellite imagery shows great promise, but is a research area still in relative infancy (cited from Adam Van Etten’s Medium post on satellite imagery).
Below is an example of object detection in action:
Second example is change detection — use case, which has numerous applications, such as crop, land use, urban infrastructure (e.g. road segments) environmental (deforestation, water reserves) and humanitarian crisis monitoring.
Technically, the goal of a change detection algorithm is to create a map, in which changed areas are separated from unchanged ones. Two types of change detection can be further delineated: A binary change detection (when each pixel is associated with one of two possible classes; changed or unchanged) and multi-class change detection, when each transition type can be explicitly identified.
In a case of multi-class classification only supervised techniques can be utilized as ground truth data. This is required to provide explicit labels on detected transitions from one class to another (say, building to water in crisis response situation). Ground truth data collection is a complex and time-consuming task (often overseen by teams of humans), thereby building multi-class classification solution on a global scale is tremendously complex. Normalizing satellite images is another ongoing challenge related to satellite imagery. Weather, lenses, and natural changes in lighting throughout the day (or the year) make this especially challenging.
Another complexity is that all images in the dataset should be normalized to be ready for input into change detection process – this is also a difficult task, taking into account many factors, related to satellite imagery. Building change detection solutions requires competence in machine learning and strong experience with remote sensing data.
Below is an application of change detection in action:
“Multi-level” Satellite Data Applications
As it was mentioned above, “multi-level” applications are ones in which information, extracted from satellite imagery (using “one-level” methods described above) is only a line of features in more complex ML system. As a lot of thing are going on in this field I’d provide short description of top-5 most interesting use cases and potential verticals interested in such products
First one is prediction of retailers’ profits by counting cars on the parking lots – which is only one of many AI applications in the retail sector. As the number of cars on the parking lot is a some kind of proxy for retailers’ revenue (more cars – more customers – more revenue), if there is an automated algorithm to accurately detect and count cars on satellite imagery at parking lots across the globe and augment this data using other sources (e.g. socioeconomic ones) it’s possible to predict retailers’ profits before annual/quarter report is published. And this is obviously interesting for investors as it’s some kind of inside information provides a chance to beat the market. Another customer segment is operating companies, interested in monitoring their competitors from space.
Second example is crops’ yields and prices prediction. Historically, vegetation indices, such as NDVI, build on remote sensing data (see definition) have proven a very successful tool for estimation yields. From the other side, it’s obvious that yield-related task cannot be solved only by vegetation indices, thus more features should be included, such as weather and soil-related data. And if we augment these features with prices and sentiment analysis, it’s possible to create a model beating official forecasts in terms of timing and accuracy.
Interestingly, NDVI indices that are critical for agriculture-related models can be obtained using open source government-backed satellite imagery sources, such as Landsat and Sentinel, which significantly reduces barriers to entry. Speaking of potential customers, everyone who needs ag-related intelligence can be one – commodity traders, insurers, market research firms, etc. and don’t forget farmers themselves.
In our interview with Saman Farid of Comet Labs (a San Francisco-based VC firm), he mentioned that high-value crops – such as apples or berries – may justify a higher technology investment from AI-related products in the near future. We can imagine that satellite image applications will likely be used first in the places where farming companies believe will have the highest yield.
Third example is estimation of world’s oil inventory by detecting shadows on them. The idea is pretty straightforward: as most of the fuel tanks have floating top, it’s possible to calculate it’s volume and the amount of in it, utilizing very simple trigonometry – picture below helps a lot in understanding. Also, if we know locations of all such oil tanks (extract them from satellite imagery or gathered from some database) we can calculate global oil inventories. Potential customers in this case are investors (yes, again), O&G companies (to monitor competitors) and probably government agencies, interested in timely information on other countries’ energy resources.
Another application (4th in our list), combining (in some sense) all use cases above is monitoring economic activity of some countries across the globe, especially – hard-to-reach ones, such as China. Really, if you can give an independent satellite-powered measure of cities development (car density, construction rates, electricity consumption through nighttime illumination) and import and export operations (ship count and sizes, aircraft count in airports), combine it with traditional surveys, statistics and media data and, finally, pack it in some index – it can be valuable for investors and government agencies.
And last but not the least example, correlating with the previous one and having humanitarian flavour is prediction of poverty through satellite imagery. While accurate measurements of economic characteristics significantly influence governments’ policies and resources allocation, there is still lack of data on key measures of economic development in development world (as well as in rural areas in developed world) because data is gathering only through surveys, which are very time-consuming. Given the obvious difficulties of scaling up survey-based methods, another way to measure poverty has been developed – from orbit. In this case different objects (urban areas, water, roads, etc.) and nighttime luminosity are using as features on which it’s possible to estimate economic activity.
To finalize “pipeline-based” applications description – here is the table summarizing them by key parameters: (i) required imagery; (ii) features, extracted from imagery; (iii) data, augmenting imagery; (iv) customers and (v) companies, delivering one or another applications.
And this table is a good segue from technical side of things to a next question I’d like to outline in this write-up: what is the market environment for “EO+ML” startups and how this environment would evolve in future.
Market Prospects
Despite that there are some positive signals on the market (e.g. a number of M&A’s last year), it’s difficult to estimate how much “EO+ML” startups are successful in terms of revenue – the only measure right now is the amount of raised funding; and speaking of M&A’s it’s very likely an acqui-hire type of situation than an acquisition of a revenue-generating business.
Here are a few observations on the market environment as well as some guesses on how “EO+ML” ecosystem would evolve in coming years.
Trend 1: More data, more startups
As it can be seen from the table above, very-high-resolution imagery (1m resolution) is required for most of the commercially viable applications and right now it is one of the barriers binding the market as there are only 2 providers of such imagery out there: Digital Globe and Airbus (i.e. prices are high and the amount of data is limited). However, there are quite a few startups that plan to launch very-high-resolution imaging satellites (Black Sky Global, AxelSpace and Hera Systems to name a few) and if they’d deploy their constellations successfully, it’d drive prices down (say, significantly increase availability of satellite imagery) and foster EO+ML ecosystem accordingly. Thus, we’d see a new breed of startups on the intersection of satellite imagery and machine learning.
Trend 2: Satellite imagery analysis would be integrated with other data sources
Most of the use cases mentioned above are “vertical-agnostic” thus it’s difficult to outline real pain points they’re solving. And speaking from customers’ perspective, it’s also difficult to point out any application in which satellite imagery is critical to make a decision (apart from Military/Intelligence): let’s just look on the finance-related applications – there are billions of ways beyond satellites to predict what’s going to be next on the market. Thereby, remote sensing data analysis would be merged with other customers’ data sources and integrated with existing workflows to provide answers instead of pixels or number of cars. Accordingly, existing startups would double efforts in this direction
Trend 3: Vertical-tailored solutions would be on the rise
As a some kind of contrary with previous point, there are some niches in which satellite imagery has unique advantages compared to other solutions and most notable – it is infrastructure monitoring. Global scale, automated and cheap infrastructure monitoring brings significant value, thereby I see an opportunity for vertical-tailored solutions utilizing satellite imagery analysis. Cape Analytics is a perfect example of a startup delivering product, focused on one particular industry – Insurance. Speaking of next industries that would leverage analysis of remote sensing data I see retail, security, real estate and so-called “smart cities.”
Concluding Thoughts on AI in Space Tech
Summing up, market prospects for startups, applying machine learning to satellite imagery is uncertain, but rather favorable, than not as:
- Significant increase in available data is going to happen quite soon, in 2-3 years, which would drive process down and reduce barrier to entry for developers
- Fusion with other data sources and integration into customers’ workflows is get going. And as satellite imagery and analytical products would be commoditized, we’d see adoption in other verticals.
As for the technical side of things, developing AI that can analyze remote sensing data is a valuable line of work as:
- Current open-source AI tools are not optimized for satellite imagery
- The amount of data, generated from satellites is already far more than people can effectively analyze and it’s growing exponentially.
Image credit: Skyline
Disclaimer: The data behind the graphs in this article was compiled by this article’s author, Valery Komissarov of the Skolkovo Foundation (a Russian centre for the development and commercialization of advanced technologies based near Moscow). Emerj was not collect or compile the data behind these representations.
