Most people are probably unaware that applications relating to mixed reality (virtual reality, augmented reality, etc.) have been around since the beginning of the 20th century.1,2 But the technology at the time was pretty much still at concept or proof-of-concept stage than anything else. Nothing of note came out of it until the last few years.
It took Niantic’s collaboration with Nintendo to raise augmented reality’s global public awareness via Pokemon Go. The augmented reality game has since been downloaded more than 500 million times and has generated $600 million mobile AR revenues in its first 3 months. As of 2016, VR has generated $2.7 billion and AR $1.2 billion in revenues for a total $3.9 billion MR market.3A market research study estimated that the entire VR device market will reach $11 billion in about four years.
Mixed Reality Patent and Patent Application Categories
To identify the various categories of utility patents and patent applications (P/PA) directed to virtual reality, augmented reality, etc. (collectively referred to here as “mixed reality” or “MR”), we analyzed more than 4,000 mixed reality-related P/PA we obtained from a preliminary patent search. Fig. 1 shows the different categories ranked according to the number of P/PA assigned to each of them.
Fig. 1. Ranking of the various MR-related categories in terms of the number of P/PA assigned to each of them having filing dates covering 2002-2016.
The following five categories have the lowest P/PA over the period 2002-2016:
These least active categories would be logical areas for possible development for those seeking to explore uncrowded investment sectors (at least basing only on the number of P/PA). But this may be misleading because MR applications involving search systems, machine intelligence, virtual assistant systems, robotics, and automation are expected to assume greater importance as AI becomes more deeply embedded in already multiplying market segments and application areas.
Top Market Segment Categories
Among P/PA with claims that either expressly recited or clearly suggested the market segment they are directed to, Gaming garnered the highest number of P/PA. The top three categories Gaming, Healthcare, and Commerce have significantly more P/PA than other categories. Some of these categories are expected to overlap in terms of applications, e.g., Gaming and Collaborative Systems. Somewhat unexpected is Design and Transportation’s being ranked 4th and 5th, respectively, ahead of market segments in which MR is expected to be widely-used, e.g., Entertainment, Social Network, and Industrial Applications.
Table 1. Number of P/PA for the top 10 market segment categories (2002-2016).
| Patent Number | Assignee | Number of Forward Citation | Cited By | Effective Filing Date | Number of Claims |
|---|---|---|---|---|---|
| 1. US8547401 (“Portable augmented reality device and method”) | Sony | 320 | Xerox, Nintendo, Ford, Dassault, Intel, AT&T, Microsoft, Sharp, Nokia, Lockheed Martin, Ebay, IBM, Disney, Google, Qualcomm, Samsung, Magic Leap, Activision, etc. | 8/19/04 | 13 claims (3 independent) |
| 2. US9384594 (“Anchoring virtual images to real world surfaces in augmented reality systems”) | Qualcomm | 260 | Samsung, Apple, Intel, AT&T, Microsoft, LG, Google, Sony, IBM, Xerox, etc. | 3/29/11 | 108 claims (4 independent) |
| 3. US8576276 (“Head-mounted display device which provides surround video”) | Microsoft | 187 | Google, Toshiba, Osterhout, Dell, Siemens, Magic Leap, Apple, Qualcomm, IBM, Nokia, Samsung, etc. | 11/18/10 | 20 claims (3 independent) |
| 4. US8012023 (“Virtual entertainment”) | Microsoft | 173 | Disney, Google, Nokia, Magic Leap, Sun Microsystems, Matsushita, Compaq, Fujitsu, Nintendo, Nvidia, Lucent, IBM, Sony, NEC, etc. | 9/28/06 | 20 claims (3 independent) |
| 5. US8814691 (“System and method for social networking gaming with an augmented reality”) | Microsoft | 141 | Philips, Sony, Apple, Nintendo, Lockheed Martin, Google, Magic Leap, Nokia, Amazon, Samsung, Motorola, Intel, Qualcomm, Nvidia, LG, Johnson & Johnson, Philips, Toyota, Toshiba, etc. | 2/28/10 | 32 claims (3 independent) |
| 6. US7991220 (“Augmented reality game system using identification information to display a virtual object in association with a position of a real object”) | Sony | 139 | Samsung, Canon, Amazon, Blackberry, Disney, Microsoft, Nintendo, etc. | 9/1/04 | 26 claims (4 independent) |
| 7. US8644842 (“Personal augmented reality advertising”) | Nokia | 130 | Sony, LG, Samsung, Microsoft, Qualcomm, HP, etc. | 9/4/07 | 20 claims (4 independent) |
| 8. US9292973 (“Automatic variable virtual focus for augmented reality displays”) | Microsoft | 126 | Sony, Magic Leap, Qualcomm, Panasonic, Osterhout, Motorola, Google, Intel, etc. | 11/8/10 | 17 claims (3 independent) |
| 9. US8094928 (“Stereo video for gaming”) | Microsoft | 125 | LG, Microsoft, Intel, Nokia, Sony, Apple, Amazon, HTC, etc. | 11/14/05 | 27 claims (3 independent) |
| 10. US9342610 (“Portals: registered objects as virtualized, personalized displays”) | Microsoft | 122 | Sony, Qualcomm, Google, Nokia, Honda, Disney, Fujitsu, Panasonic, Audi, AT&T, Universal City Studios, Oculus VR, etc. | 8/25/11 | 20 claims (3 independent) |
| 11. US8998414 (“Integrated eye tracking and display system”) | Microsoft | 121 | Sony, Oculus VR, Google, Osterhout, IBM, etc. | 9/26/11 | 12 claims (2 independent) |
| 12. US8933889 (“Method and device for augmented reality message hiding and revealing”) | Nokia | 117 | Fujitsu, AT&T, Apple, Motorola, Microsoft, Sony, Samsung, Qualcomm, IBM, Google, etc. | 7/29/05 | 17 claims (3 independent) |
| 13. US8611015 (“User Interface”) | 115 | Osterhout, Microsoft, Oculus VR, Magic Leap, Sony, LG, Samsung, etc. | 11/22/11 | 25 claims (4 independent) |
|
| 14. US9348141 (“Low-latency fusing of virtual and real content”) | Microsoft | 111 | Siemens, Qualcomm, Amazon, Google, Boeing, Motorola, Magic Leap, Nokia, Lockheed Martin, Audi, Sony, Lucasfilm, HTC, Samsung, etc. | 10/27/10 | 13 claims (2 independent) |
| 15. US8847988 (“Exercising applications for personal audio/visual system”) | Microsoft | 107 | Cisco, Magic Leap, Sony, Google, Osterhout, etc. | 9/30/11 | 20 claims (3 independent) |
| 16. US8427508 (“Method and apparatus for an augmented reality user interface”) | Nokia | 105 | Qualcomm, Google, Microsoft, Sony, Magic Leap, AT&T, IBM, etc. | 6/25/09 | 20 claims (3 independent) |
| 17. US8264505 (“Augmented reality and filtering”) | Microsoft | 104 | Sony, LG, Qualcomm, Honda, IBM, Cisco, Disney, Samsung, Nokia, Snapchat, Facebook, Warner Bros., etc. | 12/28/07 | 17 claims (3 independent) |
| 18. US8767014 (“Automatic text scrolling on a display device”) | Microsoft | 104 | Fujitsu, Amazon, Panasonic, Nokia, Sony, Google, Qualcomm, HP, Siemens, Oculus VR, Magic Leap, etc. | 7/22/11 | 17 claims (3 independent) |
| 19. US9213405 (“Comprehension and intent-based content for augmented reality displays”) | Microsoft | 103 | LG, Sony, Magic Leap, Panasonic, Google, Qualcomm, HP, etc. | 12/16/10 | 20 claims (3 independent) |
| 20. US8487838 (“Gaze detection in a see-through, near-eye, mixed reality display”) | Microsoft | 98 | Google, Sony, Qualcomm, Intel, Nokia, Motorola, Oculus VR, Audi, Osterhout, Volkswagen, Lockheed Martin, etc. | 8/29/11 | 19 claims (3 independent) |
Another thing to note is the average percentage of patents relative to the total number of combined patents and patent applications was 31% over 2002-2016, if we exclude Healthcare (20%) and Industrial Applications (22%). This suggests that about 1/3 of the mixed reality-related utility patent applications were granted during the 2002-2016 period. This percentage is significantly lower than the 43% average over 2002-2015 for all US utility patent applications from US filers that eventually issued into patents.4
Disclaimer: Data used for the preparation of this article was based on a preliminary search only. Parola Analytics, Inc. does not make any representation regarding the accuracy or validity of the data, analyses, and conclusions presented in this article.
References
1 Frontiers in Human Neuroscience, vol. 8, pp. 1-15, Mar. 2014.
2 “A New IEEE Initiative Focuses on Augmented Reality, Virtual Reality, and Human Augmentation Technologies,” Dec. 2016, https://theinstitute.ieee.org/technology-topics/consumer-electronics/a-new-ieee-initiative-focuses-on-augmented-reality-virtual-reality-and-human-augmentation-technologies.
3 “After mixed year, mobile AR to drive $108 billion VR/AR market by 2021,” Jan. 2017, https://www.digi-capital.com/news/2017/01/after-mixed-year-mobile-ar-to-drive-108-billion-vrar-market-by-2021/#.WfMOAteWYoMa.
4 U.S. Patent Statistics Chart Calendar Years 1963 – 2015, USPTO, https://www.uspto.gov/web/offices/ac/ido/oeip/taf/us_stat.htm
