DesktopBuilder Postmortem - Can Augmented Reality Help Computer Builders?

Our team put together a mobile app utilitizing Vufroria's Object Scanner functionality with the intent of helping people build their own computers. The core audience was people who haven't built a computer before, and might feel nervous about putting their own computer together without breaking any expensive computer parts.

What worked well

1. Ease of Vuforia plugin Integration enabled completion of requirements under tight timeframe
   1. We only had 3 weeeks to complete this project as a team of 5 people, so we needed a system that was easy to integrate with.
   2. The Vuforia system was easy to pick up and use, and we handled integrating the system into our code in around 3 days, which gave us around 2 weeks to handle making the UI and setting up the other systems of the application
2. Object scanning phone app easy to use
   1. As part of the scanning process, you need to download the application to your phone. It was easy to put the APK on my phone, store the files on the phone, and then bring the object files over to the Vuforia Developer Portal using a USB cord. The app is a TON of fun to play with, because it shows a green overlay where you have scanned, so it becomes sort of a game to scan the objects in.

What didn't work out as expected

1. Computer part objects failing to be recognized in scans
   1. We couldn't predictso we scanned objects multiple times under different lighting conditions to try to get the object recognition to  work under different lighting conditions
   2. Computer parts have some reflective surfaces, which make scanning difficult
   3. Many computer parts are flat, boxy, and have the same type of shape. This means that it was hard to generate unique scans for the object
   4. Many computer parts are small, and small objects make for poor scanning targets with Vuforia. We would not recommend trying to scan objects that take up less than half the space on the printed Vuforia image target
2. Computer part object scanning false positives
   
   1. We had scans for each of the different steps of assembly for the workflow, as well as each of the different parts. However, because the different steps largely resembled each other, sometimes the assembly step scans would be triggered when the user was still scanning in the parts
   2. In order to get around this, we made sure that the workflow functionality wouldn't advance on a false positive, and only advance at the appropriate time to do so. But this also meant that the app wasn't going to give accurate information back to the users about what they just scanned, which makes the application far less useful.
3. Testing reacted differently on the computer and phone cameras
   1. In some cases, the phone camera would scan objects better than the developers' laptop computer cameras, and in some cases the phone camera would scan better. We weren't able to determine why these differences existed, but one explanation would be that the camera had less processing power, but allowed for different lighting angles. Depending on whether processing power or lighting conditions were the issue, the computer and phone cameras reacted differently to the objects.
4. Object database size caused Git crash
   1. Because we scanned objects multiple times, eventually we hit over 100 MB on the in-app object database with around 35 object scans. Since we were using GitHub for version control, and GitHub has a file size upload limit of 100MB (at least for our account level), we had to erase some commits to get the files into the online repository
   2. In Vuforria, you can set up multiple different scan databases and put them inside of your app. If you have a lot of object scans, make sure to set up different object databases to prevent this issue and keep your database sizes on the smaller side.

Future Opportunities and Recommendations

1. For Augmented Reality, focus on instruction instead of verification
   1. Our app was focused on verifying that users had all the parts they needed, and that after they performed the steps they checked to see if they did it right. 
   2. The app might have been more effective if instead we created a placeable image target that could be put on a surface, then had 3d models showing how the user should be completing the computer building steps in Augmented Reality. That way, even if they didn't have the exact computer parts we had scanned, they could still roughly follow along with instructions.
   3. In this way, when the user is ready to move on to the next step, they can do so at their own pace, rather than deal with the gatekeeping of the object verification as part of the user experience.
2. Avoid small, technically detailed objects and images in augmented reality
   1. We would love augmented reality identification to be on pace with humans' ability to recognize objects and images, but the technology just isn't there yet. For augmented reality, try to focus on big objects (but not too big, of course, think in terms of the size of a piece of letter paper) and dynamic colors in what you want to recognize.