From Concept to Creation

December 14, 2018

Using AI powered mock-ups to drive conversation

One challenge working in agile web development is figuring out how to communicate quickly and effectively with clients. Typically, this is done using mock ups. A designer on staff, or someone else with the skillset, sits down and drafts up several images of websites to act as clarifying points for discussion with the client. This ensures that valuable developer time and effort isn’t wasted producing code and styling that ultimately doesn’t meet the client’s end vision. Using mock-ups allows the entire team to iterate until a mutual vision is discovered and agreed upon.

As an artificial intelligence (AI) practitioner, anytime someone is working with pictures, my AI senses tingle! Pictures are extremely dense with information as every pixel contains discrete red, green, and blue values – therefore pictures are ripe for exploitation by advanced AI techniques.

So, in this situation we are using specialized, third-party labor (designers, in this case) to develop specialized static, non-interactive products to communicate with our clients. This is where AI can step in to assist!

Through use of advanced AI techniques, machines can ingest screenshots, images or hand-drawn papers to create full-up static websites. In fact, there’s really no need to stop at websites. In one use case, the author was able to show an over 77% of accuracy for three different platforms; creating applications for iOS, Android and web-based technologies from a single image. The technology is so robust that developers at Airbnb are using hand sketched documents to create mock ups for use today. The team sketches blocks that represent either text, pictures or headers and the machine converts the pictures into applications automatically.

The only challenge is sitting down with your team and finding a visual representation and ingest mechanism that works best for your effort. Open source code is available today to help you leverage this technology for your mission needs. Happy hacking!


Airbnb design. Sketching Interface – Generating code from low fidelity wireframes.


GitHub (Open Source Code) – emilwallner:

GitHub (Open Source Code) – tonybeltramelli: Author

By Brad Mascho
Chief Artificial Intelligence Officer, NCI