Expert interview: home automation using augmented reality

Before we jump to your smart home and AR vision, tell us what your role is at Overly?

Like other backend developers, I do quite a lot of things on the server-side, which is often not directly visible. I would describe my role as linking Overly’s customers and external services. I ensure that Overly Creator works as smoothly as possible, from project creation to payment integration and processing. I am also connecting the dots to ensure that our users can retrieve Overly’s augmented reality projects on their mobiles. I develop new features alongside my colleagues, and the latest projects we launched include 3D editor integration and NFT visualization from OpenSea.

Alright, and how did you get into home automation?

My friend was building his family home and decided to integrate some smart home functionalities to measure temperature, automate lights, locks, security, etc. At the time, I was also planning to renovate my home, so I was keen to learn more about his ideas and visited to see what he was setting up.

I was surprised to learn that one smart home thermometer cost approximately $160, and as there were not many providers on the market, it was a price you’d have to pay. Leading providers also require home installations to be built with home automation in mind to ensure compatibility. 

If we think about all the rooms that would need a thermometer plus other household appliances and tasks that can be automated, the costs quickly add up. My house was already being renovated and the new electrical installation had been put in place without considering possible smart home elements. That’s when I started to look for workarounds.

And where did your research lead you?

After researching API requirements, I realized that I could try setting something up myself. In 2016 I wrote my first temperature script, and we built five thermometers for my friend that cost $20 in total. The same functionality would have cost $800 with the company he was initially using. By the way, these thermometers function to this day. 

After our initial success, I realized how easy it is to code home automation tools. I was also excited that I didn’t need to rehash the entire house’s electrical installation or buy the latest appliances to build a smart house.

What’s the difference between experienced brands and your approach that lets you cut costs so easily?

The market is not saturated, and  most companies leading the way require the entire electrical installation of a home to be compatible with their systems. They connect all devices to a single home gateway where various appliances communicate; signals are sent and received to perform specific actions. However, this means that most households cannot access major home automation systems unless they update all wiring and gadgets.

I use multiple WiFi relays and sensors for each action I want to automate at home. I combine them all in a free app called Home Assistant. All I need for my system work is a stable WiFi connection, and I have been able to automate most mundane tasks at home.

What does one need to be able to automate some tasks at home?

Of course, specific knowledge lets me write code. I also recently became a qualified electrician, but that is more to do with my interests.

Even people with little IT knowledge can automate gadgets available on a home internet network using relays and sensors. They may need some technical support to connect various older appliances to this network, but there is no need to invest in the latest technologies to do so.

Most popular things to automate include heating and cooling based on temperature changes. People also appreciate the ability to open/close windows remotely, switch off lights, open gates, etc. You also never have to worry about leaving on your iron because you can switch anything off remotely.

So what exactly have you automated at your home so far?

Most of my home is automated to some level. All lights and household appliances are on my smart home network. I also use the tools outside of the house and have set up greenhouse ventilation and garden watering based on the weather. If it hasn’t rained in a while and the weather is relatively warm, the water sprinkler turns on. The next step is adding a moisture sensor to the mix. Home heating is also completely automated.

What are the benefits of home automation?

The initial idea for me was to automate tasks to be able to focus on more important things and living life. However, I see that I have achieved a lot more by adding innovative elements to my home. 

Automation can help if you’re conscious of your expenses and your impact on the environment too. My household is on a dynamic electricity contract, and we pay each day for the consumption based on the stock market prices. However, home automation helps align the best time to run some appliances based on market costs in real-time. Once the prices drop, particular gadgets turn on and complete their tasks.

Smart home functions can also save a lot of headaches caused by dealing with the consequences of broken home installations or gadgets. The system does not have a mind of its own, but the other benefit is its ability to recognize any deviations from the norm. For example, it notified me of a hike in water usage. This led me to explore the issue and find that one of our water pipes had burst, causing significant leakage. 

I’ve got sensors that notify me if my sewerage system malfunctions and doesn’t turn on within a 24 hour period. At the same time, the flood sensors would shut off our water supply to prevent flooding. Security is a big one as your smart home devices include smoke detection and motion sensors. You can also remotely check if your doors and windows are closed.

Another benefit is the ability to skip mundane tasks. Previously I had to manually read the water meter and email the usage information to the local provider to get an accurate invoice. Then as usual I had to schedule a payment of the needed amount. I scrapped this process by creating a sensor which reads the water usage impulses and sends them to the home server. The system automatically informs the local council and manages the payment process.

How come you decided to combine your already successful smart home app with augmented reality?

I just wanted to try combining the two because I work with Overly. However, once I tried adding augmented reality to my home automation projects, I realized its potential.

Augmented reality can make the management of smart appliances and homes more efficient. Currently, my family uses the Home Assistant app, which features countless tasks that we can accomplish with it. However, to call a specific action, a person has to open the application, search for the function, and press a button to action something. 

However, if I want to turn on a robot vacuum cleaner, is it easier to walk to this vacuum or open an app where I have to look for the right switch and functionality? The automation doesn’t seem efficient in this example. 

It is where augmented reality is the perfect solution. You can take your phone and point at any building, room or appliance to call an action. For example, I have a sauna in my garden that has to get to a certain temperature level before people get in. AR enables me to skip multiple steps to get information. I no longer have to open the Home Assistant app and select the right building and the function I need. I can point my phone camera at the sauna and see if it is hot enough to enjoy.

We’re all anticipating smart glasses in the coming years. Home automation will go hand in hand with it. Wherever we point our camera now to get some interaction with augmented reality, we’ll soon just have to look at it, and AR will give us information automatically.

What have you been able to achieve with Overly’s marker-based AR app?

I have tried setting up simple processes using its trigger functionality. Although the Overly app is a marker-based application that prefers flat images instead of objects, computer vision can still recognize distinct settings if you approach them from the same side and don’t change the surroundings much.

Currently, I have tested the app for its ability to retrieve information and trigger functionality. For example, I can now scan any thermostat with Overly to read the room’s temperature, scan a lamp to switch on lights, scan my garden and start watering it. It works great, and I can see AR’s potential impact on the home automation sector.

What other augmented reality applications do you see for home automation?

There are two sides to the potential of AR in terms of smart homes. We can see how homeowners can benefit, but AR can also provide enormous benefits for those planning, building and renovating homes.

For example, I’m a trained electrician, and I think it would be great if I could go into my room, scan it with my phone and see electrical wiring visualization in 3D on the walls. This would be much more efficient than looking through paperwork to understand what I should be doing. The same applies to piping and construction.

As a homeowner, I believe the visualization opportunities that AR provides can help people live more comfortably. Augmented reality is alsu crucial in helping people maintain their homes safe as well as manage their households better. 

For example, if I am not at home and there are some issues with electricity, anyone from my family members could use their phone to scan the electricity transformer and switch off the power supply without physical contact. Similarly, if a water pipe burst and I wasn’t at home, others could use their phone to scan the bathroom and turn off the main water supply. They don’t need to go into an app and try to select the necessary functionality.

👉Leave a comment below if you have any questions for Artis, and we’ll get them answered. You can also engage directly and connect with Artis on LinkedIn.