Rayban Meta Smart Glasses: Review and Integration with Home Assistant

The new Ray-Ban Meta Smart Glasses are not only stylish but also have the potential to be a real asset. However, in Germany, where Meta AI is not yet available, their functionality is significantly limited. In my latest DIY project, I developed a solution: I control my Home Assistant smart home through the glasses—indirectly via WhatsApp messages! And that’s not all—you can even connect your own AI to the glasses through this method.

So, if you’re looking for a creative way to make your smart home even smarter, you’re in the right place!

Check out the following YouTube video, which includes a review and a tutorial on the WhatsApp Bot hack:

Review of the Ray-Ban Meta Smart Glasses

The Ray-Ban Meta Smart Glasses are, at first glance, a really cool gadget. The design is classic and stylish, and only upon closer inspection does it become clear that these are no ordinary glasses. The integration of smart features makes them an intriguing companion for everyday life. But how do they perform in practice?

The photo and video quality, while decent, is not breathtaking—don’t expect results comparable to an iPhone, Google Pixel, or DSLR. In low-light conditions, the image quality is disastrous, which is understandable given the limitations of such a small sensor. However, in bright daylight, the glasses shine by enabling you to capture spontaneous moments without the hassle of pulling out your smartphone. This is particularly practical on vacation when you want to quickly capture a cool scene.

Another highlight is the hands-free functionality. Thanks to the built-in speakers and microphones, you can listen to music, watch videos, or make calls without needing additional devices like AirPods. This hands-free convenience is especially enjoyable during walks, travels, or everyday tasks.

However, the glasses have a major drawback in Germany: without the Meta AI feature, which is not available here, the product feels significantly limited in functionality. Meta AI could be a real game-changer to fully unlock the glasses’ potential as a smart everyday assistant.

Overall, the package is still appealing, especially for tech enthusiasts who appreciate smart gadgets. Yet without Meta AI, the glasses’ full potential remains untapped.

Integration with Home Assistant and Custom AI

Currently, the glasses are heavily locked into Meta’s ecosystem, with almost no third-party integrations (except for Spotify, Calm, Amazon Music and Apple Music). This means it is impossible to directly connect the glasses to a smart home system or use any AI systems other than Meta AI.

I explored ways to bypass this limitation and came across a post by Ryan Steckler, who suggested indirectly connecting the glasses to other systems via WhatsApp messages (a feature natively supported by the glasses).

Here’s how I implemented this solution, including detailed steps to recreate it:

You’ll need a second WhatsApp number to serve as the bot account. Once you have the number, proceed to your Home Assistant setup and create a new automation.

In Home Assistant, create a new automation with a Webhook trigger. Copy the generated webhook URL and save the automation with a clear name.

Install the following Docker image on a local server in your home network. This server should ideally run 24/7.

Clone the repository and open the docker-compose.yml file.

Replace the BASE_WEBHOOK_URL environment variable with the webhook URL copied earlier.

Follow the GitHub instructions to start the container and initiate a session.

When the session starts, scan the QR code with the WhatsApp app linked to your second number.k

Open your automation and add a condition using the following template code:

{{ trigger.json.dataType == "message_create" and trigger.json.data.message.id.remote == "MYCHATID" and trigger.json.data.message.id.fromMe == false }}

It is important to replace MYCHATID with the specific chat ID of your WhatsApp conversation with the bot. This can also be a group chat. That’s how I set it up: create a group chat (e.g., named “My Home”) that includes you (with the WhatsApp account linked to your Ray-Ban Meta Glasses) and the bot account as participants. Then send a test message to the group. You should see that your automation is triggered—this is the first sign of success!

Next, you can use the “Traces” feature to check which specific variables are set during the webhook processing. In the Traces section, click on the first step of the automation (the trigger) and then switch to the sub-tab “Changed Variables”. Here, you should see a long JSON object containing several variables under “trigger.json”. Look for trigger.json.data.message.id.remote and copy the ID into your template where MYCHATID is specified.

Interim conclusion: We have now ensured that the automation is triggered by every event generated by the WhatsApp API. Additionally, we have added conditions to ensure that it only listens to new messages sent to the group you created (“My Home,” or whatever you named it). Most importantly, it excludes messages sent by the bot itself. This prevents the automation from running endlessly when the bot responds to a command, as it would otherwise be triggered by every reply from the bot.

Now we can respond to the messages sent to the WhatsApp group. To do this, we create a new action. We execute an action and switch to YAML mode during the action creation. Then, we copy the following YAML code into it:

action: conversation.process

data:
  
  agent_id: conversation.openai_conversation
  text: |-
 
  {{ trigger.json.data.message.body
}}. Please keep the answer as short as possible.

response_variable: response_gpt

This requires that you have already activated the OpenAI Conversation integration in your Home Assistant installation. You will need an API key from OpenAI for this. Additionally, you should configure the OpenAI integration so that Home Assistant Assist is used for controlling devices in your smart home.

Important note: Of course, you can also use any other LLM, including a locally hosted one! In that case, the call above will need to be adjusted accordingly.

With the action above, we send the text contained in the WhatsApp messages to the MyHome group to OpenAI for processing and ask ChatGPT to keep the responses as brief as possible. This is because the Ray-Ban Meta Smart Glasses can only read short WhatsApp messages aloud directly.

The response from OpenAI is stored in the variable “response_gpt”. The final step is to send this response back to the MyHome group via WhatsApp.

To do this, we need to save our automation again. Then, we switch to the File Editor or Studio Code Server (depending on which add-on you have installed) to edit the configuration file of Home Assistant, configuration.yaml.

At the end of the file, we add the following code:

rest_command:
  send_message_via_whatsapp:

    url: "http://MYIP:3000/client/sendMessage/{{ sessionId}}"

    method: post

    content_type: "application/json"

    payload: '{ "chatId": "{{ chatId }}", "contentType": "string“, "content“: "{{ content }}" }'

This creates a new action that sends a POST request to the WhatsApp API. Please replace MYIP with the IP address of your local server where the WhatsApp API is running. The variables sessionId, chatId, and content will then be populated directly from the automation.

Please save the configuration.yaml file and restart Home Assistant. Afterward, the new action should be available.

We can now reopen our automation and add another action after the OpenAI action. Please create another template action with the following content:

action: rest_command.send_message_via_whatsapp

data:

  content: "{{ response_gpt.response.speech.plain.speech }}"

  sessionId: MYSESSIONID

  chatId: MYCHATID

enabled: true

We populate the content variable with the response from ChatGPT. The sessionId variable should be replaced with the WhatsApp session ID you assigned, and the chatId variable should be set to the ID of your WhatsApp group (as mentioned above).

That’s it – so sollte es nun möglich sein, WhatsApp Nachrichten an die Gruppe MyHome zu senden (auch von der Brille aus per Sprachbefehl) und die Antwort als WhatsApp Nachricht zurück zu erhalten (auch via Sprache über die Brille).

Please let me know how it worked out for you and what you built. Please also don’t hesitate to comment or contact me if you have any problems.