Answer WhatsApp text, voice, image and video messages with OpenAI and Gemini

Description

Quick overview This workflow replies to incoming WhatsApp Business messages (text, voice notes, images, and videos) by transcribing or describing media with OpenAI and Google Gemini, then generating a conversational response with an OpenAI-powered agent that can reference Wikipedia and remembers each sender. How it works Triggers when a new WhatsApp message is received and splits the payload into individual message parts. Routes each message by type (text, audio, image, or video) to the appropriate processing path. For voice notes, retrieves the WhatsApp media URL, downloads the audio, and transcribes it to text with OpenAI. For images and videos, retrieves and downloads the media from WhatsApp, then analyzes images with OpenAI Vision and videos with Google Gemini to produce a description. Normalizes the message (text/transcription/description plus any caption) and sends it to an OpenAI agent with per-user memory and an optional Wikipedia lookup tool. Sends the agent’s reply back to the original sender via WhatsApp, trimming the text to the configured maximum length. Setup Connect WhatsApp Trigger OAuth and WhatsApp Business Cloud credentials, then configure the webhook in Meta so incoming messages reach this workflow. Add an OpenAI API credential for chat replies, audio transcription, and image analysis. Add a Google Gemini (PaLM) API credential for video analysis. In the WhatsApp send step, select your WhatsApp Business Phone Number ID and ensure replies target the sender’s phone number. Review the Configuration values (system prompt, model IDs, image/video prompts, memory window, and max reply length) to match your needs. Additional info Answer WhatsApp text, voice, image and video messages with GPT-5 and Gemini 📥 Open full documentation on Notion Need help customizing? Contact me for consulting and support : Linkedin MY NEW YOUTUBE CHANNEL 👉 Subscribe to my new YouTube channel. Here I'll share videos and Shorts with practical tutorials and FREE templates for n8n. An n8n automation workflow template by Dr. Firas.

Categories

Tags

Capabilities

• 8 nodes
• 2 views