Last week's challenge was a little more technical. (If you missed it, you can check it out here.) So this week, I thought we could do something a little more fun and experiment with the image recognition and interpretation abilities of modern AI models.
Over the last year, AI tools have gained the ability to interpret the visual world around you. Photos, screenshots, handwritten notes, dashboards, receipts, whiteboards, diagrams, messy fridges, cluttered desktops. The purpose of all of this is to make it easier to share context about the world around you with AI systems.
This week’s challenge is experimenting with image-based context sharing.
Before we start: if you are getting this email forwarded to you and are not yet part of the lab - you can join the tiny AI lab here.
The Challenge:
This week, you're going to use AI vision tools to analyze something from your home or working area. This challenge was actually inspired by Allie K. Miller, who used AI to analyze her desk setup and get recommendations on how to improve it. You can check out the prompt she used in her post.
The goal for this challenge is to use images, screenshots, or photos as input and see what AI can help you understand, organize, improve, or decide. Try a couple of different images to see how well it does at interpreting and analyzing different types of context.
The prompt to get you started
For this week, the prompts are simple, so I’m going to give you a few different ideas and then the prompt to use for each:
Take photos of your fridge, or pantry and ask AI:
“What meals can I make from these ingredients in under 20 minutes?”
Upload photos of different pieces of clothing and ask AI:
“What pieces am I missing to help make this a capsule wardrobe?”
or
“What can you tell me about my style from these photos?”
Take a photo of handwritten notes or a whiteboard and ask:
“Turn this into organized notes and give me a summary I can send to my colleagues.”
Upload photos of a room and ask:
“How would you redesign this space to make it more functional?”
Too Easy?
Try combining multiple types of context together.
For example:
upload screenshots of your calendar + inbox + task list
upload a photo of your whole workspace
upload meal photos across the week
upload screenshots of analytics or dashboards over time
Then ask AI to synthesize patterns across all of it using this prompt:
“I’m going to upload several different types of images from my work and life. I want you to look for patterns, inefficiencies, priorities, habits, or blind spots that I may not be noticing. Be specific and practical. If you need more context before answering, ask follow-up questions first.”
Why this matters?
Most people still think of AI as mostly text based. But the larger shift happening is that AI systems are becoming multimodal. AI can increasingly reason across text, images, documents, voice, diagrams, and pictures and pull context from each of these sources. That changes the way we interact with AI systems.
Instead of translating your life into text for a machine, you can increasingly just show the machine the context directly.
As AI has continued to improve, prompting has become a less valuable skill, while the ability to provide good context is becoming more and more important. These systems are designed to operate more like colleagues, so the more context they have, the more helpful they can be.
The improvements in imaging and image recognition also point to the interest the major AI labs have in AI working inside phones, glasses, cars, homes, workplaces, robots, and everyday devices. The long-term direction is not just AI that can answer questions, but AI that can observe, understand, and interact with the world around you in real time.
That’s it for this week.
Hit reply and tell me what you thought of the challenge this week.
I read every response, and your experience helps me shape these challenges. If you get stuck or have questions, reach me at [email protected]