By AppleVis
The Be My Eyes app is set to gain a new feature called the Virtual
Volunteer, powered by OpenAI's recently announced GPT-4 model.
This new tool has the potential to be a game changer for people with visual
impairments, providing a virtual sighted assistant that can generate context
and understanding for images in the same way as a human volunteer can.
Be My Eyes has been providing technology for the blind and low-vision
community since 2012, connecting users with volunteers for assistance with
everyday tasks like navigating airports and identifying products. The new
feature, which uses GPT-4's visual input capability, will allow the app to
offer an even greater degree of independence to users and expand the use
cases for the app.
Michael Buckley, CEO of Be My Eyes, stated that "in the short time we've had
access, we have seen unparalleled performance to any image-to-text object
recognition tool out there," and added that "the implications for global
accessibility are profound. In the not so distant future, the blind and low
vision community will utilize these tools not only for a host of visual
interpretation needs, but also to have a greater degree of independence in
their lives."
The difference between GPT-4 and other language and machine learning models
is both the ability to have a conversation and the greater degree of
analytical prowess offered by the technology. Basic image recognition
applications can only identify what's in front of them, while GPT-4 can
extrapolate, analyze, and understand context, allowing it to offer much more
comprehensive assistance.
One of the most exciting aspects of the Virtual Volunteer feature is its
ability to offer instantaneous visual assistance for a wide variety of
tasks.
In a given example, users can send an image of the contents of their fridge
to the tool, which will then not only identify what's in the fridge, but
also suggest recipes that can be prepared with those ingredients.
The new feature has undergone beta testing with a select group of Be My Eyes
employees, yielding overwhelmingly positive feedback. Testers have praised
the feature's functionality, particularly in one instance where a user was
able to expertly navigate the railway system. The feature provided detailed
information about the user's location on a map and offered step-by-step
instructions on how to safely reach their desired destination.
In the video accompanying the announcement, a Be My Eyes user shows the app
helping her with a number of tasks, including describing the look of a
dress, identifying a plant, reading a map, translating a label, directing
her to a certain treadmill at the gym and telling her which buttons to push
at a vending machine.
Jesper Hvirring Henriksen, CTO of Be My Eyes, explains that traversing the
complicated physical world is only half the story. There are challenges
faced by visually impaired people when using screen readers to understand
web pages, especially when it comes to images. "GPT-4 is able to summarize
the search results the way the sighted naturally scan them-not reading every
minuscule detail but bouncing between important data points." This
technology could simplify tasks such as reading news online and navigating
cluttered e-commerce sites for people with visual impairments.
The Virtual Volunteer will be available to users in the coming months, and
it has been hailed as "game changing" by Buckley. "Ultimately, whatever the
user wants or needs, they can re-prompt the tool to get more information
that is usable, beneficial and helpful, nearly instantly."
If and when the Virtual Volunteer is unable to answer a question, it will
automatically offer users the option to be connected via the app to a
sighted volunteer for assistance.
The tool will be free for all blind and low-vision community members using
the Be My Eyes app.
You can currently register in the app to be placed on a waiting list for
access.
We are excited to discover how the Virtual Assistant will compare to other
similar apps in the market, such as Seeing AI and Envision AI.
If it lives up to the hype, this should indeed be a game changing
development.
===========================================================
The fb-exchange mailing list
Manage account,
List Page: https://www.freelists.org/list/fb-exchange
Archive: https://www.freelists.org/archive/fb-exchange
To unsubscribe: log onto the List page and select "Unsubscribe".
Administrative contact: insight@xxxxxxxxxxxxxxxxxxxx
===========================================================
