I tested Google Bard. It was surprising — in a bad way

March 22, 2023

`I tested Google Bard. It was surprising -- in a bad way`

Innovation

I tested Google Bard. It was surprising — in a bad way

AI chatbots have wowed users with their text generation and other advanced abilities, as well as some bizarre behavior. Bard has its own surprises in store.

Written by Sabrina Ortiz, Associate Editor on

Google Bard got off to a rocky start when the demo provided incorrect information about the James Webb Telescope. It’s been over a month since that incident and Google just began rolling out access to Bard via a waitlist. ZDNET got access and here are my first impressions.

Also: What is Google Bard? Here’s everything you need to know

ChatGPT has the ability to write code, hold conversations, pass benchmark exams, co-edit papers and so much more. Its biggest limitations are that it doesn’t have access to the internet and is limited to information before 2021.

With that in mind, in theory, an AI chatbot with access to the internet should make the chatbot just that much more capable than ChatGPT. Bing Chat met that expectation and placed the bar higher, and I was ready to be knocked off my feet with Google Bard. I was in for a surprise.

Despite being integrated with the world’s most popular search engine, Google Bard is limited in its function, knowledge and features.

The first difference I noticed was how long it takes to generate an answer. Although it takes under thirty seconds, that’s considerably longer than ChatGPT and Bing Chat take to generate answers.

Also: 5 best AI art generators

This could be caused by the difference between the large language models (LLMs) used. Both Bing Chat and ChatGPT run on GPT series LLMs, as opposed to Bard which runs on a lightweight version of LaMDA, an LLM exclusive to Google.

Another thing that surprised me was the chatbot’s inability to answer many straightforward, textbook questions. For example, I asked Bard “Who are all the presidents of the United States of America?” and was met with no response.

google-bard-us-presidents reuslts screenshot — Screenshot by Sabrina Ortiz/ZDNET

Users on Twitter are also sharing frustrations with the same sorts of Google Bard idiosyncrasies. One user shared that Google Bard couldn’t provide her with the response to what the 50 states of America are.

Considering I, a mere human, can recall and sing you the 50 states song I learned in elementary school, this seems like something Bard should know.

PLEASE USE GOOGLE BARD
PLEASE USE IT IT’S JUST LIKE CHATGPT I PROMISE pic.twitter.com/XcJJhwVpU7

— near (@nearcyan) March 22, 2023

Another issue I have with the chatbot is that it does not provide sources to the answer it generates. Bing Chat, which is also an AI chatbot connected to the internet provides footnotes and sources that you can click on to learn more or verify the information. With Bard, you just have to take its word.

Also: How to make ChatGPT provide sources and citations

For example, for the “What are the biggest news stories today?” prompt, Bard did a nice job at rounding up the latest stories, but if I wanted to learn more about any of them, I would have to go to the search bar and type it in myself.

Although ChatGPT doesn’t provide sources either, which is a major downside of the chatbot, it makes sense because it is not connected to the internet. That is one of the first issues a chatbot linked to the web should have worked to resolve.

Also: How to use ChatGPT: Everything you need to know

Once you get your response, there is a button that says “Google it” which conveniently brings you to the Google search engine results for your prompt. However, you still have to scroll to find your answer like you would with a regular Google search as seen below.

The chatbot’s explanations of basic subjects falls short of what even a Google search could provide. Ethan Mollick, a professor at Wharton who has been documenting his experience with AI from the release of ChatGPT, tested Bard for its learning capabilities and had a similar experience.

In a paper he worked on, he delineated some prompts that could help an AI chatbot accelerate learning for students. Google Bard failed to output results that were on par to ChatGPT’s.

Its early, but Google’s Bard does not seem as capable as a learning tool as Bing or GPT-4. In our paper on AI and teaching, we outline some AI prompts that can help. Not only does Bard not do interactive prompts, but the results are poor so far.
Paper: https://t.co/yG23LUljdV pic.twitter.com/1sHdLkDdvP

— Ethan Mollick (@emollick) March 22, 2023

Despite proving to not be an ideal tool for learning or searching information, I found that Bard can help with career building tools such as resume, cover letter and essay generating.

Also: How to use ChatGPT to build your resume

I asked Google Bard to output both resume bullet points and professional summary for my resume based only on my job title. Bard did a decent job at providing some basic information I could work from or tweak to include in a resume.

For its text generating aspect, Bard held its weight as well. I input text from a previous article I had written about Bard for ZDNET and asked Bard to proofread it. Within seconds it provide easy to implement suggestions that could help optimize the text.

To test its writing skills, I also asked the AI chatbot to write me an essay on Google Bard and it generated a pretty lengthy and detailed description of Bard. Although it showed no resources, neither does ChatGPT, so when using text from either of those chatbots, you would want to fact check.

Lastly, it was time to test the chatbot’s coding skills, which is where Bard failed. I input two different coding prompts, including a very straightforward one and a more convoluted one. For both, Bard didn’t have an answer and shot me the same error message.

Also: How to use ChatGPT to write code

The more straightforward prompt was, “Write a function to reverse a string in Python.” I input this prompt into all three chatbots to see the difference between responses. As seen below, Google Bard’s was the worst as it couldn’t even generate one.

Bard’s response to the prompt:

google-bard-write-a-function-to-reverse-a-string-in-python-2.png — Screenshot by Sabrina Ortiz/ZDNET

ChatGPT’s response to the prompt:

chatgpt-write-a-function-to-reverse-a-string-in-python.png — Screenshot by Sabrina Ortiz/ZDNET

Bing Chat’s response to the prompt:

bing-chat-write-a-function-to-reverse-a-string-in-python.png — Screenshot by Sabrina Ortiz/ZDNET

The results are disappointing because one of ChatGPT’s biggest advantages is its advanced coding abilities and Bard has proven to not be reliable in that area.

Also: The best AI chatbots to try

However, Bard is a reliable tool as a text editor or generator, which can include difficult career tasks such as composing a cover letter, resume or even creating essays.

Although Bard is still useful, ChatGPT and Bing Chat are more capable overall, which is surprising since Bard is relying on the biggest search engine in the world.

Also: How to use DALL-E 2 to turn your creative visions into AI-generated art

It is important to acknowledge that Google Bard is still in its very early stages and using a limited version of LaMDA. Google said in its Bard announcement that the chatbot, “will be updated with newer, more capable models over time.”

About The Author

admin