I Entered Google's Generative AI App Competition

August 27, 2024

I Entered Google's Generative AI App Competition

Hello there! I've been wanting to create something using AI, and when I heard about this competition, I decided to give it a shot!

Gemini API Developer Competition

A competition to develop innovative apps using Google's generative AI, Gemini. Open to participants worldwide.

Total prize money: 150 million yen, with the winner receiving a DeLorean!

This is Saya, the wife, speaking.

Generative AI × App Planning

With one month until the deadline, we started planning in early July. The concept:"A camera app that generates love poems from photos."

When I heard the theme was AI, I got the idea from the love letter ghostwriter that Mahiro, the protagonist of NHK's historical drama "Hikaru Kimi e" worked as.

Reasons for choosing this concept:

Positive concept
Seems doable in 3 weeks
No similar apps exist

Tackling Prompts

You can't avoid prompts when using generative AI, right? After finishing the planning and design, it was time for prompt engineering.

I tried prompt engineering for the first time

Using Google AI Studio

Google AI Studio

By asking Gemini, I created a prompt to generate short love poems based on photos. It's nicely inspired by the photos.

Expanding to 4 Prompts

This alone wasn't emotional enough, so I wanted to divide it into 4 love emotions for users to choose from when generating!

4 divided love emotions

💖 First love excitement
💑 Deep love with a partner
🖋 Philosophical love poem
💧 Bittersweet unrequited love

When I wrote prompts for each, the expressions became monotonous. For example, with [💧 Bittersweet unrequited love], phrases like "painful love" and "love is unfulfilled" kept repeating.

Fine-tuning

When I asked Gemini about this problem, it suggested, "Maybe there's not enough training data?" So, I created training data and set up a Python environment to clean the files for the first time!

But I could only create 20 data points... Even if I could make 1000, it wouldn't be nearly enough for training, so I gave up.

I guessed that the monotonous expressions might be due to mixing up the analysis and generation steps. I thought it might be better to use a program to combine a prompt that analyzes the photo and outputs text, with another that creates 4 types of poems from that. I tried creating a Python program with Gemini's help!

Changing Perspective

We tried prompts with a different approach, discussing various ideas. We found a method that worked and managed to improve it using just prompts. What a relief...

What Language is Best for Prompts?

Just when I thought we had finally created the prompts, switching the output to English resulted in rather strange poems.

Gemini explained that while Japanese focuses on syllable patterns like 5-7-5, English poetry emphasizes rhyme, among other differences.

Multilingual support isn't straightforward

We intended to call English prompts and output in each country's language, but we abandoned this idea as it lowered the quality of Japanese output. It didn't fit our concept, so we scrapped it.

We're writing prompts in both English and Japanese. Using only English would reduce input tokens and lower API usage costs, but unfortunately...

The App is Working!

The day before the deadline, we managed to call prompts from the program, take photos, and create poem photos! But then a problem arose. The output was different from yesterday!? Why...?

We repeatedly adjusted and tested to stabilize the output. We were able to shoot a demo video, so we're pulling an all-nighter to finish the submission video.