GenAI Mini-Tasks: Mining Themes in Survey Responses

using DataFramesMeta, CSV
using PromptingTools
const PT = PromptingTools

## Survey Austin
df_survey = CSV.File("Community_Survey_cItyofaustin.csv") |> DataFrame
last(names(df_survey), 10)

10-element Vector{String}:
 "Q25 - If there was one thing yo" ⋯ 77 bytes ⋯ "stion, etc.), what would it be?"
 "Q25 - One thing to share with Mayor Topics"
 "Quality of Life Topics"
 "Economic Opportunity and Affordability Topics"
 "Mobility Topics"
 "Health and the Environment Topics"
 "Safety Topics"
 "Government That Works for All Topics"
 "Culture and Lifelong Learning Topics"
 "Share with the Mayor Topics"

col = "Q25 - If there was one thing you could share with the Mayor regarding the City of Austin (any comment, suggestion, etc.), what would it be?"
responses = @chain df_survey begin
    @rsubset :Year == 2018
    dropmissing(col)
    _[:, col]
end

# Check available templates (optional as a reminder)
# aitemplates("Themes")

# Let's take a subset of 100 
sample_responses = last(responses, 100)
msg = aigenerate(:AnalystThemesInResponses; question=col, responses=sample_responses, instructions="None.")

[ Info: Tokens: 3785 @ Cost: \$0.0058 in 7.7 seconds
AIMessage("# Theme 1: Traffic and Transportation Improvement
- Best illustrated by: "Traffic is the main concern and needs to be addressed. Buses cause a major backup, they need to be allowed to pull over without backing up traffic even if the sidewalk needs to be pushed back in order to accommodate this, then it should be done. Many other cities do this."
- This theme is focused on the issues of traffic congestion, buses causing backups, and the need for improvement in transportation infrastructure.

# Theme 2: Affordable Housing
- Best illustrated by: "More affordable housing or part of Austin primary Black and Hispanic will disappear."
- This theme highlights the need for more affordable housing options in Austin to prevent displacement and maintain the diversity of the community.

# Theme 3: Homelessness and Public Safety
- Best illustrated by: "We have to do something about the homeless population. The homeless are downtown, under freeway passes, living in camps in the greenbelt, panhandling on every corner, all over UT Campus. Many seem to be on drugs and are unstable and aggressive. I don't feel safe at night walking downtown alone."
- This theme addresses concerns about the homeless population, including safety issues and the need for proper support and intervention strategies.")

using Markdown
msg.content |> Markdown.parse

aigenerate(:AnalystThemesInResponses; question=col, responses=sample_responses, instructions="Identify 10 themes. Focus on Green/Sustainability topics.", model="gpt4t")

[ Info: Tokens: 4137 @ Cost: \$0.0532 in 34.2 seconds
AIMessage("```


# Theme 1: Infrastructure and Traffic Management
- Best illustrated by: "Traffic is the main concern and needs to be addressed. Buses cause a major backup, they need to be allowed to pull over without backing up traffic even if the sidewalk needs to be pushed back in order to accommodate this, then it should be done. Many other cities do this."

# Theme 2: Public Transportation Improvements
- Best illustrated by: "WE NEED BETTER TRANSPORTATION. A LIGHT RAIL WOULD BE NICE."

# Theme 3: Affordable Housing and Cost of Living
- Best illustrated by: "More affordable housing or part of Austin primary Black and Hispanic will disappear."

# Theme 4: Homelessness and Support Services
- Best illustrated by: "We have to do something about the homeless population. The homeless are downtown, under freeway passes, living in camps in the greenbelt, panhandling on every corner, all over UT Campus. Many seem to be on drugs and are unstable and aggressive. I don't feel safe at night walking downtown alone."

# Theme 5: Environmental Sustainability and Clean Energy
- Best illustrated by: "IMPROVE PUBLIC TRANSPORTATION AND A BIGGER PUSH FOR GREEN ENERGY AND GENERAL SERVICES"

# Theme 6: Traffic and Non-Automotive Transportation Safety
- Best illustrated by: "Please keep bicycles off roads that are too narrow and have no bike lanes. Impress upon bicycle riders that they are REQUIRED to follow the same laws as cars if they are driving in the street lanes - NO RUNNING RED LIGHTS. If bicycle riders don't want cars to be rude to them, they need to ride appropriately."

# Theme 7: Green Spaces and Parks
- Best illustrated by: "I would like to have a city park near my home with a walking trail. Poor neighbourhoods rarely have a park nearby."

# Theme 8: Utilities and Energy
- Best illustrated by: "The electric bill is too high, even with assistance program."

# Theme 9: Environmental Health and Safety
- Best illustrated by: "Sanction those who blow leaves and trimmings into the street. Create ways to accelerate traffic flow on Lamar and Airport interception."

# Theme 10: Pollution Control and Waste Management
- Best illustrated by: "THERE ARE MANY CRAZY PEOPLE WHO WANT FLUORIDE OUT OF THE DRINKING WATER HERE, PLEASE SEE PAST THE CRAZY AND REALIZE THERE ARE RATIONAL, LOGICAL AND FISCAL REASONS TO REMOVE FLUORIDE FROM THE WATER, IT IS REMOVED PEOPLE CAN STILL OPT-IN AND THE CITY CAN GET THEM FLUORIDE IN ANOTHER WAYS BUT IF IT REMAINS IN IT IS FORCED ON EVERYONE EVEN THOSE WHO DON'T WANT IT AND THOSE WITH MEDICAL CONDITION FLUOROSIS REMOVE FLUORIDE ITS A WASTE OF MONEY"


```")