Assessment 2 Guidelines

Generative AI for Predictive Analysis

DATA5000 - Artificial Intelligence Programming in Business Analytics

Week 9 | In-Class Assessment

Assessment Overview

Weight: 35% of your final grade

Duration: 3 hours in class

Format: Group work (4-5 members)

Structure

Component	Marks	Time
Part A: Data Analysis & Business Problem	10 marks	Hour 1
Part B: Generative AI Marketing Strategy	15 marks	Hour 2
Part C: Team Presentation	10 marks	Hour 3

Scenario: Your team is a marketing agency analysing social media performance for a client using Facebook and Instagram data.

What You Submit

At End of Hour 2

Completed Notebook — Submit via Dropbox portal
Slide Deck — Submit via Turnitin portal

During Hour 3

Team Presentation — Present your campaign to the class

Submission Checklist

All code cells executed with outputs visible
All 10 questions answered
Business problem statement completed
All prompt engineering tasks filled in
Slide deck saved and uploaded

Warning: Uploading wrong files or incomplete work will be penalised as poor project management.

Part A: Data Analysis (6 marks)

You will analyse Facebook and Instagram campaign data to identify patterns and formulate a business problem.

Sections

Section	Tasks	Marks
Section 2: Exploratory Analysis	Calculate engagement rates, analyse by content type and age group	2
Section 3: SHAP Analysis	Run predictive model, interpret feature importance	2
Section 4: Treatment Effects	Calculate ATE and CATE for video content	1
Section 5: Business Problem	Write problem statement and identify stakeholders	1

Key Concepts Tested: Engagement rate calculation, SHAP interpretation, treatment effects (ATE/CATE), correlation vs causation

Part A: Code You Must Complete

Task 1: Calculate Engagement Rate

Formula: engagement_rate = (engagement / impressions) * 100

fb_data['engagement_rate'] = (fb_data['engagement'] / fb_data['impressions']) * 100
ig_data['engagement_rate'] = (ig_data['engagement'] / ig_data['impressions']) * 100
            

Task 2: Group by Content Type (Instagram)

Use the same pattern shown for Facebook, but with column name 'content_type'

ig_by_type = ig_data.groupby('content_type')['engagement_rate'].mean().sort_values(ascending=False)
            

Questions to Answer (Q1-Q7)

Read your code outputs carefully to answer:

Which platform/content type/age group performs best?
What does SHAP tell us (prediction vs causation)?
Which age group benefits most from video content?

Part A: Key Concepts to Understand

SHAP Values (Week 4)

SHAP shows which features predict the outcome.

SHAP does NOT prove causation.

Example: High impressions predicts high engagement, but buying impressions may not cause more engagement.

Treatment Effects (Week 4/5)

ATE: Average effect across everyone

CATE: Effect for specific subgroups

Example: Video content may work better for 18-24 age group than 45-54.

Business Problem Statement

Your statement should:

Be specific (mention actual numbers from your analysis)
Identify the opportunity or issue
Connect to stakeholder needs

Example format: "Analysis shows [finding] which suggests [opportunity]. This affects [stakeholders] because [impact]."

Part B: Generative AI Strategy (4 marks)

Use generative AI tools to develop a marketing campaign based on your Part A insights.

Sections

Section	Tasks	Marks
Section 6: Strategy Planning	Define target audience, objectives, and AI outputs needed	2
Section 7: Prompt Engineering	Write prompts using CRAFT framework	2

Key Concepts Tested: CRAFT framework, chain of thought prompting, temperature settings, prompt iteration

Part B: Prompt Engineering (Week 8)

CRAFT Framework

Letter	Meaning	Example
C	Clarity	Be specific about what you want
R	Role	"You are a social media marketing expert..."
A	Action	"Create 3 Instagram ad headlines..."
F	Format	"Present as a numbered list with..."
T	Tone	"Use energetic, youthful language"

Chain of Thought Prompting

Ask the AI to reason step-by-step:

"Think through this step-by-step:
1. First, identify the target audience characteristics
2. Then, determine what messaging appeals to them
3. Next, consider the platform best practices
4. Finally, recommend the campaign approach
Show your reasoning for each step."
            

Team Presentation (25 marks)

In Hour 3, your team will present your campaign to the class.

What to Include

Part A Insights Summary — Key findings from your data analysis
Business Problem — What problem you identified and why it matters
Target Audience — Who you are targeting and justification from data
Campaign Strategy — Your marketing approach
AI-Generated Assets — Show your slogans, visuals, and other outputs
Evaluation Strategy — How you will measure success?

Marking Criteria

Criteria	Marks
Overview of Part A insights and connection to campaign	5
Demonstration of AI outputs and their relevance	5
Business problem and stakeholder alignment	5
Use of creative tools	5
Clarity, conciseness, and engagement	5

Tips for Success

Do

Run all code cells before answering questions
Read outputs carefully before writing answers
Connect Part B strategy to Part A data insights
Use specific numbers from your analysis
Document all AI interactions as you go
Save your work frequently

Avoid

Submitting notebook with unexecuted cells
Generic answers not based on data
Forgetting to fill in the AI log
Claiming SHAP proves causation
Using wrong temperature for task type
Last-minute rush on documentation