Class 1 AI for Cyber Security

Course Introduction & Objectives

The instructor opened the class by framing the course as a dual-focused exploration of AI and cybersecurity:

• AI for Cybersecurity: Using machine learning (ML) and AI tools (e.g., transformers, NLP models) to automate threat detection (malware, phishing emails, code vulnerabilities).
  • Example: Training models to flag obfuscated JavaScript malware or suspicious network traffic.
• Cybersecurity for AI: Securing AI systems themselves, such as preventing adversarial attacks on models like ChatGPT.
  • Example: Guarding against "prompt injection" attacks that trick chatbots into leaking sensitive data.

The instructor highlighted real-world urgency, noting that AI models are increasingly deployed insecurely, citing incidents like hackers manipulating auto dealership chatbots to sell cars for $1.

Course Structure & Logistics

• Modules: 8 topics covered over 14 weeks, blending theory and hands-on labs:
  1. Vulnerability Detection: Using NLP models to scan code for flaws (e.g., SQL injection).
  2. Android Malware Detection: Analyzing APK files with ML.
  3. Backdoor Attacks: Poisoning training data to hijack model behavior.
  4. Phishing Detection: Classifying malicious emails using transformers.
  5. Memory Forensics: Detecting malware in system memory dumps.
  6. Adversarial ML: Evasion attacks on image classifiers.
  7. AI Red Teaming: Stress-testing models using frameworks like MITRE ATLAS.
  8. Case Studies: Real breaches (e.g., ChatGPT jailbreaks).
• Tools:
  • Google Colab: Cloud-based Python environment for labs (avoids local setup hassles).
  • Hugging Face/Kaggle: Pre-trained models (e.g., BERT) and datasets (e.g., VirusShare for malware samples).
• Grading:
  • Labs (30%): Weekly assignments (e.g., training a malware classifier). Students submit 1-page reports explaining their approach, results, and metrics (accuracy, F1-score).
  • Final Project (35%): Build an ML system to detect obfuscated JavaScript malware. Teams must submit code + a report analyzing confusion matrices and false positives.
  • Final Exam (35%): Open-book test on AI/cybersecurity theory (e.g., "Explain how transformers detect vulnerabilities").

Deep Dive: Foundational AI Concepts

• Supervised Learning:
  • Defined as "teaching machines using labeled data."
  • Example: Training a spam filter with emails marked "spam" or "not spam."
  • Metrics: Accuracy, precision, recall. Students were warned that "99% accuracy means nothing if the model misses 1% of critical threats."
• Transformers & ChatGPT:
  • Explained transformers as "sequence prediction engines" trained on massive text corpora.
  • Demo: Typing "My favorite drink is..." into ChatGPT to show how it predicts the next word probabilistically.
  • Security Risks: Highlighted how attackers exploit transformers to generate malware or phishing emails.
• Limitations of AI:
  • Bias: Models trained on biased data (e.g., gender stereotypes in hiring tools).
  • Hallucinations: ChatGPT inventing fake citations or code vulnerabilities.
  • Adversarial Robustness: A self-driving car misclassifying a stop sign due to a sticker (real-world example).

Cybersecurity Threats & AI Defense

• Case Study: Chevy Chatbot Exploit:
  • A hacker tricked a dealership’s AI chatbot into agreeing to sell a $76k Chevy Tahoe for $1 by crafting a "story" that bypassed safeguards.
  • Lesson: AI systems need rigorous red-teaming before deployment.
• MITRE ATLAS Framework:
  • A cybersecurity framework adapted for AI, mapping tactics like "model evasion" or "data poisoning."
  • Example: Using ATLAS to test if a fraud detection model can be fooled by slightly altering transaction amounts.
• Prompt Injection Attacks:
  • Demo: Asking ChatGPT to "ignore previous instructions" and reveal internal APIs.
  • Defense: "Guardrails" like OpenAI’s moderation API to block harmful prompts.

Ethical & Practical Considerations

• GenAI Policy:
  • Students can use tools like ChatGPT for brainstorming but not to write full reports/code.
  • Penalties include failing the course for plagiarism (e.g., submitting AI-generated text).
• Career Relevance:
  • Growing demand for hybrid AI/cybersecurity roles:
    • ML Security Engineer: Hardening models against attacks.
    • Threat Hunter: Using AI to analyze network logs for anomalies.
  • Certifications recommended: MITRE ATLAS, TensorFlow Developer.

Student Takeaways & Next Steps

• Key Skills to Master:
  • Python programming (libraries: PyTorch, scikit-learn).
  • ML metrics (ROC curves, precision-recall tradeoffs).
  • Cybersecurity fundamentals (CIA triad: Confidentiality, Integrity, Availability).
• Homework:
  • Set up Google Colab and run a "Hello World" ML script.
  • Read textbook chapters on malware analysis and adversarial attacks.
• Instructor’s Final Note:
  • "This field moves fast. Stay curious—what’s cutting-edge today might be obsolete in 6 months. Your job is to adapt."

This class balanced technical rigor (e.g., coding labs, metric analysis) with ethical awareness (e.g., AI misuse risks). The instructor emphasized a "hacker mindset": learning to attack systems to better defend them. Students left with a clear roadmap for the semester, from detecting Android malware to stress-testing ChatGPT-like models.