Navigating Data Privacy

This Stanford study examined how six major AI companies (Anthropic, OpenAI, Google, Meta, Microsoft, and Amazon) handle user data from chatbot conversations.  Here are the main privacy concerns. 👀 All six companies use chat data for training by default, though some allow opt-out 👀 Data retention is often indefinite, with personal information stored long-term 👀 Cross-platform data merging occurs at multi-product companies (Google, Meta, Microsoft, Amazon) 👀 Children's data is handled inconsistently, with most companies not adequately protecting minors 👀 Limited transparency in privacy policies, which are complex and hard to understand and often lack crucial details about actual practices Practical Takeaways for Acceptable Use Policy and Training for nonprofits in using generative AI: ✅ Assume anything you share will be used for training - sensitive information, uploaded files, health details, biometric data, etc. ✅ Opt out when possible - proactively disable data collection for training (Meta is the one where you cannot) ✅ Information cascades through ecosystems - your inputs can lead to inferences that affect ads, recommendations, and potentially insurance or other third parties ✅ Special concern for children's data - age verification and consent protections are inconsistent Some questions to consider in acceptable use policies and to incorporate in any training. ❓ What types of sensitive information might your nonprofit staff  share with generative AI?  ❓ Does your nonprofit currently specifically identify what is considered “sensitive information” (beyond PID) and should not be shared with GenerativeAI ? Is this incorporated into training? ❓ Are you working with children, people with health conditions, or others whose data could be particularly harmful if leaked or misused? ❓ What would be the consequences if sensitive information or strategic organizational data ended up being used to train AI models? How might this affect trust, compliance, or your mission? How is this communicated in training and policy? Across the board, the Stanford research points that developers’ privacy policies lack essential information about their practices. They recommend policymakers and developers address data privacy challenges posed by LLM-powered chatbots through comprehensive federal privacy regulation, affirmative opt-in for model training, and filtering personal information from chat inputs by default. “We need to promote innovation in privacy-preserving AI, so that user privacy isn’t an afterthought." How are you advocating for privacy-preserving AI? How are you educating your staff to navigate this challenge? https://lnkd.in/g3RmbEwD

McDonald’s exposed 64 million jobseekers’ data; because someone used “123456” as the password. This was a basic hygiene failure: ❌ No password security ❌ No 2FA ❌ No API access restrictions ❌ No vendor oversight Let me be direct with you… Too many companies are rushing to add AI to their hiring. But they’re skipping the fundamentals: - No one’s thinking about privacy - No one’s thinking about security - No one’s asking the hard questions about vendor risk. Here’s what happened: McDonald's used a third-party AI chatbot for recruitment. The test admin account was never disabled. No two-factor authentication. No real access controls. And in less than 30 minutes, researchers were inside the system. They pulled full names, emails, phone numbers, locations Even private chat transcripts from job seekers. 64 million people. This wasn’t some advanced cyberattack. It was a copy-paste URL and a default password. The real problem: Privacy and security are still treated as afterthoughts; Even in systems that collect sensitive personal data. And if McDonald’s can get it this wrong… What do you think is happening in businesses with no privacy team at all? It’s time to stop outsourcing risk and hoping for the best. ✔️ If you’re building with AI → bake in privacy from the beginning ✔️ If you’re using vendors → demand more than checkboxes and SLAs. ✔️ If you're leading privacy → don't assume → Verify. Because trust isn’t built with buzzwords. It’s built with discipline. If you’re serious about building trust in 2025, stop playing catch-up; Build your privacy and security programs with intention. Because next time, it could be your brand on the front page.

UPDATE: On November 22, the update was added to the article basically saying that Google’s recent wording change around Gmail “smart features” caused major confusion — including early reports suggesting emails were being used to train Google’s AI models by default. After reviewing Google’s documentation, the author of the article concluded that “it doesn’t appear to be the case”. Gmail does scan content for built-in features like spam filtering and suggestions, but that is supposedly separate from training generative AI. 🤔 “… doesn’t appear to be the case” is the operative phrase in that update… Isn’t it? (Link to the updated source is in the comments). 🚨 Heads-up, cyber friends: your inbox might be humming with more than just deadlines. According to Malwarebytes, Gmail is automatically opting you in to have all your emails and attachments used for training its AI models. Unless you manually opt out, your private correspondence may now be fueling AI-features behind the scenes. Here are the key takeaways: 🔍 Opt-in by default matters — Instead of asking you first, the service assumes consent. This flips the script on personal privacy: it’s no longer “do you want to participate?” but “you are participating unless you act.” That shifts the power and — for many — erodes trust. 🤖 Training AI on consumer data without explicit consent is becoming a worrying trend. Using everyday user content (emails, attachments, chats) to refine AI models means personal information is being repurposed in unexpected ways. Even if anonymized, the fact that your private communications become a training set should raise eyebrows. 🛡️ Implications for professionals and individuals alike — If you handle sensitive info (clients, students, research, education), this isn’t just a nuisance; it’s a risk. Consent needs to be real, transparent and meaningful — not buried under settings toggles. 🧠 What you can do: Go into your Gmail settings, turn off “Smart features” in both Gmail/Chat/Meet and Workspace sections. Because yes, you have to flip both. In an era where data is called “the new oil,” assuming people want to pump their private life into AI-refineries without explicit agreement feels deeply off-brand for what privacy should mean. If we’re teaching the next generation how to think, how to work ethically, we can’t give tacit permission to a default that says “we’ll use your stuff unless you speak up.” As someone who lives at the intersection of cybersecurity, teaching, and digital citizenship, I say: We have to call this out. Let’s insist that “Yes” means yes, not “We quietly opted you in; you could opt out if you found it.” Control over personal data isn’t a bonus—it’s fundamental. #WomenInCyber #CyberSecurityLeadership #DataPrivacy #AIethics #ConsentFirst #StopAndSmellTheFlowers #ISSA #CyberThreatIntelligence #TechTrends #DigitalRights

This new white paper by Stanford Institute for Human-Centered Artificial Intelligence (HAI) titled "Rethinking Privacy in the AI Era" addresses the intersection of data privacy and AI development, highlighting the challenges and proposing solutions for mitigating privacy risks. It outlines the current data protection landscape, including the Fair Information Practice Principles, GDPR, and U.S. state privacy laws, and discusses the distinction and regulatory implications between predictive and generative AI. The paper argues that AI's reliance on extensive data collection presents unique privacy risks at both individual and societal levels, noting that existing laws are inadequate for the emerging challenges posed by AI systems, because they don't fully tackle the shortcomings of the Fair Information Practice Principles (FIPs) framework or concentrate adequately on the comprehensive data governance measures necessary for regulating data used in AI development. According to the paper, FIPs are outdated and not well-suited for modern data and AI complexities, because: - They do not address the power imbalance between data collectors and individuals. - FIPs fail to enforce data minimization and purpose limitation effectively. - The framework places too much responsibility on individuals for privacy management. - Allows for data collection by default, putting the onus on individuals to opt out. - Focuses on procedural rather than substantive protections. - Struggles with the concepts of consent and legitimate interest, complicating privacy management. It emphasizes the need for new regulatory approaches that go beyond current privacy legislation to effectively manage the risks associated with AI-driven data acquisition and processing. The paper suggests three key strategies to mitigate the privacy harms of AI: 1.) Denormalize Data Collection by Default: Shift from opt-out to opt-in data collection models to facilitate true data minimization. This approach emphasizes "privacy by default" and the need for technical standards and infrastructure that enable meaningful consent mechanisms. 2.) Focus on the AI Data Supply Chain: Enhance privacy and data protection by ensuring dataset transparency and accountability throughout the entire lifecycle of data. This includes a call for regulatory frameworks that address data privacy comprehensively across the data supply chain. 3.) Flip the Script on Personal Data Management: Encourage the development of new governance mechanisms and technical infrastructures, such as data intermediaries and data permissioning systems, to automate and support the exercise of individual data rights and preferences. This strategy aims to empower individuals by facilitating easier management and control of their personal data in the context of AI. by Dr. Jennifer King Caroline Meinhardt Link: https://lnkd.in/dniktn3V

AI vendor management. One of the most pressing challenges companies face these days is vetting and contracting with AI vendors. This comes up in two contexts: (a) vetting solutions from AI vendors that your company considers adopting, and (b) vetting solutions from AI vendors that your vendors have adopted. Where do you start? *** The second question comes up a lot in a GDPR or state privacy law context. Your vendors (processors / service providers) are required by law and/or contract to notify you when they start using a new subprocessor. Consider that companies with hundreds of vendors now get thousands such notices. “We started using ChatGPT”. “We’re now using GitHub Copilot”. What’s a GC or CPO to do with such notices? In many cases they haven’t even approved the use of the same tools internally themselves.... *** When the EU AI Act comes into force, under Article 10(6a) of the Parliament’s draft, the obligations of an AI provider could flow down to deployers of AI: “Where the provider cannot comply with the obligations laid down in this Article because that provider does not have access to the data and the data is held exclusively by the deployer, the deployer may, on the basis of a contract, be made responsible for any infringement of this Article.” All the more reason for companies to *closely* vet the solutions they’re implementing. *** And the new draft CCPA regs on risk assessments, require “A service provider or contractor ... [to] cooperate with the business in its conduct of a risk assessment pursuant to Article 10...” The regs focus on such risk assessments for automated decision making as well as the use of data for training AI.  https://lnkd.in/ejk4fYgQ *** There are a few useful checklists out there. The attached was created by Amber Nicole Ezzell from FPF based on convos with more than 30 experts. https://lnkd.in/e8E_t64W This one from PwC is usefully role based: https://lnkd.in/eZwVAZmB And this one from CNIL provides insight into a regulator’s approach: https://lnkd.in/ewgBrXKj *** I suggest going back to basics: what are the risks to the company’s PII and IP? Is there potential for bias or discrimination? Are there accountability mechanisms, including audit and log trails? Can you ensure explainability?   *** From a contractual perspective: look closely at the definitions of customer data, usage data and confidential information. *** I also recommend consulting with outside counsel. From our vantage point, we see how many companies across industry sectors – both vendors and deployers - cope with and respond to these complex challenges.  

𝟔𝟔% 𝐨𝐟 𝐀𝐈 𝐮𝐬𝐞𝐫𝐬 𝐬𝐚𝐲 𝐝𝐚𝐭𝐚 𝐩𝐫𝐢𝐯𝐚𝐜𝐲 𝐢𝐬 𝐭𝐡𝐞𝐢𝐫 𝐭𝐨𝐩 𝐜𝐨𝐧𝐜𝐞𝐫𝐧. What does that tell us? Trust isn’t just a feature - it’s the foundation of AI’s future. When breaches happen, the cost isn’t measured in fines or headlines alone - it’s measured in lost trust. I recently spoke with a healthcare executive who shared a haunting story: after a data breach, patients stopped using their app - not because they didn’t need the service, but because they no longer felt safe. 𝐓𝐡𝐢𝐬 𝐢𝐬𝐧’𝐭 𝐣𝐮𝐬𝐭 𝐚𝐛𝐨𝐮𝐭 𝐝𝐚𝐭𝐚. 𝐈𝐭’𝐬 𝐚𝐛𝐨𝐮𝐭 𝐩𝐞𝐨𝐩𝐥𝐞’𝐬 𝐥𝐢𝐯𝐞𝐬 - 𝐭𝐫𝐮𝐬𝐭 𝐛𝐫𝐨𝐤𝐞𝐧, 𝐜𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞 𝐬𝐡𝐚𝐭𝐭𝐞𝐫𝐞𝐝. Consider the October 2023 incident at 23andMe: unauthorized access exposed the genetic and personal information of 6.9 million users. Imagine seeing your most private data compromised. At Deloitte, we’ve helped organizations turn privacy challenges into opportunities by embedding trust into their AI strategies. For example, we recently partnered with a global financial institution to design a privacy-by-design framework that not only met regulatory requirements but also restored customer confidence. The result? A 15% increase in customer engagement within six months. 𝐇𝐨𝐰 𝐜𝐚𝐧 𝐥𝐞𝐚𝐝𝐞𝐫𝐬 𝐫𝐞𝐛𝐮𝐢𝐥𝐝 𝐭𝐫𝐮𝐬𝐭 𝐰𝐡𝐞𝐧 𝐢𝐭’𝐬 𝐥𝐨𝐬𝐭? ✔️ 𝐓𝐮𝐫𝐧 𝐏𝐫𝐢𝐯𝐚𝐜𝐲 𝐢𝐧𝐭𝐨 𝐄𝐦𝐩𝐨𝐰𝐞𝐫𝐦𝐞𝐧𝐭: Privacy isn’t just about compliance. It’s about empowering customers to own their data. When people feel in control, they trust more. ✔️ 𝐏𝐫𝐨𝐚𝐜𝐭𝐢𝐯𝐞𝐥𝐲 𝐏𝐫𝐨𝐭𝐞𝐜𝐭 𝐏𝐫𝐢𝐯𝐚𝐜𝐲: AI can do more than process data, it can safeguard it. Predictive privacy models can spot risks before they become problems, demonstrating your commitment to trust and innovation. ✔️ 𝐋𝐞𝐚𝐝 𝐰𝐢𝐭𝐡 𝐄𝐭𝐡𝐢𝐜𝐬, 𝐍𝐨𝐭 𝐉𝐮𝐬𝐭 𝐂𝐨𝐦𝐩𝐥𝐢𝐚𝐧𝐜𝐞: Collaborate with peers, regulators, and even competitors to set new privacy standards. Customers notice when you lead the charge for their protection. ✔️ 𝐃𝐞𝐬𝐢𝐠𝐧 𝐟𝐨𝐫 𝐀𝐧𝐨𝐧𝐲𝐦𝐢𝐭𝐲: Techniques like differential privacy ensure sensitive data remains safe while enabling innovation. Your customers shouldn’t have to trade their privacy for progress. Trust is fragile, but it’s also resilient when leaders take responsibility. AI without trust isn’t just limited - it’s destined to fail. 𝐇𝐨𝐰 𝐰𝐨𝐮𝐥𝐝 𝐲𝐨𝐮 𝐫𝐞𝐠𝐚𝐢𝐧 𝐭𝐫𝐮𝐬𝐭 𝐢𝐧 𝐭𝐡𝐢𝐬 𝐬𝐢𝐭𝐮𝐚𝐭𝐢𝐨𝐧? 𝐋𝐞𝐭’𝐬 𝐬𝐡𝐚𝐫𝐞 𝐚𝐧𝐝 𝐢𝐧𝐬𝐩𝐢𝐫𝐞 𝐞𝐚𝐜𝐡 𝐨𝐭𝐡𝐞𝐫 👇 #AI #DataPrivacy #Leadership #CustomerTrust #Ethics

The US is looking to require online platforms to obtain your express consent before they can use your data to train their AI. But does consent still work in the age of AI? This week, US senators (Ben Ray Luján and Peter Welch) introduced the draft bill of the "Artificial Intelligence Consumer Opt-In, Notification Standards, and Ethical Norms for Training Act" (the AI CONSENT Act). What a well-put acronym! Key points about this bill: ▪ This bill would apply to "covered entity" - i.e. person/partnership/corporation subject to the jurisdiction of the Federal Trade Commission (FTC) under section 5(a)(2) of the FTC Act (which prohibits unfair or deceptive practices). Basically, anyone who needs to comply with unfair/deceptive practice law needs to comply with this bill (which is a broad scope). But the relevant press release suggests that this bill is really intended to target online platforms. ▪This bill that would require covered entities to obtain consumers’ "express informed consent" before using their personal data (known as "covered data") to train AI models. ▪Failure to do so would be considered an unfair or deceptive practice, subject to FTC enforcement. ▪The bill also directs the FTC to study the efficacy of data de-identification (e.g. anonymising data). 💡 MY THOUGHTS 1️⃣ While consent regimes are not new, the consumer law angle is interesting. Normally, consent regimes on the use of data for AI fall under privacy law. If a company misused your personal data for AI training, your remedy was limited to seeking injunctions or civil penalties for breach of privacy (which can differ between states). But this bill will create a consumer law angle to AI training - i.e. if your personal data is misused for AI training, you will have access to both privacy and consumer law remedies. It also means more FTC action this space. 2️⃣ But does consent still work in this age? What if the required consent wording is buried deep within clickwrap T&Cs which can only be accepted or rejected as a whole (not parts of it)? At least, the AI CONSENT bill will: ✔ require the covered entity to make "clear and conspicuous disclosure" of how they will use your data ✔ provide standards around how the disclosure must be made (e.g. size, font, colour, or other visual affects of the disclosure wording, etc) ✔ require that the user has the option to withhold or revoke consent that is "at least as prominent as the option to accept" Based on these proposed requirements, it might be that notices of AI training will look similar to flashy pop-up cookie notices on websites. So perhaps the consent might still work. But I can see AI training consent becoming a battle of words, where companies skilfully draft consent notices that barely comply with the law but collects the most data. Thoughts? 👓 Want more? Check out my global AI regulation tracker: https://lnkd.in/gtTBKhPp #ai #artificialintelligence #airegulation #usa #breaking

How To Handle Sensitive Information in your next AI Project It's crucial to handle sensitive user information with care. Whether it's personal data, financial details, or health information, understanding how to protect and manage it is essential to maintain trust and comply with privacy regulations. Here are 5 best practices to follow: 1. Identify and Classify Sensitive Data Start by identifying the types of sensitive data your application handles, such as personally identifiable information (PII), sensitive personal information (SPI), and confidential data. Understand the specific legal requirements and privacy regulations that apply, such as GDPR or the California Consumer Privacy Act. 2. Minimize Data Exposure Only share the necessary information with AI endpoints. For PII, such as names, addresses, or social security numbers, consider redacting this information before making API calls, especially if the data could be linked to sensitive applications, like healthcare or financial services. 3. Avoid Sharing Highly Sensitive Information Never pass sensitive personal information, such as credit card numbers, passwords, or bank account details, through AI endpoints. Instead, use secure, dedicated channels for handling and processing such data to avoid unintended exposure or misuse. 4. Implement Data Anonymization When dealing with confidential information, like health conditions or legal matters, ensure that the data cannot be traced back to an individual. Anonymize the data before using it with AI services to maintain user privacy and comply with legal standards. 5. Regularly Review and Update Privacy Practices Data privacy is a dynamic field with evolving laws and best practices. To ensure continued compliance and protection of user data, regularly review your data handling processes, stay updated on relevant regulations, and adjust your practices as needed. Remember, safeguarding sensitive information is not just about compliance — it's about earning and keeping the trust of your users.

The internet must be a safe space for everyone. Especially children and young people. Today, President Ursula von der Leyen and I outlined two key actions shaping our work across the EU: rolling out a privacy-preserving, EU-wide age verification solution and enforcing the Digital Services Act. Age verification solution is ready as of today. It allows users to prove their age securely and anonymously using zero-knowledge proof. No personal data is shared. Verification happens through an anonymous QR code exchange, without requiring platforms to collect sensitive data. Next step: scale. One interoperable solution across the EU, not 27 fragmented systems. Seven Member States are already acting as front-runners, and the open-source blueprint allows private companies to build compatible solutions under strict privacy standards. A safer internet for children is not optional. We’re making it happen.

Microsoft AI Teams will soon tell your boss where you are. Starting December 2025, Teams can automatically detect when you connect to your company’s Wi-Fi and update your location to “in the office.” It sounds like a small feature. It isn’t. Location tracking through workplace networks is the newest frontier in digital surveillance, and it’s coming through your collaboration software. Microsoft says the feature is opt-in. That is very good. But, that decision will rest largely with employers and admins, not the average employee trying to meet deadlines. If you work for a Microsoft-using organization, now is the time to ask: Is our company planning to activate this feature? Has consent been properly documented? If you represent a union, this deserves to be on your next agenda. The GDPR and UK Data Protection Act require transparency, necessity, and proportionality for any location tracking. Under the EU AI Act, this may also fall under high-risk processing of biometric and personal data for workplace management. Employers must conduct a fundamental rights impact assessment before rolling it out. This isn’t paranoia. It is risk management, employee rights, and compliance. Workplace tracking without explicit, informed consent can violate privacy law in multiple jurisdictions, and it may open employers to liability under both GDPR and the EU AI Act’s risk provisions. If your organization uses Microsoft Teams with minors, such as schools or training programs, the stakes are even higher. Here’s what to do as an employee, parent, or guardian: 🔹 Ask your IT administrator if “location autodetection” is enabled. 🔹 Request a copy of the company’s Data Protection Impact Assessment (DPIA). 🔹 Ensure opt-in consent is voluntary and revocable. 🔹 Check that logs are deleted regularly and not used for performance evaluation. Transparency is not optional. #DigitalSovereignty #WorkplacePrivacy #AICompliance #GDPR #MicrosoftTeams Image source: SlashGear, https://lnkd.in/di5WvY2e From Microsoft: Microsoft 365 Roadmap: https://lnkd.in/dYc3N9TX Microsoft Learn (Configure auto-detect of work location): https://lnkd.in/dtEkYNqB