The Feed · Complete archive
AI at work: research and practice
A server-rendered record of the evidence, ideas and firsthand practices screened by The Feed. Every entry links to its original source.
Use the interactive FeedPage 11 of 12 · 1144 items
practice · Artificial Ignorance ·
AI's Missing Multiplayer Mode
Framework for moving from using AI as a tool to treating it as a collaborative teammate in workflows.
Going from digital tools to digital teammates. Going from digital tools to digital teammates. When we look at the explosive growth of AI over the past few years, it's easy to be awestruck by the pace of innovation (and indeed, I often am). ChatGPT, Claude, and their increasingly capable cousi…
ways of working · Charlie Guo
research · arXiv ·
Precision Proactivity: Measuring Cognitive Load in Real-World AI-Assisted Work
Financial professionals using GPT-4o show higher quality work with AI assistance, but extraneous cognitive load from AI causes three times larger performance declines than task complexity.
Systems like ChatGPT and Claude assist billions through proactive dialogue-offering unsolicited, task-relevant information. Drawing on Cognitive Load Theory, we study how cognitive load shapes performance in AI-assisted knowledge work. We recruited 34 financial professionals to complete a complex valuation task using GPT-4o and developed a transcript-based framework estimating intrinsic and extraneous load from computational indicators anchored in a task decomposition and knowledge graph. Across 1,178 participant-subtask observations, AI-generated content usage is positively associated with qu
productivity · Matt Beane
practice · Paul Ford (Aboard) ·
The Funnel of the Future is Upside Down
AI-powered sales outreach is inverting the funnel: companies now need to build trust and deliver value before asking for attention.
You know what the marketing funnel is, right? You run ads on Google or Facebook so people learn about your brand or product, or you buy a list of names and send them cold emails. That’s “top of funnel.” Some fish nibble: They fill out a form, you schedule a call, and they don’t show up to the call. You send them a follow-up email saying you hope to talk soon anyway, because you are a worm with no pride. And so on, down the funnel, until eventually very few people are left, like at the end of a bad party, and you get one of them to sign on the line that is dotted before the lights come back on.
management org · Paul Ford
practice · Sangeet Paul Choudary ·
The fugu guide to jobs in a world of AI
AI doesn't just automate tasks; it breaks the systems and credentials that gatekeep work, forcing role redesign.
Stop measuring AI by the tasks it fails at. Start noticing the systems it breaks. Stop measuring AI by the tasks it fails at. Start noticing the systems it breaks. In Japan, a licensed fugu chef occupies a unique position in the food economy.
jobs skills · Sangeet Paul Choudary
practice · Harper Reed ·
Basic Claude Code
Developer workflow using reasoning models to generate specs and prompts, then Claude Code for implementation.
I really like this agentic coding thing. It is quite compelling in so many ways. Since I wrote that original blog post a lot has happened in Claude land: Claude Code MCP etc I have received hundreds (wat) of emails from people talking about their workflows and how they have used my workflow to get ahead. I have spoken at a few conferences, and taught a few classes about codegen. I have learned that computers really want to spellcheck codegen to codeine, who knew! I was talking to a friend the other day about how we are all totally fucked and AI will take our jobs (more on that in a later post)
ways of working · Harper Reed
practice · Paul Ford (Aboard) ·
Is “Specification Repair” the AI Endgame?
AI models use iterative code execution and self-prompting to refine reasoning, feeding output back as new input until solving problems.
Recently, I had a nice solid glimpse of the future and I wanted to tell you about it. Not long ago, I visited an office with a beautiful view, and I took this photo: She seems nice. That day, people online had been discussing how good ChatGPT was at guessing where a picture was taken , so I fed it this image and asked it not just to figure out what building, but which floor. It identified the exact building ( 17 State Street ), and then suggested it was the 34th floor (it was the 30th). This was, I thought, surprisingly accurate. This is another dangerous superpower we’re not prepared to handl
ways of working · Paul Ford
research · Management Science ·
Algorithm Reliance: Fast and Slow
Lab experiment shows workers rely on superior algorithms more under high load, improving quality and speed, but fail to speed up with inferior algorithms despite potential gains.
In algorithm-augmented service contexts where workers have decision authority, they face two decisions about the algorithm: whether to follow its advice and how quickly to do so. The pressure to work quickly increases with the speed of arriving customers. In this paper, we ask the following. How do workers use algorithms to manage system loads? With a laboratory experiment, we find that superior algorithm quality and high system loads increase participants’ willingness to use their algorithm’s advice. Consequently, participants with the superior algorithm make higher-quality recommendations th
judgment
research · arXiv ·
Regulating Algorithmic Management: A Multi-Stakeholder Study of Challenges in Aligning Software and the Law for Workplace Scheduling
Multi-stakeholder interviews reveal how regulatory gaps emerge when scheduling software, law, and workplace practices misalign in practice.
Algorithmic management (AM)'s impact on worker well-being has led to calls for regulation. However, little is known about the effectiveness and challenges in real-world AM regulation across the regulatory process -- rule operationalization, software use, and enforcement. Our multi-stakeholder study addresses this gap within workplace scheduling, one of the few AM domains with implemented regulations. We interviewed 38 stakeholders across the regulatory process: regulators, defense attorneys, worker advocates, managers, and workers. Our findings suggest that the efficacy of AM regulation is inf
policy
research · Management Science ·
Leveraging Expert Consistency to Improve Algorithmic Decision Support
Machine learning models for decision support perform better when trained on expert decisions for consistent cases and outcomes for inconsistent ones.
Machine learning (ML) is increasingly being used to support high-stakes decisions. However, there is frequently a construct gap: a gap between the construct of interest to the decision-making task and what is captured in proxies used as labels to train ML models. As a result, ML models may fail to capture important dimensions of decision criteria, hampering their utility for decision support. Thus, an essential step in the design of ML systems for decision support is selecting a target label among available proxies. In this work, we explore the use of historical expert decisions as a rich—yet
judgment
practice · Eugene Yan ·
Building News Agents for Daily News Recaps with MCP, Q, and tmux
Building a news recap agent using MCP and Amazon Q CLI to automate daily information gathering.
Learning to automate simple agentic workflows with Amazon Q CLI, Anthropic MCP, and tmux.
ways of working · Eugene Yan
research · Proceedings of the ACM on Human-Computer Interaction ·
Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information
Contextual information about AI capabilities and task domain significantly improves human-AI team performance in delegation decisions.
The integration of artificial intelligence (AI) into human decision-making processes at the workplace presents both opportunities and challenges. One promising approach to leverage existing complementary capabilities is allowing humans to delegate individual instances of decision tasks to AI. However, enabling humans to delegate instances effectively requires them to assess several factors. One key factor is the analysis of both their own capabilities and those of the AI in the context of the given task. In this work, we conduct a behavioral study to explore the effects of providing contextual
judgment
research · JAMA Network Open ·
Evaluation of an Ambient Artificial Intelligence Documentation Platform for Clinicians
Ambient AI documentation reduced clinician time on notes per appointment and off-hours EHR work, with mixed effects on burnout.
Importance: The increase of electronic health record (EHR) work negatively impacts clinician well-being. One potential solution is incorporating an ambient artificial intelligence (AI) documentation platform. Objective: To understand clinician experience before and after implementing ambient AI. Design, Setting, and Participants: This quality improvement study was a pilot evaluation with before and after survey and EHR metrics conducted at a large health care organization in Northern and Central California. Clinicians were purposively sampled to be representative of region and specialty. Ambie
productivity
research · Proceedings of the ACM on Human-Computer Interaction ·
Secret Use of Large Language Model (LLM)
Survey and experiment show workers hide LLM use from employers and colleagues; task type drives secretive behavior through fear of judgment.
The advancements of Large Language Models (LLMs) have decentralized the responsibility for the transparency of AI usage. Specifically, LLM users are now encouraged or required to disclose the use of LLM-generated content for varied types of real-world tasks. However, an emerging phenomenon, users' secret use of LLM , raises challenges in ensuring end users adhere to the transparency requirement. Our study used mixed-methods with an exploratory survey (125 real-world secret use cases reported) and a controlled experiment among 300 users to investigate the contexts and causes behind the secret u
worker experience
research · Proceedings of the ACM on Human-Computer Interaction ·
Rideshare Transparency: Translating Gig Worker Insights on AI Platform Design to Policy
Analysis of over 1 million gig worker comments reveals transparency gaps in rideshare algorithms affecting driver earnings, route decisions, and task allocation.
Rideshare platforms exert significant control over workers through algorithmic systems that can result in financial, emotional, and physical harm. What steps can platforms, designers, and practitioners take to mitigate these negative impacts and meet worker needs? In this paper, we identify transparency-related harms, mitigation strategies, and worker needs while validating and contextualizing our findings within the broader worker community. We use a novel mixed-methods study combining an LLM-based analysis of over 1 million comments posted to online platform worker communities with semi-stru
worker experience
research · Proceedings of the ACM on Human-Computer Interaction ·
AURA: Amplifying Understanding, Resilience, and Awareness for Responsible AI Content Work
Survey and interviews with content moderation and data labeling workers reveal challenges and needs in responsible AI work, proposing a support framework.
Behind the scenes of maintaining the safety of technology products from harmful and illegal digital content lies unrecognized human labor. The recent rise in the use of generative AI technologies and the accelerating demands to meet responsible AI (RAI) aims necessitates an increased focus on the labor behind such efforts in the age of AI. This study investigates the nature and challenges of content work that supports RAI efforts, or "RAI content work," that spans content moderation, data labeling, and red teaming -- through the lived experiences of content workers. We conduct a formative surv
worker experience · Jina Suh · Mary Gray
research · Proceedings of the ACM on Human-Computer Interaction ·
Making ChatGPT Work for Me
Teachers use ChatGPT in four distinct modes: make, find, jump-start, and iterate, with make-for-me dominating at 55 percent of prompts.
Increasingly, work happens through human collaboration with generative AI (e.g., ChatGPT). In this paper, we present a qualitative study of this collaboration for real-life work tasks. We focus our study on US K12 public school teachers (N = 24) who regularly design and complete text-generation tasks such as creating quizzes, slide decks, word problems, reading passages, lesson plans, classroom activities, and projects. In one-on-one video- and audio-recorded virtual sessions, we observe each teacher using ChatGPT-4 for work tasks of their choosing for 15 minutes, then debrief their experience
adoption
research · Proceedings of the ACM on Human-Computer Interaction ·
'Always Nice and Confident, Sometimes Wrong': Developer's Experiences Engaging Generative AI Chatbots Versus Human-Powered Q&A Platforms
Developers prefer ChatGPT's tone and speed over Stack Overflow but distrust its confident errors; lack voting validation reduces reliability signals.
Software engineers have historically relied on human-powered Q&A platforms like Stack Overflow (SO) as coding aids. With the rise of generative AI, developers have started to adopt AI chatbots, such as ChatGPT, in their software development process. Recognizing the potential parallels between human-powered Q&A platforms and AI-powered question-based chatbots, we investigate and compare how developers integrate this assistance into their real-world coding experiences by conducting a thematic analysis of 1700+ Reddit posts. Through a comparative study of SO and ChatGPT, we identified each platfo
adoption
research · Proceedings of the ACM on Human-Computer Interaction ·
Navigating Automated Hiring: Perceptions, Strategy Use, and Outcomes Among Young Job Seekers
Young job seekers distrust automated hiring tools and rely on referrals over practice strategies, while referrals and family income predict success more than AEDT gaming.
As the use of automated employment decision tools (AEDTs) has rapidly increased in hiring contexts, especially for computing jobs, there is still limited work on applicants' perceptions of these emerging tools and their experiences navigating them. To investigate, we conducted a survey with 448 computer science students (young, current technology job-seekers) about perceptions of the procedural fairness of AEDTs, their willingness to be evaluated by different AEDTs, the strategies they use relating to automation in the hiring process, and their job seeking success. We find that young job seeke
jobs skills
research · Proceedings of the ACM on Human-Computer Interaction ·
Chatbots in Collaborative Settings and their Impact on Virtual Teamwork
Group chat AI assistance improved team performance and reduced information-request response times; private chat increased perceived effort.
Chatbots have emerged as a powerful tool for collaborative teamwork. In this paper, we investigate the influence of chatbots on virtual teamwork within the context of a collaborative online activity. To assess chatbots' impact on group dynamics and performance, we designed a novel collaborative activity with an associated online platform and a custom chatbot assistant. We recruited 72 participants divided into four-person teams, with the teams split into four conditions depending on the chatbot's assistance strategy (none, by private chat, by group chat, or both). We found that chatbot assista
teams
research · Proceedings of the ACM on Human-Computer Interaction ·
EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders
A framework and interactive system for non-technical stakeholders to collectively decide on AI fairness metrics through explanation, preference elicitation, review, and negotiation.
Numerous fairness metrics have been proposed and employed by artificial intelligence (AI) experts to quantitatively measure bias and define fairness in AI models. Recognizing the need to accommodate stakeholders' diverse fairness understandings, efforts are underway to solicit their input. However, conveying AI fairness metrics to stakeholders without AI expertise, capturing their personal preferences, and seeking a collective consensus remain challenging and underexplored. To bridge this gap, we propose a new framework, EARN ( Explain, Ask, Review, and Negotiate ) Fairness, which facilitates
judgment
research · NBER ·
Shifting Work Patterns with Generative AI
Field experiment across 66 firms and 7,137 workers shows how generative AI integrated into daily applications changes work patterns over six months.
We present evidence from a field experiment across 66 firms and 7,137 knowledge workers. Workers were randomly selected to access a generative AI tool integrated into applications they already used at work for email, meetings, and writing. In the second half of the 6-month experiment, the 80% of (Eleanor W. Dillon , Sonia Jaffe , Nicole Immorlica , Christopher T. Stanton)
adoption · Sonia Jaffe
research · NBER ·
Still Waters, Rapid Currents: Early Labor Market Transformation under Generative AI
Survey data linked to Danish administrative records show rapid chatbot adoption by employers, worker-reported productivity gains, and early labour market shifts across occupations.
We study the early labor market impacts of AI chatbots by linking large-scale adoption surveys to administrative labor market records in Denmark. We document rapid currents: most employers in exposed occupations have adopted chatbot initiatives, workers report productivity benefits, and new AI (Anders Humlum , Emilie Vestergaard)
adoption · Anders Humlum
practice · Latent Space ·
Please stop forcing Clippy on those who want Anton
AI assistants should adapt helpfulness levels to user skill: beginners need Clippy-like guidance, experts need minimal intervention.
ChatGPT-4o's glazing embarrassment lays open Clippy vs Anton: The two extremes of desires in AI post-training and product ChatGPT-4o's glazing embarrassment lays open Clippy vs Anton: The two extremes of desires in AI post-training and product Update: Steven Sinofsky agrees: 1) Schticks get tired, 2) Helpfulness gets less helpful as skill level improves. Society advances by the number of operations you can do without thinking, not by the n…
judgment · Shawn Wang (swyx)
research · npj Digital Medicine ·
Opportunities and risks of artificial intelligence in patient portal messaging in primary care
Doctors failed to catch 35-45% of errors in AI-drafted patient messages, despite perceiving AI as safe and workload-reducing.
The rapid increase in patient portal messaging has heightened the workload for primary care physicians (PCPs), contributing to burnout. The use of generative artificial intelligence (AI) to draft responses to patient messages has shown promise in reducing cognitive burden, yet there is still much unknown about the safety and perceptions of using AI drafts. This cross-sectional simulation study assessed whether PCPs could identify and correct errors in AI-generated draft responses to patient portal messages. Twenty practicing PCPs reviewed 18 patient portal messages, four of which contained err
judgment
research · Government Information Quarterly ·
AI adoption in public administration: Perspectives of public sector managers and public sector non-managerial employees
Public sector managers and employees report different perspectives on AI adoption barriers and enablers in government agencies.
adoption
practice · Paul Ford (Aboard) ·
Using AI to Redirect Yourself—and Save Money!
Using AI chatbots as a redirection tool to interrupt impulse shopping by reframing desire into creative constraint.
Right now, Shopify and ChatGPT are starting to work together so that you can buy stuff from right inside of an AI chat. Is this a good idea? Stop asking and get your wallet out! But I want to throw out a way that I’ve been using ChatGPT not to buy things. Do you know about redirection? Redirection is an important concept in dealing with toddlers and middle-aged adults—there’s a valuable video to watch from Head Start (while we still have Head Start) if you’re curious. The whole idea is pretty simple: Intervene before the bad behavior starts. It’s not just saying, “Don’t do that.” It’s more lik
ways of working · Paul Ford
research · Journal of Empirical Legal Studies ·
Hallucination‐Free? Assessing the Reliability of Leading AI Legal Research Tools
Legal AI research tools by LexisNexis and Thomson Reuters hallucinate 17-33% of the time despite vendor claims of elimination.
ABSTRACT Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. However, the large language models used in these tools are prone to “hallucinate,” or make up false information, making their use risky in high‐stakes domains. Recently, certain legal research providers have touted methods such as retrieval‐augmented generation (RAG) as “eliminating” or “avoid[ing]” hallucinations, or guaranteeing “hallucination‐free” leg
judgment
research · Organizational Behavior and Human Decision Processes ·
The transparency dilemma: How AI disclosure erodes trust
Thirteen experiments show people trust colleagues less when they disclose AI use, even with mandatory disclosure or when AI accuracy is high.
As generative artificial intelligence (AI) has found its way into various work tasks, questions about whether its usage should be disclosed and the consequences of such disclosure have taken center stage in public and academic discourse on digital transparency. This article addresses this debate by asking: Does disclosing the usage of AI compromise trust in the user? We examine the impact of AI disclosure on trust across diverse tasks—from communications via analytics to artistry—and across individual actors such as supervisors, subordinates, professors, analysts, and creatives, as well as acr
judgment
practice · How I AI ·
Gumroad CEO's playbook to 40x his team's productivity with v0, Cursor, and Devin | Sahil Lavingia
Gumroad achieves 40x faster feature delivery by routing work through v0, Cursor, and Devin agents, shifting engineers to architecture roles.
Sahil Lavingia is the founder and CEO of Gumroad, where AI agents are already writing 41% of all code commits, and he’s targeting 80% by year’s end. Sahil demonstrates how this approach allows him to transform what would typically be two-week projects into two-hour implementations—a 40x productivity increase. What you’ll learn: The exact AI workflow Sahil uses to build features 40x faster—from prototyping in v0 to implementation with Devin How Gumroad incentivizes AI adoption across the organization with $33,000 bounties for engineers who outperform the CEO How to use component libraries like
productivity · Claire Vo
research · Industrial and Labor Relations Review ·
Lucy and the Chocolate Factory: Warehouse Robotics and Worker Safety
Warehouse robots cut severe injuries 40% but increase non-severe injuries 77%, partly due to accelerated work pace.
The authors examine the implications of robotics for warehouse worker safety. While warehouse automation has the potential to reduce injuries by eliminating high-risk tasks, it may also increase injuries among remaining non-automated tasks because of reduced task variety and an accelerated pace of work. Findings provide evidence of both effects: Warehouse robotics are associated with a 40% decrease in severe injuries but a 77% increase in non-severe injuries. The authors provide subsequent evidence that the rise in non-severe injuries is at least partially attributable to the increased pace of
worker experience
practice · Latent Space ·
AI Agents, meet Test Driven Development
Five-stage framework for applying test-driven development practices to AI agent development.
Guest Post: 5 stages for embracing test-driven development (TDD) to build stronger, more reliable AI systems! One of our top talks from AIE Online Track Guest Post: 5 stages for embracing test-driven development (TDD) to build stronger, more reliable AI systems! One of our top talks from AIE Online Track swyx here! We’re delighted to bring you another guest poster, this time talking about something we’ve been struggling to find someone good to articulate: TDD for AI. This is part of a broader discuss…
ways of working
practice · Eugene Yan ·
An LLM-as-Judge Won't Save The Product—Fixing Your Process Will
Using scientific method and eval-driven development to fix quality problems instead of relying on LLM judges alone.
Applying the scientific method, building via eval-driven development, and monitoring AI output.
judgment · Eugene Yan
research · Management Science ·
Algorithmic Writing Assistance on Jobseekers’ Resumes Increases Hires
Field experiment: jobseekers using algorithmic writing assistance on resumes were hired 8% more often at 10% higher wages, with no employer satisfaction loss.
There is a strong association between writing quality in resumes for new labor market entrants and whether they are ultimately hired. We show this relationship is, at least partially, causal: In a field experiment in an online labor market with nearly half a million jobseekers, treated jobseekers received nongenerative algorithmic writing assistance on their resumes. Treated jobseekers were hired 8% more often at 10% higher wages. Contrary to concerns that the assistance takes away a valuable signal, we find no evidence that employers were less satisfied. We find that the writing on treated jo
jobs skills
practice · Harper Reed ·
An LLM Codegen Hero's Journey
A staged adoption path for codegen: Copilot to Claude web to Cursor to agents, with advice on when each fits.
I have spent a lot of time since my blog post about my LLM workflow talking to folks about codegen and how to get started, get better, and why it is interesting. There has been an incredible amount of energy and interest in this topic. I have received a ton of emails from people who are working to figure all of this out. I started to notice that many people are struggling to figure out how to start, and how it all fits together. Then I realized that I have been hacking on this process since 2023 and I have seen some shit. Lol. I was talking about this with friends (Fisaconites’s represent) and
adoption · Harper Reed
practice · Hacker News ·
12-factor Agents: Patterns of reliable LLM applications
Production AI agents succeed by embedding LLMs in well-engineered software, not by using agent frameworks alone.
I've been building AI agents for a while. After trying every framework out there and talking to many founders building with AI, I've noticed something interesting: most "AI Agents" that make it to production aren't actually that agentic. The best ones are mostly just well-engineered software with LLMs sprinkled in at key points. So I set out to document what I've learned about building production-grade AI systems: https://github.com/humanlayer/12-factor-agents . It's a set of principles for building LLM-powered software that's reliable enough to put in the hands of production customers. In the
ways of working
research · Microsoft Research ·
Engagement, user expertise, and satisfaction: Key insights from the Semantic Telemetry Project
AI users who tackle complex professional tasks show higher sustained engagement and usage frequency than those doing simpler work.
Semantic Telemetry Project data show that people who use AI for more professional and complex tasks are more likely to keep using the tool and to use it more often. Novice AI users engage in simpler tasks, but their usage is becoming more complex. The post Engagement, user expertise, and satisfaction: Key insights from the Semantic Telemetry Project appeared first on Microsoft Research .
adoption · Siddharth Suri · Scott Counts
practice · Geoffrey Litt ·
Stevens: a hackable AI assistant using a single SQLite table and a handful of cron jobs
Personal AI assistant built with SQLite table and cron jobs, sending daily family briefs via Telegram with calendar, weather, mail and reminders.
There’s a lot of hype these days around patterns for building with AI. Agents, memory, RAG, assistants—so many buzzwords! But the reality is, you don’t need fancy techniques or libraries to build useful personal tools with LLMs. In this short post, I’ll show you how I built a useful AI assistant for my family using a dead simple architecture: a single SQLite table of memories, and a handful of cron jobs for ingesting memories and sending updates, all hosted on Val.town . The whole thing is so simple that you can easily copy and extend it yourself. Meet Stevens The assistant is called Stevens,
ways of working · Geoffrey Litt
research · Information Systems Research ·
Can Providing Algorithmic Performance Information Facilitate Humans’ Inventory Ordering Behaviors?
Field experiments show that disclosing algorithmic performance information, especially negative results, improves managers' inventory ordering decisions by increasing deliberation.
Over recent years, companies have been increasingly adopting algorithmic decision systems (ADS) to replace humans. In this paper, we focus on how ADS facilitates human managers’ decision making rather than replacing humans altogether in the context of inventory ordering decisions, specifically on the effect of providing ADS performance information to human managers. Using a pair of field experiments, our results suggest that providing ADS performance information can improve their inventory ordering decisions. Interestingly, providing both positive and negative ADS performance information enhan
judgment
practice · Kent Beck ·
Social AI Adoption: Lessons from Hybrid Corn
Kent Beck uses hybrid corn adoption patterns to explain how AI tools spread through organizations via social influence and peer practices.
Reflexive AI usage is now a baseline expectation at Shopify Reflexive AI usage is now a baseline expectation at Shopify
adoption · Kent Beck
practice · Harper Reed ·
Waterfall in 15 Minutes or Your Money Back
AI code generation is shifting testing from human-led verification to automated coverage, changing how programmers think about craft.
I recently had a conversation with a friend that started out as a casual catch-up and spiraled into a deep exploration of AI-assisted coding and what it’s doing to our workflows, teams, and sense of “craft.” It spanned everything from rewriting old codebases to how automated test coverage changes the nature of programming. I took the transcript from granola, popped it into o1-pro, and asked it to write this blog post. Not terrible. Representative of my beliefs. I sent it to a few friends, and they all were interested in sending it to a few more friends. That means I gotta publish it. So here g
ways of working · Harper Reed
research · Public Administration Review ·
How Public Officials Perceive Algorithmic Discretion: A Study of Status Quo Bias in Policing
UK police officers resist algorithmic discretion mainly due to status quo bias: transition costs, loss aversion, and performance uncertainty.
ABSTRACT Algorithms are disrupting established decision‐making practices in public administration. A key area of interest lies in algorithmic discretion or how public officials use algorithms to exercise discretion. The article develops a framework to explain algorithmic discretion by drawing on status quo bias theory and bureaucratic discretion. A study with police officers in the UK shows that—while officers still value their discretion—it is resistance via the aspects of status quo bias that accounts for a more substantial explanation. Transition costs, loss aversion, and performance uncert
judgment
practice · Paul Ford (Aboard) ·
The Spiders Versus the Web
Web infrastructure companies are blocking AI training spiders as a new form of infrastructure conflict.
There is a very strange war brewing between AI companies and classic web companies, and it’s being brought into sharp relief in the form of a new product launched by Cloudflare. Cloudflare is a huge web-hosting company (they host about a fifth of all sites ), and they’re very assertive about bots, denial of service attacks, and the like—they have to be, or they won’t be able to deliver the web pages. One of their most recent attack vectors has been AI company web spiders. To explain: In order to build an LLM, you need to gobble up as much content as possible—ideally all the content in the worl
management org · Paul Ford
research · HBS AI Institute ·
The Cybernetic Teammate: How AI is Reshaping Collaboration and Expertise in the Workplace
Field experiment at P&G tests how generative AI affects teamwork and expertise in organisational settings.
In an era where AI is rapidly transforming business operations, a recent working paper, “The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise,” describes research that explored how generative AI is reshaping teamwork and expertise in organizational settings. This research, conducted through a large-scale field experiment at Procter & Gamble (P&G), […] The post The Cybernetic Teammate: How AI is Reshaping Collaboration and Expertise in the Workplace appeared first on Harvard Business School AI Institute .
teams
practice · Paul Ford (Aboard) ·
The More Things Change
AI tools are maturing from hype to predictable, standardized products with clear use cases and limits.
On the podcast this week , I spoke with my co-founder, Richard, about how AI is a little…boring lately. As the technology stabilizes and matures, usage patterns are starting to appear. You can trust the chat in some things—summarizing documents, or generating Ghibli-esque images without ethical constraints , or helping you along with code. You need to validate findings in others, like when it makes up citations for research reports. But I don’t know how many revolutionary product releases await us. Probably one or two! There’s billions of dollars of gas in the tank, but it feels like we’re sta
management org · Paul Ford
research · NBER ·
Measuring Human Leadership Skills with AI Agents
Leaders' problem-solving performance with AI agents strongly correlates (r=0.81) with their performance leading human groups.
We show that leadership skill with artificially intelligent (AI) agents predicts leadership skill with human groups. In a large pre-registered lab experiment, human leaders worked with AI agents to solve problems. Their performance on this AI leadership test was strongly correlated (=0.81) with (Ben Weidmann , Yixian Xu , David J. Deming)
management org
research · NBER ·
The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise
Field experiment with 776 P&G professionals shows how generative AI affects team performance, expertise sharing, and social engagement in real product innovation work.
We examine how artificial intelligence transforms the core pillars of collaborationperformance, expertise sharing, and social engagementthrough a pre-registered field experiment with 776 professionals at Procter & Gamble, a global consumer packaged goods company. Working on real product innovation (Fabrizio Dell'Acqua , Charles Ayoubi , Hila Lifshitz , Raffaella Sadun , Ethan Mollick , Lilach Mollick , Yi Han , Jeff Goldman , Hari Nair , Stewart Taub , Karim Lakhani)
teams · Hila Lifshitz-Assaf · Karim Lakhani · Ethan Mollick · Fabrizio Dell'Acqua · Raffaella Sadun · Charles Ayoubi · Lilach Mollick
research · Management Science ·
Mitigating the Negative Effects of Customer Anxiety by Facilitating Access to Human Contact
Field experiment shows offering human contact during self-service loan approval increases customer uptake by 24% by reducing anxiety-driven dissatisfaction.
Prior research in social psychology has shown that when people feel anxious, they seek advice from others. Yet, companies that operate in high-anxiety settings (like financial services, healthcare, and education) are increasingly deploying self-service technologies (SSTs), through which anxious customers transact without access to human contact. These companies may therefore face a classic efficiency–service trade-off where gains in operational efficiency through automation also hamper service outcomes by neglecting customer anxiety. This paper, set in the high-anxiety domain of financial serv
worker experience
practice · Paul Ford (Aboard) ·
Bad Vibes Coding
Developers who skip reading AI-generated code and trust the output are adopting a new workflow pattern despite its risks.
When I think something is a truly terrible idea, it often succeeds beyond my wildest imagination. One example is Twitter. When it launched, I thought it was shallow nonsense for dimwits, but it turned out to be a world-shaping, influential platform, even if it’s now an enormous cultural superfund site. I thought the same thing about Facebook, and Instagram, and TikTok. And pretty much anything blockchain. In general, if I really hate something, you should invest in it. I’ve also loathed a lot of technologies that became popular, like HTML5, or JavaScript, or React. They all became huge winners
ways of working · Paul Ford
research · Information Systems Research ·
Artificial Intelligence and Firm Resilience: Empirical Evidence from Natural Disaster Shocks
Firms with 2.4% AI-related jobs recover faster from natural disasters; benefits hampered by lack of complementary org design.
Artificial intelligence (AI) has been increasingly deployed in business operations over the past decade, whereas direct evidence of its effectiveness in uncertain contexts is limited. Our work examines the contribution of AI to corporate resilience under natural disaster shocks, particularly concentrating on AI-using and goods-producing firms. We measure firm AI investment by the cumulative AI-relevant skills extracted from a comprehensive job posting database and firm resilience by the changes in corporate valuation in response to operational shocks. Evidence suggests that AI generates resili
productivity
practice · Latent Space ·
Agent Engineering
Framework for what makes an agent and why agent engineering is the biggest opportunity for AI engineers now.
Defining Agents, Why now, and why Agents are the biggest opportunity for AIEs Defining Agents, Why now, and why Agents are the biggest opportunity for AIEs This post contains elaborations on swyx’s 2025 AI Engineer Summit keynote, which also serves as a cohesive overview of a selection of Agents talks from the conference which link-clickers can preview.
ways of working · Shawn Wang (swyx)
practice · Hamel Husain ·
A Field Guide to Rapidly Improving AI Products
AI teams improve products faster by measuring outcomes and iterating with domain experts, not by choosing tools.
Most AI teams focus on the wrong things. Here’s a common scene from my consulting work: AI TEAM Here’s our agent architecture – we’ve got RAG here, a router there, and we’re using this new framework for… ME [Holding up my hand to pause the enthusiastic tech lead.] “Can you show me how you’re measuring if any of this actually works?” … Room goes quiet This scene has played out dozens of times over the last two years. Teams invest weeks building complex AI systems, but can’t tell me if their changes are helping or hurting. This isn’t surprising. With new tools and frameworks emerging weekly, it’
management org · Hamel Husain
research · Management Science ·
Human-Algorithm Collaboration with Private Information: Naïve Advice-Weighting Behavior and Mitigation
Humans systematically overweight algorithm predictions, ignoring private information; feature transparency and targeted interventions reduce prediction error by 25-34%.
Even if algorithms make better predictions than humans on average, humans may sometimes have private information that an algorithm does not have access to that can improve performance. How can we help humans effectively use and adjust recommendations made by algorithms in such situations? When deciding whether and how to override an algorithm’s recommendations, we hypothesize that people are biased toward following naïve advice-weighting (NAW) behavior; they take a weighted average between their own prediction and the algorithm’s prediction, with a constant weight across prediction instances r
judgment
research · JAMA Network Open ·
Physician Perspectives on Ambient AI Scribes
Physicians using ambient AI scribes report ease of use and positive tool quality, with detailed barriers and facilitators to adoption identified.
Importance: Limited qualitative studies exist evaluating ambient artificial intelligence (AI) scribe tools. Such studies can provide deeper insights into ambient AI implementations by capturing lived experiences. Objective: To evaluate physician perspectives on ambient AI scribes. Design, Setting, and Participants: A qualitative study using semistructured interviews guided by the Reach, Efficacy, Adoption, Implementation, Maintenance/Practical, Robust Implementation, and Sustainability Model (RE-AIM/PRISM) framework, with thematic analysis using both inductive and deductive approaches. Physici
adoption
research · arXiv ·
Collaborating with AI Agents: Field Experiments on Teamwork, Productivity, and Performance
Human-AI teams produced 50% more ads per worker with higher text quality; human teams produced better images, revealing uneven AI capability.
We examined the mechanisms underlying productivity and performance gains from AI agents using a large-scale experiment on Pairit, a platform we developed to study human-AI collaboration. We randomly assigned 2,234 participants to human-human and human-AI teams that produced 11,024 ads for a think tank. We evaluated the ads using independent human ratings and a field experiment on X which garnered ~5M impressions. We found human-AI teams produced 50% more ads per worker and higher text quality, while human-human teams produced higher image quality, suggesting a jagged frontier of AI agent capab
teams
research · ACM Transactions on Software Engineering and Methodology ·
Investigating the Role of Cultural Values in Adopting Large Language Models for Software Engineering
Among 188 software engineers, habit and performance expectancy drive LLM adoption; cultural values do not significantly moderate adoption patterns.
As a socio-technical activity, software development involves the close interconnection of people and technology. The integration of Large Language Models (LLMs) into this process exemplifies the socio-technical nature of software development. Although LLMs influence the development process, software development remains fundamentally human-centric, necessitating an investigation of the human factors in this adoption. Thus, with this study we explore the factors influencing the adoption of LLMs in software development, focusing on the role of professionals’ cultural values. Guided by the Unified
adoption
practice · Paul Ford (Aboard) ·
The Zeno Effect
AI does not solve the fundamental tension between scope creep and shipping deadlines in software projects.
There’s a joke in software development: Now that you’ve got the first 80% done, you need to do the remaining 80%. Of course, this applies to most things—writing and editing, creating art, PhD dissertations—but in code the fact that things don’t ship gets a lot of attention because it’s so expensive . If a PhD doesn’t land on time, that’s suffering for one person and their family. When code doesn’t land, that’s a whole team, tons of money, and a big corporate plan going “whoosh.” So right as you see things landing, right as marketing gets the press release in order, you realize you’re going to
management org · Paul Ford
practice · Artificial Ignorance ·
Breaking Into AI Engineering
Writing publicly about learning AI tools helped transition an engineer into an AI engineering role.
How building in public transformed my career path. From writing about ChatGPT to becoming a professional AI engineer. Two years ago, I knew almost nothing about generative AI. Like millions (now billions) of others, I was stunned when ChatGPT first launched - but I did have both an engineering background and the stu…
jobs skills · Charlie Guo
practice · Laurie Voss ·
AI's effects on programming jobs
AI will expand the programming workforce by creating new layers of abstraction, not eliminate programmer jobs through higher demand for software.
There's been a whole lot of discussion recently about the impact of AI on the market for web developers, for programmers in general, and even more generally the entire labor market. I find myself making the same points over and over, and whenever I do that it's time to write a blog post about it, so this is that. Doom and utopia are not our only options There are two extreme takes on the impact of AI on programmer jobs: AI will take all programming jobs (usually advanced by people selling AI) AI will not take anyone's job (usually advanced by grumpy older developers, which I usually am) I woul
jobs skills · Laurie Voss
research · arXiv ·
How Generative AI Adoption Alters the Demand for Cognitive and Social Skills Within Roles: A Skill-Centric Analysis
GenAI adoption reduced demand for social skills by 3.4% in adopting roles while cognitive skill demand stayed steady, contradicting predictions of skill reallocation.
A common view holds that generative AI (GenAI) automates cognitive tasks, reshaping roles to emphasize social skills over cognitive ones. Drawing on the framework we develop in this paper, we argue that other outcomes are theoretically possible. We analyze seven million job postings from 595 U.S. public firms that adopted GenAI in 2022-2024, estimating difference-in-differences models comparing GenAI-adopting and non-adopting roles around ChatGPT's launch. We find no evidence of greater emphasis on social skills in GenAI-adopting roles; instead, their relative demand for social skills declined
jobs skills · Phanish Puranam
research · Léonard Boussioux ·
The Dangers of Deferring to AI: It Seems So Right Even When It's Wrong
Field experiment shows AI-generated explanations cause evaluators to reject good ideas more often than human explanations.
Harvard Business School · Working Knowledge — On the field experiment finding that AI-generated narrative explanations made evaluators reject good ideas more often: “you really need humans synthesizing and validating.”
judgment · Léonard Boussioux
research · Management Science ·
Designing AI-Based Work Processes: How the Timing of AI Advice Affects Diagnostic Decision Making
Diagnostic accuracy improves when AI advice is given after initial diagnosis, due to deeper cognitive engagement with AI reasoning.
Although clinical artificial intelligence (AI) systems can augment medical diagnosis decisions by providing competent second opinions, how to effectively integrate AI into routine diagnostic processes, such as when to present AI advice to human physicians, remains largely unexplored. Therefore, our research experimentally examines how the timing of AI advice affects diagnostic decision making using a think-aloud approach. Physicians perform medical diagnoses under three conditions: ex post advice (AI advice given after an initial diagnosis), ex ante advice (AI advice given concurrently with cl
judgment
practice · Paul Ford (Aboard) ·
The $15 Volvo
Enterprise buyers are skeptical of AI tools promised as cheap replacements for legacy systems, seeing cosmetic solutions rather than robust infrastructure.
In the conversation about AI coding, lots of organizations are promising to completely change the way software is developed. You will tell the computer what you need, AI will build it, and it will be good. Companies like Bolt and v0 offer impressive abilities to spin up websites or other software tools just using words. It’s still early days, but eventually, they say that if you can describe it, you’ll have it. This creates a problem—around the office, we call it “the $15 Volvo problem.” Building a website that looks nice and sits around on the web is a relatively low-risk proposition. That’s
management org · Paul Ford
research · Management Science ·
Statistical Tests for Replacing Human Decision Makers with Algorithms
Algorithm outperformed individual doctors on abnormal birth detection, with higher true positive and lower false positive rates on real diagnostic data.
This paper proposes a statistical framework of using artificial intelligence to improve human decision making. The performance of each human decision maker is benchmarked against that of machine predictions. We replace the diagnoses made by a subset of the decision makers with the recommendation from the machine learning algorithm. We apply both a heuristic frequentist approach and a Bayesian posterior loss function approach to abnormal birth detection using a nationwide data set of doctor diagnoses from prepregnancy checkups of reproductive-age couples and pregnancy outcomes. We find that our
judgment
practice · Artificial Ignorance ·
Hallucinations Are Fine, Actually
Rethinking when AI hallucinations matter in real workflows rather than dismissing them as a universal flaw.
Why I changed my mind about AI's imperfections. Why I changed my mind about AI's imperfections When I first started using ChatGPT, I quickly discovered its tendency to confidently make things up. Whether you want to call it lying, fabricating, or just bullshitting - it had no problems inventin…
judgment · Charlie Guo
practice · Geoffrey Litt ·
Avoid the nightmare bicycle
AI tools should expose underlying structure and user control rather than hiding complexity behind mode buttons.
In my opinion, one of the most important ideas in product design is to avoid the “nightmare bicycle”. Imagine a bicycle where the product manager said: “people don’t get math so we can’t have numbered gears. We need labeled buttons for gravel mode, downhill mode, …” This is the hypothetical “nightmare bicycle” that Andrea diSessa imagines in his book Changing Minds . As he points out: it would be terrible! We’d lose the intuitive understanding of how to use the gears to solve any situation we encounter. Which mode do you use for gravel + downhill? It turns out, anyone can understand numbered g
ways of working · Geoffrey Litt
research · NBER ·
AI and the Extended Workday: Productivity, Contracting Efficiency, and Distribution of Rents
Workers in AI-exposed occupations work longer hours; study uses time diary data from 2004-2023 to measure intensive-margin employment effects.
This study investigates how occupational AI exposure impacts employment at the intensive margin, i.e., the length of workdays and the allocation of time between work and leisure. Drawing on individual-level time diary data from 20042023, we find that higher AI exposurewhether stemming from the (Wei Jiang , Junyoung Park , Rachel (Jiqiu) Xiao , Shen Zhang)
worker experience
research · ACM Transactions on Software Engineering and Methodology ·
Accountability in Code Review: The Role of Intrinsic Drivers and the Impact of LLMs
Software engineers report four intrinsic accountability drivers; LLM code review shifts accountability from individual to collective, reducing personal responsibility.
Accountability is an innate part of social systems. It maintains stability and ensures positive pressure on individuals’ decision-making. As actors in a social system, software developers are accountable to their team and organization for their decisions. However, the drivers of accountability and how it changes behavior in software development are less understood. In this study, we look at how the social aspects of code review affect software engineers’ sense of accountability for code quality. Since Software Engineering (SE) is increasingly involving Large Language Models (LLM) assistance, w
worker experience
practice · Understanding AI (Timothy B. Lee) ·
My brother explains what it's like to run an AI startup
Email app uses six different AI models in production, each chosen for specific tasks within a real product.
The Shortwave email app uses six different models for a range of tasks. The Shortwave email app uses six different models for a range of tasks. I’m pleased to present this cross-posted episode of AI Summer, a podcast I co-host with Dean Ball of the Mercatus Center. Most episodes of AI Summer are not cross-posted here, so if you enjoy this conversation please subscribe to the newsletter or search for “AI Summer” in your favorite podcast app.
ways of working · Timothy B. Lee
research · arXiv ·
Predictive AI Can Support Human Learning while Preserving Error Diversity
AI support during training and practice improves diagnostic accuracy and preserves error diversity, affecting group decision-making quality.
We examined the effects of predictive AI deployment on the immediate performance and learning of medical novices. In two pre-registered field experiments, we varied whether AI input was provided during the training or practice of lung cancer diagnoses, or both. Our results show that different AI deployments have distinct implications for human professionals. AI input during training or practice independently improves individuals' diagnostic accuracy, whereas deployment across both phases yields gains that exceed either approach alone. Furthermore, AI input in both training and earlier practice
judgment · Phanish Puranam
research · Information and Organization ·
Novice risk work: How juniors coaching seniors on emerging technologies such as generative AI can lead to learning failures
Junior consultants given early access to GPT-4 recommended risky practices due to shallow understanding of AI's uncertain capabilities and scope.
Historically, junior professionals have mentored senior professionals around new technologies, because juniors are typically more willing than seniors to perform lower-level tasks to learn new skills, better able than seniors to engage in real-time experimentation close to the work itself, and more willing than seniors to learn innovative methods that conflict with traditional identities and norms. However, we know little about what happens when emerging technologies have a high level of uncertainty in their use, because they have wide-ranging capabilities and are exponentially changing. With
adoption · Hila Lifshitz-Assaf · Katherine Kellogg · Karim Lakhani · Ethan Mollick · Fabrizio Dell'Acqua · Edward McFowland III · Steven Randazzo · François Candelon
practice · Paul Ford (Aboard) ·
Hegel on the Death Star
Using AI to quickly bridge knowledge gaps by asking absurd questions reveals how LLMs map between conceptual domains.
This weekend I saw someone online talking about the philosopher Hegel, and I thought to myself, What’s all that about? I opened up Wikipedia, and I will say: Georg Wilhelm Friedrich Hegel has one of the most forbidding faces I’ve ever seen. The gravity of his countenance! But also his wiki page is enormous—countless sections and footnotes. I know Hegel is very important, but I had to make dinner and get the laundry done. All I wanted was a taste. A little Hegel dabble. So—you know where this is going—I used ChatGPT. What is the dumbest question I could ask, the most humiliating way of learning
ways of working · Paul Ford
practice · Understanding AI (Timothy B. Lee) ·
We're Finding Out What Humans are Bad At
AI improves fastest in domains where humans use unnatural techniques that align with how machines think.
AI Advances Fastest When We Find Unnatural Ways of Doing Things AI Advances Fastest When We Find Unnatural Ways of Doing Things Magnus Carlsen is widely considered to be the greatest chess player of all time. An obvious prodigy (his Wikipedia page mentions solving 50-piece jigsaw puzzles when he was 2 years old), he earned the rank of grandmaster at age 13, despite (or because of?) being initially self-taugh…
judgment · Timothy B. Lee
research · JAMA Network Open ·
Clinician Experiences With Ambient Scribe Technology to Assist With Documentation Burden and Efficiency
Ambient scribing reduced time in notes per appointment and after-hours work, with clinicians reporting lower documentation burden.
Importance: Timely evaluation of ambient scribing technology is warranted to assess whether this technology can lessen the burden of clinical documentation on clinicians. Objective: To investigate the association of ambient scribing technology with efficiency, quality, and perceived burden of clinical documentation in the outpatient setting. Design, Setting, and Participants: This prospective, single-group pre-post quality improvement study was conducted between April and June 2024 in the outpatient setting of an academic health system in Philadelphia, Pennsylvania. Participants included physi
productivity
practice · Latent Space ·
The Inventors of Deep Research
DeepMind's Deep Research agent moves from search results to fully cited reports, showing a working use case for AI agents in knowledge work.
DeepMind's Aarush Selvan and Mukund Sridhar on creating the killer agent usecase, going from 10 blue links to fully cited reports, and building ontologies of AI use cases DeepMind's Aarush Selvan and Mukund Sridhar on creating the killer agent usecase, going from 10 blue links to fully cited reports, and building ontologies of AI use cases The free livestreams for AI Engineer Summit are now up! Please hit the bell to help us appease the algo gods. We’re also announcing a special Online Track later today.
ways of working · Shawn Wang (swyx)
research · Management Science ·
Till Tech Do Us Part: Betrayal Aversion and Its Role in Algorithm Use
Workers show betrayal aversion to human experts but not algorithms, even when risk is identical, affecting advice-following and earnings.
Failing to follow expert advice can have real and dangerous consequences. While any number of factors may lead a decision maker to refuse expert advice, the proliferation of algorithmic experts has further complicated the issue. One potential mechanism that restricts the acceptance of expert advice is betrayal aversion, or the strong dislike for the violation of trust norms. This study explores whether the introduction of expert algorithms in place of human experts can attenuate betrayal aversion and lead to higher overall rates of seeking expert advice. In other words, we ask: are decision ma
judgment
practice · Harper Reed ·
My LLM codegen workflow atm
Developer workflow: brainstorm spec, plan-the-plan, then discrete LLM codegen loops for greenfield and legacy code.
tl:dr; Brainstorm spec, then plan a plan, then execute using LLM codegen. Discrete loops. Then magic. ✩₊˚.⋆☾⋆⁺₊✧ I have been building so many small products using LLMs. It has been fun, and useful. However, there are pitfalls that can waste so much time. A while back a friend asked me how I was using LLMs to write software. I thought “oh boy. how much time do you have!” and thus this post. (p.s. if you are an AI hater - scroll to the end) I talk to many dev friends about this, and we all have a similar approach with various tweaks in either direction. Here is my workflow. It is built upon my o
ways of working · Harper Reed
research · Computers in Human Behavior ·
Beyond efficiency: Trust, AI, and surprise in knowledge work environments
Real-time automated feedback increases trust in algorithmic performance ratings when task uncertainty is high, in a controlled experiment.
Contemporary management practices are often designed with the needs of knowledge-based workers in mind, but an increasingly pressing challenge today is how to manage and effectively handle non-routine work. This paper revisits the job characteristics model through the lens of self-determination theory, specifically in the context of algorithmic performance management. Non-routine work is inherently unpredictable, and individuals often struggle with prolonged uncertainty. However, automated interventions that help individuals make sense of their work in uncertain conditions may help overcome th
judgment · Anita Woolley
research · Strategic Management Journal ·
How does artificial intelligence improve human decision‐making? Evidence from the AI ‐powered Go program
Professional Go players improved move quality significantly after an AI program outperformed the best human player, with younger and less skilled players gaining most.
Abstract Research Summary We study how humans learn from artificial intelligence (AI), leveraging an introduction of an AI‐powered Go program (APG) that unexpectedly outperformed the best professional player. We compare the move quality of professional players to APG's superior solutions around its public release. Our analysis of 749,190 moves demonstrates significant improvements in players' move quality, especially in the early stages of the game where uncertainty is highest. This improvement was accompanied by a higher alignment with AI's suggestions and a decreased number and magnitude of
judgment
research · Research Policy ·
A processual approach to skill changes in digital automation: The case of the platform economy in the service sector
Processual analysis of ride-hailing shows automation interrupts workers' micro-adaptations and repositions skills rather than replacing them, excluding workers from learning transferable skills.
We introduce the “processual approach” to skill changes in the current wave of digital automation, which imposes comprehensive and complex impacts on skills. The approach conceptualizes work as a set of processes, each consisting of a sequence of events. In each event, a worker and/or machine make judgments and take actions to move to the next event. The processual approach asks whether and how machines influence workers' judgments or actions during each event and interrupt or transform relationships between judgments and actions. The approach enables micro-to-middle-range, inductive theorizat
jobs skills
practice · Artificial Ignorance ·
Two years of Artificial Ignorance
A writer's reflections on what they learned about AI and work during a year of sustained observation and writing.
A belated 2024 year in review. A belated 2024 year in review. This week marks the second anniversary of Artificial Ignorance. 2024 was my first full calendar year of biweekly writing, and it felt like I started to find my footing as a writer (creator? Substacke…
ways of working · Charlie Guo
practice · Hacker News ·
How I use LLMs as a staff engineer
Staff engineer shares specific ways they use LLMs daily for code review, design docs, and unblocking teammates.
248 points on Hacker News. Discussion: https://news.ycombinator.com/item?id=42938409
ways of working
research · Organization Science ·
Algorithmic Recommendation Tools and Experiential Learning in Clinical Care
Clinical decision support adoption weakens the relationship between doctor experience and patient outcomes, especially in routine tasks.
This study examines the relationship between the adoption of algorithmic recommendation tools and experiential learning. We argue that the adoption of an algorithmic recommendation tool will harm experiential learning in organizations by limiting knowledge retention and retrieval. We further argue that the adverse relationship between algorithmic tool adoption and experiential learning will be stronger in organizations operating in low-task-difficulty environments than those in high-task-difficulty ones because organizational members in such organizations are likely to rely more on algorithmic
adoption
research · NBER ·
AI and Women's Employment in Europe
Female employment share increased in occupations with more AI-enabled technology diffusion across 16 European countries, 2011-2019.
We examine the link between the diffusion of artificial intelligence (AI) enabled technologies and changes in the female employment share in 16 European countries over the period 2011-2019. Using data for occupations at the 3-digit level, we find that on average female employment shares increased in (Stefania Albanesi , António Dias da Silva , Juan F. Jimeno , Ana Lamo , Alena Wabitsch)
jobs skills
research · NBER ·
Artificial Intelligence and the Labor Market
Tasks with higher AI exposure experience reduced labour demand in subsequent periods, measured through NLP analysis of task content across firms and occupations 2010-2023.
We use advances in natural language processing to construct new measures of workers task-level exposure to artificial intelligence (AI) and machine learning from 2010 to 2023, capturing variation across firms, occupations, and time. Tasks with higher AI exposure subsequently experience reduced labor (Menaka Hampole , Dimitris Papanikolaou , Lawrence D.W. Schmidt , Bryan Seegmiller)
jobs skills
research · NBER ·
The Labor Market Impact of Digital Technologies
Regional variation in Korea's digital tech adoption shows significant labour market impacts from AI, big data, and IoT.
We investigate the impact of digital technology on employment patterns in Korea, where firms have rapidly adopted digital technologies such as artificial intelligence (AI), big data, and the internet of things (IoT). By exploiting regional variations in technology exposure, we find significant (Sangmin Aum , Yongseok Shin)
jobs skills
research · Management Science ·
Who Is AI Replacing? The Impact of Generative AI on Online Freelancing Platforms
ChatGPT reduced job posts for writing and coding freelancers by 21% within eight months; image AI reduced image jobs by 17%.
This paper studies the impact of generative artificial intelligence (AI) technologies on the demand for online freelancers using a large data set from a leading global freelancing platform. We identify the types of jobs that are more affected by generative AI and quantify the magnitude of the heterogeneous impact. Our findings indicate a 21% decrease in the number of job posts for automation-prone jobs related to writing and coding compared with jobs requiring manual-intensive skills within eight months after the introduction of ChatGPT. We show that the reduction in the number of job posts in
jobs skills
research · HBS AI Institute ·
The Creative Edge: How Human-AI Collaboration is Reshaping Problem-Solving
Human-AI collaboration changes how organizations solve creative problems, but the actual findings are not detailed here.
As artificial intelligence (AI) capabilities rapidly advance, organizations are exploring new ways to leverage these technologies for creative problem-solving and innovation. A recent HBS working paper, “The Crowdless Future? Generative AI and Creative Problem Solving”, – by Léonard Boussioux, Assistant Professor at the University of Washington; Jacqueline N. Lane, Assistant Professor at Harvard Business School […] The post The Creative Edge: How Human-AI Collaboration is Reshaping Problem-Solving appeared first on Harvard Business School AI Institute .
adoption · Léonard Boussioux
practice · Paul Ford (Aboard) ·
Use Worse Tools
Using a weaker AI model teaches more about how coding with AI actually works than optimizing for the best tool.
I’ve been on this big quest to learn how to write software with AI. Not so much because I need to write code; I’m a co-founder, which means I’m in sales. But I used to be a programmer, and I do want to fully understand how these new systems work, and their limitations. It’s surprisingly hard to get good guidance here. The space is evolving very quickly. You just have to try stuff. For a while I was using Claude and ChatGPT, which are the big players in the space. I used them through “ aider ,” an intermediary program that runs in the terminal on your computer. It sits between you and the chatb
ways of working · Paul Ford
research · Journal of Labor Economics ·
Profits of Prejudiced Algorithms
Firms profit more from discriminatory algorithms when training data reflects human prejudice, but affirmative action can reverse this incentive.
Firms are starting to replace humans with algorithms in important screening decisions, but there are potential spillovers of human biases contained in datasets to subsequent algorithmic predictions. When these biases are motivated by human prejudices, there are risks of algorithms perpetuating discrimination. I prove that when datasets are generated by a sufficiently discriminatory human, firms are more profitable when training discriminatory algorithms. If instead enough affirmative action is instituted in favor of a disadvantaged group, firms are more profitable when training algorithms that
jobs skills
practice · Laurie Voss ·
What I've learned about writing AI apps so far
LLMs excel at text summarisation and transformation; builders should design around this strength, not fight it.
I started writing a post called "how to write AI apps" but it was over-reach so I scaled it back to this. Who am I to tell you how to write anything? But here's what I'll be applying to my own writing of AI-powered apps, specifically LLM applications. A battle I've already lost is that we shouldn't call LLMs "AI" at all; they are machine learning and not the general intelligence that is implied to the layman by the name. It is an even less helpful name than "serverless", my previous candidate for worst technology name. But alas, we're calling LLMs AI and any parts of the field that are not LLM
ways of working · Laurie Voss
research · HBS AI Institute ·
The Future of Decision-Making: How Generative AI Transforms Innovation Evaluation
Field experiment tests how generative AI affects evaluation of business ideas and innovations.
As businesses grapple with an ever-growing volume of ideas, products, and solutions to evaluate, decision-making processes are being reshaped by artificial intelligence (AI). Generative AI, in particular, has emerged as a game-changer in creative problem-solving and evaluation, as demonstrated by a recent field experiment described in the working paper “The Narrative AI Advantage? A Field […] The post The Future of Decision-Making: How Generative AI Transforms Innovation Evaluation appeared first on Harvard Business School AI Institute .
judgment
practice · Harper Reed ·
New Media
Harper Reed automated a personal media dashboard pulling from Goodreads, Spotify and RSS feeds using AI-assisted development.
tl;dr: I added a media section to my website to track and display what I’m reading, listening to, and bookmarking online through automated data collection. visit it here Recently I spent a few hours hanging out with my friends Claude, Aider, and ChatGPT and added a media section to this site. I really enjoy AI-aided development (maybe a post for later). The media section has a log of all my books read (tracked from Goodreads), my recently saved tracks (tracked from Spotify), and my links (tracked from feedbin/netnewsreader). I started posting my links a month or so ago. I wanted to see how the
ways of working · Harper Reed
practice · Paul Ford (Aboard) ·
Software Theater
AI coding works best when you have it write tests alongside code, creating a feedback loop that catches its own errors.
Over the holiday I decided to see what it would take to code an actual application in AI. I wanted to make something pretty complex—a database-backed, component-driven web tool that calls external APIs and updates webpages in real time. I worked on it in the evenings and didn’t finish it, but for a good reason. Because I was worried it would become self-aware and take over the world! Just kidding—I am above the age of 15. I’ll explain the actual reason in a minute. Right now, a consensus about programming with AI is forming, which breaks down to (1) LLMs write pretty good code; (2) they write
ways of working · Paul Ford
practice · The Pragmatic Engineer ·
How AI-assisted coding will change software engineering: hard truths
How AI-assisted coding reshapes what software engineering means, with implications for expectations and skill requirements.
A field guide that also covers why we need to rethink our expectations, and what software engineering really is. A guest post by software engineer and engineering leader Addy Osmani A field guide that also covers why we need to rethink our expectations, and what software engineering really is. A guest post by software engineer and engineering leader Addy Osmani Hi, this is Gergely with a bonus issue of the Pragmatic Engineer Newsletter. In every issue, we cover topics related to Big Tech and startups through the lens of software engineers and engineering leaders. To get articles like this in y
jobs skills · Gergely Orosz · Addy Osmani
practice · Latent Space ·
AI Engineering for Art — with comfyanonymous, of ComfyUI
Image generation workflows are moving from text prompts to node-based directed acyclic graphs for more control.
Using models for "Art Engineering", building hard to use UIs, and how image generation is moving from text boxes to DAGs Using models for "Art Engineering", building hard to use UIs, and how image generation is moving from text boxes to DAGs Applications for the NYC AI Engineer Summit, focused on Agents at Work, are open!
ways of working · Shawn Wang (swyx)
research · Academy of Management Annals ·
Managing with Artificial Intelligence: An Integrative Framework
Integrative framework linking human-AI collaboration and algorithmic management literatures, showing how they examine complementary sides of managing with AI in organisations.
Managing with artificial intelligence (AI) refers to humans’ interaction with algorithms performing managerial tasks in organizations. Two literatures exploring this interaction—human-AI collaboration (HAIC) and algorithmic management (AM)—have focused on distinct managerial tasks: while HAIC examines executive decision-making, AM focuses on managerial control. This article presents a review of both literatures to identify opportunities for integration and advancement. We observe that HAIC’s and AM’s micro-level emphases on different managerial tasks have resulted in diverging conceptualizatio
management org · Sebastian Raisch
research · Journal of Applied Psychology ·
Whither bias goes, I will go: An integrative, systematic review of algorithmic bias mitigation.
Systematic review integrates computer science and organizational research on algorithmic bias mitigation in personnel assessment across model development stages.
Machine learning (ML) models are increasingly used for personnel assessment and selection (e.g., resume screeners, automatically scored interviews). However, concerns have been raised throughout society that ML assessments may be biased and perpetuate or exacerbate inequality. Although organizational researchers have begun investigating ML assessments from traditional psychometric and legal perspectives, there is a need to understand, clarify, and integrate fairness operationalizations and algorithmic bias mitigation methods from the computer science, data science, and organizational research
judgment
research · Proceedings of the National Academy of Sciences ·
The unequal adoption of ChatGPT exacerbates existing inequalities among workers
Women are 16 percentage points less likely to use ChatGPT at work; higher-earning workers adopt it first despite lower tenure.
We study the adoption of ChatGPT, the icon of Generative AI, using a large-scale survey linked to comprehensive register data in Denmark. Surveying 18,000 workers from 11 exposed occupations, we document that ChatGPT is widespread, especially among younger and less-experienced workers. However, substantial inequalities have emerged. Women are 16 percentage points less likely to have used the tool for work. Furthermore, despite its potential to lift workers with less expertise, users of ChatGPT earned slightly more already before its arrival, even given their lower tenure. Workers see a substan
adoption · Anders Humlum
research · The Quarterly Journal of Economics ·
Generative AI at Work
AI assistant in customer support increased productivity 15% on average, with larger gains for less experienced workers and evidence of improved learning and work experience.
Abstract We study the staggered introduction of a generative AI–based conversational assistant using data from 5,172 customer-support agents. Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers. The effects vary significantly across different agents. Less experienced and lower-skilled workers improve both the speed and quality of their output, while the most experienced and highest-skilled workers see small gains in speed and small declines in quality. We also find evidence that AI assi
productivity · Erik Brynjolfsson · Danielle Li · Lindsey Raymond
research · arXiv ·
Complement or substitute? How AI increases the demand for human skills
AI adoption increases demand for complementary skills like analytical thinking across both AI and non-AI roles, with wage premiums, while reducing demand for substitutable skills.
Artificial Intelligence (AI) is transforming the nature of work, yet there is limited empirical evidence on how it affects demand for human skills. This paper examines whether AI adoption increases the prevalence and value of human capabilities that complement technical AI skills, such as analytical thinking, resilience, or ethical judgment, within and beyond AI-intensive job roles. Using a dataset of nearly 30 million job postings from the US, the UK and Australia, between 2018 and 2024, we distinguish between internal effects (within AI roles) and external effects (in non-AI roles) across co
jobs skills