🧙🏼 ChatGPT can see your apps

Howdy, wizards.

Here’s what’s brewing in AI this Friday.

DARIO’S PICKS

1. ChatGPT’s Mac app can soon see what’s on your screen – starting with coding apps

OpenAI is letting ChatGPT see your screen — a feature currently in Beta for MacOS desktop app. They’re starting with coding editors (probably the most straightforward use case), but will expand the range of compatible apps over time.

Work with Apps, as the feature is called, is currently only compatible with the following code editors: VS code*, Xcode and TextEdit, as well as the Terminal.

The new feature is already available to Plus and Team subscribers, with Edu and Enterprise rolling out over the next weeks.

How to enable it:

• Open a compatible app on your computer

• Select it in the Work With Apps button (in the ChatGPT chat bar)

• You’ll be prompted to enable ChatGPT in the MacOS Accessibility permissions for the app

• ChatGPT starts including context from this app in your messages

• *VS code requires an additional extension to work, which you can find here.

‎ Why it matters‎ ‎ This really sets the stage for OpenAI's “Operator” agent coming in January. In its current format, Work with Apps is a convenient way to give ChatGPT context about what the apps you’re working with; but the bigger idea here is that ChatGPT will be able to see whatever you’re working with — and take actions on your behalf. This test will give OpenAI valuable feedback (and training data) to ensure the full agent functionality works as intended.

TOGETHER WITH NEXLA

Struggling with complex data landscapes?

According to Gartner, 86% of companies focus AI efforts on resource automation, yet most projects struggle due to complex data landscapes. Join us for a panel discussion exploring how organizations are overcoming top data challenges and building solid data foundations designed to enable and accelerate impactful AI and GenAI solutions.

Our panelists will dive into their top priorities and share actionable insights as they discuss democratization of data, data foundations that fuel AI, scalable AI infrastructure, and strategies for taking projects from concepts to production.

Save your spot!

DARIO’S PICKS

2. The AI slowdown, explained

AI development is slowing down. Ilya Sutskever just went on the record saying that results from scaling AI models that use vast amounts of unlabelled data to understand language patterns and structures has plateaued. The former chief scientist at OpenAI (one of the “greatest minds of our generation” according to Sam Altman) claims that the age of scaling is over and we’ve entered a new time of exploration and discovery.

Lack of new training data is a big part of it. Current AI models have scraped all the easily accessible (and not so easily accessible ) information on the web, and the effort to use synthetic data as a way to continue improvement has yet to make breakthroughs.

CEOs of big AI companies seemingly disagree. Sam Altman recently posted on X saying simply “there is no wall”. OpenAI's focus on reasoning models like o1, which spend longer time thinking without more training data, is potentially a sound strategy for continuing to increase AI's performance. Anthropic's CEO Dario Amodei recently said he believes the scaling will continue, based on what he’d seen historically, despite lacking proof that this is the case.

‎ Why it matters‎ ‎ If you’ve been concerned about AI safety (honestly, who hasn’t?), these news might be reassuring. There’s a roadblock and it’s likely to slow down the speed of development. Maybe it will give us humans a needed breather to adapt to the new reality of having AI—at its current level of intelligence—all around us.

The terminology here matters, though. “AI development” is a bit of a vague term. For example, while Ilya's argument is echoed by Meta's Yann LeCun, he specifies that Deep Learning, the underlaying foundation of LLMs, is not hitting a wall. Rather, it’s a case of auto-regressive LLMs (the kind that guesses the next token) hitting a performance ceiling.

And the CEOs touting the notion that we haven’t plateaued? They may very well be right, although they’re probably too incentivized by raising money to take their word for it.

FROM OUR PARTNERS

Give your inbox a brain. Ceejay is an assistant that finds any email instantly and manages your calendar through natural conversation. Stop searching, just ask: 'What did Sarah say about the budget?'

Download for iPhone today, and reclaim 2-3 hours of your day.

Try Ceejay AI

THAT’S ALL FOLKS!

Affiliate disclosure: To cover the cost of my email software and the time I spend writing this newsletter, I sometimes link to products and other newsletters. Please assume these are affiliate links. If you choose to subscribe to a newsletter or buy a product through any of my links then THANK YOU – it will make it possible for me to continue to do this.